Graph neural network-based method and apparatus for monitoring computing resources of brain-inspired application

Abstract

The present disclosure relates to the technical field of computers, and provides a graph neural network-based method and apparatus for monitoring computing resources of a brain-inspired application. The method comprises: acquiring model information of a spiking neural network model for implementing a brain-inspired application, wherein the model information is used for indicating the intrinsic features of spiking neurons and the intrinsic features of synaptic connections in the spiking neural network model; inputting the model information into a pre-trained graph neural network model to determine spatial features of the spiking neurons in the spiking neural network model; acquiring a temporal feature of the brain-inspired application; and on the basis of the temporal feature, the spatial features, and a pre-trained resource monitoring model, determining computing resources occupied by the brain-inspired application during running. The spatial features and temporal feature of the brain-inspired application can be extracted so as to calculate the computing resources occupied by the brain-inspired application during running, thereby predicting the computing resources of the brain-inspired application; moreover, the running process of the spiking neurons does not need to be simulated, and therefore, the computing speed of the computing resources can also be increased.

Description

Method and apparatus for monitoring computing resources for neuromorphic applications based on graph neural networks

[0001] This application claims priority to Chinese Patent Application No. 202411215603.3, filed on August 30, 2024, entitled "Method and Apparatus for Monitoring Computing Resources of Brain-Inspired Applications Based on Graph Neural Networks", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for monitoring computing resources for brain-like applications based on graph neural networks. Background Technology

[0003] Neuromorphic applications refer to applications represented by spiking neural networks (SNNs), which consist of numerous spiking neurons and synaptic connections. The overall functionality of a given neuromorphic application is achieved through computation and communication between spiking neurons and synaptic connections within the SNN. SNNs are third-generation artificial neural networks, and their network structure simulates the activity states of neurons and the information transmission methods between neurons in biological neural systems. Unlike traditional neural networks, SNNs consider not only the activation state of spiking neurons but also the temporal information of spiking neuron activation, giving them a significant advantage in processing time-series data and real-time signals. However, due to the computational complexity of neuromorphic applications, monitoring the computational resources of these applications is crucial for allocating and optimizing hardware resources, or for optimizing algorithms and / or hardware configurations based on the monitoring results.

[0004] Currently, methods for monitoring computing resources in neuromorphic applications include the following:

[0005] The first method is to monitor computing resources based on simulation (or simulator). This involves using a specially designed SNN simulator to simulate the operation of a neuromorphic application in detail, thereby capturing the computing resources consumed by the application during runtime. Examples include Neural Simulation Tool (NEST), Brian Neural Network Simulator, or Spiking Neural Network Architecture (SpiNNaker) neural network simulators.

[0006] However, while NEST can accurately simulate the behavior of spiking neurons and synaptic connections, and can be used for large-scale network simulations and provide detailed performance data, NEST has high computational overhead and requires high-performance computing resources. In addition, NEST simulation time is long, especially for large-scale networks, which leads to the problem of high difficulty and low efficiency in monitoring computing resources.

[0007] The Brian neural network simulator has poor simulation accuracy and is not suitable for simulating large-scale networks, which leads to low accuracy and significant limitations in monitoring computing resources.

[0008] The SpiNNaker neural network simulator requires specialized hardware and technical support, which leads to the difficulty and high cost of hardware implementation for monitoring computing resources.

[0009] The second approach involves mathematically analyzing the structure and characteristics of SNNs used in neuromorphic applications to establish performance prediction models that forecast computational resource requirements. These performance prediction models typically consider factors such as the firing patterns of spiking neurons, changes in synaptic connection weights, network topology, and computational resource usage.

[0010] Generally, performance prediction models can be built based on the following models: mean-field theory models, queue theory models, and dynamical system models. Mean-field theory predicts computational resources by assuming that the behavior of spiking neurons tends to average in large-scale networks. This method can derive performance indicators such as the average firing frequency of the network and the average change in synaptic weights, simplifying the network's complexity. Queue theory is often used to analyze the utilization of computational resources, such as processor load and latency. Specifically, a SNN can be viewed as a queue system, and the delays in spiking neuron firing and synaptic signal transmission can be analyzed to assess computational resource utilization. Dynamical system models use differential equations to describe the firing behavior of neurons and the changes in synaptic weights. By solving these differential equations, the dynamic behavior and performance of the SNN can be predicted.

[0011] However, while mean-field theory simplifies the analysis of large-scale networks, its output is an average value, failing to capture the behavioral differences of individual spiking neurons, leading to inaccurate computational resource predictions. Queue theory, with its high model complexity and difficult parameter setting, also presents challenges in computational resource prediction. Similarly, dynamical system models, with their high computational complexity, also face difficulties in computational resource prediction.

[0012] The third approach is based on machine learning, which uses trained models to predict the computational resources required for neuromorphic applications. Specifically, a large amount of training data is used to train the machine learning model to learn the patterns and rules within that data, thereby establishing a performance prediction model to predict the computational resources consumed by neuromorphic applications.

[0013] Currently, using machine learning to predict program performance is very mature in general domains, but no concrete progress has been seen in predicting the computational resource consumption of brain-like applications based on SNNs. The main reason is that SNNs differ significantly from traditional applications in several aspects, which brings many difficulties to the application of the method. Specific differences include:

[0014] First, SNN computation features high concurrency and asynchronous communication. In SNNs, the firing of spiking neurons and the updating of synaptic weights are typically asynchronous, which differs significantly from the synchronous operations in traditional computational models. Traditional applications mostly rely on global clocks and synchronization mechanisms to achieve concurrency, while SNNs need to handle a large number of parallel events.

[0015] Secondly, the processing of temporal information in SNNs increases the complexity of computational resource prediction. SNNs not only need to consider the spatial distribution of spiking neurons but also the temporal characteristics of the pulse signals. This means that in the process of computational resource prediction, both spatial allocation of computational resources and temporal computational delays and time dependencies must be considered. This is quite different from the characteristics of traditional applications, which mainly focus on static data and numerical computation.

[0016] Furthermore, the high degree of customization and diversity of SNNs makes computational resource prediction more difficult. Different SNN models may employ different neuron types, synaptic models, and network structures, all of which affect the computational resources consumed by neuromorphic applications. Traditional applications can often rely on general prediction models, but for SNNs, personalized computational resource predictions must be made based on the specific application scenario and model structure, which increases the complexity and workload of prediction.

[0017] Based on the above differences, traditional machine learning methods are unable to predict the computational resources required for neuromorphic applications.

[0018] Summary of the Invention

[0019] In view of this, this disclosure proposes a method and device for monitoring computing resources of brain-like applications based on graph neural networks. This method can predict the computing resources of brain-like applications. Furthermore, compared with simulation methods for monitoring computing resources, it can improve the computing speed of computing resources because it does not require simulating the operation of spiking neurons.

[0020] According to one aspect of this disclosure, a method for monitoring computing resources for brain-like applications based on graph neural networks is provided, the method comprising:

[0021] Obtain model information of a spiking neural network model that implements the brain-like application; wherein, the spiking neural network model includes multiple spiking neurons and synaptic connections between different spiking neurons, and the model information is used to indicate the inherent characteristics of the spiking neurons and the inherent characteristics of the synaptic connections;

[0022] The model information is input into a pre-trained graph neural network model to determine the spatial features of spiking neurons in the spiking neural network model;

[0023] Obtain the temporal characteristics of the neuromorphic application;

[0024] Based on the temporal features, the spatial features, and the pre-trained resource monitoring model, the computing resources occupied by the neuromorphic application during runtime are determined.

[0025] In one possible implementation, the graph neural network model includes a first graph neural network model and a second graph neural network model; correspondingly, the step of inputting the model information into the pre-trained graph neural network model to determine the spatial features of spiking neurons in the spiking neural network model includes:

[0026] The model information is input into the first graph neural network model to determine the local spatial features of the spiking neuron based on the model information;

[0027] Based on the local spatial features and the second graph neural network model, the global spatial features of the spiking neuron are determined;

[0028] The spatial features include the local spatial features and the global spatial features.

[0029] In one possible implementation, at least one of the first graph neural network model and the second graph neural network model is a graph attention network model.

[0030] In one possible implementation, a Dropout layer is also provided between the first graph neural network model and the second neural network model during the training phase.

[0031] In one possible implementation, the graph neural network model is further connected to a vector merging layer to merge the spatial features of different spiking neurons output by the graph neural network model into a single output representation of spatial features.

[0032] In one possible implementation, obtaining model information of the spiking neural network model implementing the neuromorphic application includes:

[0033] Obtain the intrinsic features of each spiking neuron;

[0034] According to the preset classification and aggregation rules, each spiking neuron is classified and aggregated into at least two neuron groups; wherein each spiking neuron in the same neuron group has at least one common intrinsic feature;

[0035] Generate a node feature matrix based on the feature vector of each neuron group;

[0036] Generate an edge index matrix based on synaptic connections between different neuron groups;

[0037] Generate an edge feature matrix based on the connection features of synaptic connections between different neuron groups;

[0038] The model information includes the node feature matrix, the edge index matrix, and the edge feature matrix.

[0039] In one possible implementation, before classifying and aggregating the individual spiking neurons into at least two neuron groups according to a preset classification and aggregation rule, the method further includes:

[0040] Receive settings instructions;

[0041] Obtain the classification and aggregation rules of the setting operation instructions.

[0042] In one possible implementation, obtaining the temporal characteristics of the neuromorphic application includes:

[0043] Obtain pre-trained time-series feature prediction models corresponding to different hardware platforms;

[0044] Based on the device information of the electronic device currently running the neuromorphic application, the target time-series feature prediction model is determined from various time-series feature prediction models;

[0045] The model structure information and parameter information of the brain-like application are input into the target temporal feature prediction model to obtain the temporal features.

[0046] According to another aspect of this disclosure, a data processing apparatus is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing instructions stored in the memory.

[0047] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided that stores computer program instructions thereon, wherein the computer program instructions, when executed by a processor, implement the above-described method.

[0048] According to another aspect of this disclosure, a computer program product is provided, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0049] This disclosure obtains model information of a spiking neural network model for implementing brain-inspired applications. This model information indicates the inherent characteristics of spiking neurons and synaptic connections in the spiking neural network model. The model information is input into a pre-trained graph neural network model to determine the spatial characteristics of spiking neurons in the spiking neural network model. Temporal characteristics of the brain-inspired application are obtained. Based on the temporal characteristics, spatial characteristics, and a pre-trained resource monitoring model, the computational resources consumed by the brain-inspired application during runtime are determined. This addresses the problem that traditional machine learning methods cannot predict the computational resources consumed by brain-inspired applications. Since the spatial and temporal characteristics of the brain-inspired application can be extracted to calculate the computational resources consumed during runtime, the accuracy of computational resource monitoring can be guaranteed. Furthermore, compared to simulation methods for monitoring computational resources, since it does not require simulating the operation of spiking neurons, the computational speed can be improved.

[0050] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0051] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0052] Figure 1 shows a flowchart of a brain-like application computing resource monitoring method based on graph neural networks according to an embodiment of the present disclosure;

[0053] Figure 2 shows a schematic diagram of a process for monitoring computational resources of a neuromorphic application according to an embodiment of the present disclosure;

[0054] Figure 3 shows a block diagram of a brain-like application computing resource monitoring device based on graph neural networks according to an embodiment of the present disclosure;

[0055] Figure 4 shows a block diagram of a brain-like application computing resource monitoring device based on a graph neural network according to another embodiment of the present disclosure. Detailed Implementation

[0056] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0057] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0058] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0059] First, let me introduce some of the terms used in this application.

[0060] Graph Neural Networks (GNNs) are a class of deep learning models used to process graph-structured data. Graph-structured data refers to data structures composed of nodes and edges. In this application, the graph-structured data includes the model structure data of SNNs (Spiking Neurons). Accordingly, nodes include spiking neurons of the SNN, and edges include synaptic connections between different spiking neurons; or, nodes include neuron groups composed of multiple spiking neurons of the SNN, and edges include synaptic connections between different neuron groups. GNNs can effectively process the information of nodes and edges in graph-structured data, thereby capturing complex graph structure features. GNNs learn node representations by propagating information in the graph. The representation of each node depends not only on its own features but also on the features of its neighboring nodes and their connections.

[0061] For each node v, for its new representation The new representation of node v is updated by aggregating the representations of neighboring nodes and its own representation. This can be expressed by the following formula:

[0062] in, This represents the set of neighboring nodes of node v; u represents a neighboring node of node v in this set of neighboring nodes; AGGREGATE represents the aggregation function used to combine the representations of the node itself and its neighboring nodes; This represents the state when node v has not been updated. σ represents the representation of node u when it is not updated; W represents the weight matrix; σ represents the activation function.

[0063] Common GNN models include Graph Attention Networks (GAT) models. GAT models learn the weight relationships between nodes by introducing an attention mechanism. GAT calculates attention weights between each node and its neighbors to determine the priority of information propagation. This mechanism enables GAT to adaptively aggregate information from important neighbors and adaptively allocate information transfer weights between nodes and their neighbors, improving the model's flexibility and expressiveness.

[0064] For example: for the edge (i,j) connecting node i and node j, the attention score e ij The calculation is as follows:

[0065] Where LeakyReLU represents a variant of the Rectified Linear Unit (ReLU), which is an activation function; || is the vector concatenation operation; W is the linear transformation matrix; h i Represents the representation of node i; h j Represents the representation of node j; This represents the transpose of the weight parameter matrix.

[0066] Then, the attention score e is evaluated using the softmax function. ij Normalization is performed to obtain the attention weight α. ij as follows:

[0067] Where, α ij This represents the attention weight that needs to be multiplied when node j is passed to node i. Let e ​​represent the set of neighboring nodes of node i. ik Let (i,k) represent the attention score of the edge (i,k) connecting node i and its neighbor node k.

[0068] Finally, attention weight α is used. ij For the set of neighboring nodes The weighted average of the values ​​is used to update the node representation as follows:

[0069] Where, h′ i Let α represent the updated representation of node i. ij h represents the attention weight of neighbor node j. j Let W represent the representation of neighbor node j, W represent the weight matrix, and σ represent the activation function.

[0070] The method for monitoring computing resources of brain-like applications based on graph neural network models provided in this application uses graph neural network models to extract spatial features of brain-like applications and combines them with temporal features of brain-like applications to predict the computing resources occupied by brain-like applications through a resource monitoring model, so as to realize the monitoring of computing resources of brain-like applications.

[0071] The following section introduces the brain-like application computing resource monitoring method based on graph neural networks provided in this application.

[0072] Figure 1 shows a flowchart of a brain-like application computing resource monitoring method based on graph neural networks according to an embodiment of the present disclosure. This embodiment illustrates the method in an electronic device with computing capabilities, which can be a user terminal or a server. The user terminal includes, but is not limited to, computers, mobile phones, or tablet computers. This embodiment does not limit the type of electronic device or user terminal. As shown in Figure 1, the method includes:

[0073] Step 101: Obtain model information for the spiking neural network model that implements brain-like applications.

[0074] The spiking neural network model includes multiple spiking neurons and synaptic connections between different spiking neurons. The model information is used to indicate the inherent characteristics of the spiking neurons and the inherent characteristics of the synaptic connections.

[0075] In this application, a neuromorphic application refers to an application implemented based on a spiking neural network model. This neuromorphic application can provide certain functions through the computation process executed by the SNN, such as providing image and speech recognition functions, intelligent chat functions, etc. This embodiment does not limit the specific functions provided by the neuromorphic application.

[0076] In this embodiment, the model information is adapted to the input information of the graph neural network model; in other words, the model information is information that can be processed by the graph neural network model. Schematic, the model information includes the node feature matrix, the edge index matrix, and the edge feature matrix.

[0077] In one example, the node feature matrix includes a feature vector for each spiking neuron. This feature vector includes feature vectors of intrinsic features such as the spiking neuron's membrane potential and / or firing rate. Intrinsic features are characteristics inherent to the spiking neuron itself, typically set by developers during the development of neuromorphic applications; that is, these intrinsic features do not change as the application runs. The membrane potential simulates a potential value within the neuron, representing the spiking neuron's activation state. The firing rate refers to the frequency at which the spiking neuron emits spikes per unit time; in other words, the firing rate is a measure of the spiking neuron's activity, reflecting the number of action potentials generated by the spiking neuron within a given time period.

[0078] Edge index matrix: includes an indication of whether each spiking neuron is connected by an edge.

[0079] Edge feature matrix: This includes feature vectors representing the synaptic features of each edge, such as propagation delay and / or synaptic weights, which are inherent characteristics of synaptic connections. Propagation delay refers to the time required for a signal to travel from one spiking neuron to another through the synapse; synaptic weights indicate the degree to which the synapse amplifies or inhibits the signal.

[0080] Since the SNN model for brain-like applications may be implemented using millions of spiking neurons, constructing model information based on the inherent characteristics of each spiking neuron and each synaptic connection would result in an excessively large scale of model information, leading to the problem of significant difficulty in spatial feature calculation.

[0081] Based on the above issues, in another example, the model information of the spiking neural network model for implementing brain-like applications is obtained, including: obtaining the inherent features of each spiking neuron; classifying and aggregating each spiking neuron into at least two neuron groups according to a preset classification and aggregation rule; generating a node feature matrix based on the feature vector of each neuron group; generating an edge index matrix based on the synaptic connections between different neuron groups; and generating an edge feature matrix based on the connection features of the synaptic connections between different neuron groups.

[0082] Among them, each spiking neuron in the same neuronal group has at least one common intrinsic feature.

[0083] Optionally, the types of inherent features indicated by the classification aggregation rules may be the same as or different from the types of feature vectors of the spiking neurons. For example, inherent features may also include the distribution location of spiking neurons in the model (in this case, the types of inherent features are different from the types of feature vectors of spiking neurons); or, for example, inherent features may include the firing rate of spiking neurons (in this case, the types of inherent features are the same as the types of feature vectors of spiking neurons). This embodiment does not limit the types of inherent features.

[0084] Schematic illustration: Classification aggregation rules are used to indicate the inherent features used in classification aggregation. For example, if the inherent feature indicated by the classification aggregation rule is the distribution location of spiking neurons in the model, then the electronic device will classify and aggregate spiking neurons with the same distribution location into the same neuron group. Another example: if the inherent feature indicated by the classification aggregation rule is the firing rate, then the electronic device will classify and aggregate spiking neurons with the same firing rate into the same neuron group. This embodiment does not limit the type of inherent feature indicated by the classification aggregation rule.

[0085] In one example, the classification aggregation rules are pre-existing in the electronic device, and the inherent characteristics indicated by the classification aggregation rules remain unchanged.

[0086] In another example, the classification and aggregation rules are set based on user requirements. In this case, before classifying and aggregating each spiking neuron into at least two neuron groups according to the preset classification and aggregation rules, the process includes: receiving a setting operation; and obtaining the classification and aggregation rules indicated by the setting operation. Because the classification and aggregation rules can be set based on user requirements, it is possible to classify and aggregate spiking neurons according to user needs, thus improving the flexibility of classification and aggregation.

[0087] Electronic devices receive setting operations through human-computer interaction controls, which are used to set category aggregation rules.

[0088] Optionally, the electronic device classifies and aggregates each spiking neuron into at least two neuron groups based on a classification and aggregation algorithm. This classification and aggregation algorithm can be the K-Nearest Neighbors (KNN) algorithm or the random forest algorithm, etc. This embodiment does not limit the implementation of the classification and aggregation algorithm.

[0089] In one example, after classification and aggregation, the node feature matrix includes: the feature vector of each neuron group, which includes, but is not limited to: statistical features such as the average membrane potential and / or average firing rate of each spiking neuron in each neuron group. Since these statistical features are determined based on the inherent features of each spiking neuron, the node feature matrix is ​​used to indicate the inherent features of each spiking neuron in each neuron group.

[0090] The edge index matrix includes: indices of synaptic connections between different groups of neurons;

[0091] The edge feature matrix includes feature vectors of synaptic connections between different neuron groups. Since the synaptic connections between different neuron groups are determined based on the inherent features of the synaptic connections between spiking neurons in different neuron groups, the edge feature matrix is ​​used to indicate the inherent features of the synaptic connections between spiking neurons in different neuron groups.

[0092] Optionally, the model information is determined based on the organizational information set by the developers when developing the brain-inspired application. For example, the brain-inspired application uses a development framework to describe its organizational information, which includes the feature vector of each spiking neuron, whether each spiking neuron is connected by an edge, and the synaptic features of each edge. In this case, the organizational information of the development framework is read to obtain the inherent features of the spiking neurons and synaptic connections of the brain-inspired application to determine the model information. The development framework includes, but is not limited to, SNN development frameworks (e.g., the PyNN framework) or PyTorch-based deep SNN (i.e., D-SNN) development frameworks (e.g., the SpikingJelly framework, the Norse framework), etc.

[0093] Step 102: Input the model information into the pre-trained graph neural network model to determine the spatial characteristics of spiking neurons in the spiking neural network model.

[0094] In this embodiment, the graph neural network model is used to update the model information to the spatial features of spiking neurons in the SNN. Specifically, the node features of the graph neural network model are initialized based on the node feature matrix in the model information. Then, the graph neural network model combines the edge index matrix and the edge feature matrix to update the node features of each node to obtain the spatial features.

[0095] The spatial features of spiking neurons are used to indicate their location and connectivity within the SNN. Optionally, if the node feature matrix in the model information is the feature vector of each neuron group, then the spatial features of a spiking neuron are represented by the spatial features of the neuron group to which that spiking neuron belongs.

[0096] The graph neural network model is pre-trained based on training data. The specific training process is detailed in the following embodiment, which will not be elaborated further here.

[0097] In one example, an electronic device can determine the spatial features of spiking neurons using a graph neural network model. However, according to the computational principles of graph neural network models, this model can only capture the spatial features between two adjacent nodes, i.e., local spatial features.

[0098] Based on this, in another example, the graph neural network model includes a first graph neural network model and a second graph neural network model. Accordingly, inputting model information into the pre-trained graph neural network model to determine the spatial features of spiking neurons in the spiking neural network model includes: inputting model information into the first graph neural network model to determine the local spatial features of spiking neurons based on the model information; and determining the global spatial features of spiking neurons based on the local spatial features and the second graph neural network model. In this case, the spatial features include both local and global spatial features. In this embodiment, the spatial features output by the graph neural network model include both the local and global spatial features of spiking neurons.

[0099] In this embodiment, by setting a second neural network model after the first neural network model, the second neural network model can further obtain the features of more distant neighbor nodes based on the local spatial features output by the first neural network model, thereby capturing global spatial features.

[0100] In addition, in order to enable the second graph neural network model to calculate global spatial features, the edge index matrix and edge feature matrix in the model information will also be input into the second graph neural network model.

[0101] Optionally, to further improve the accuracy of the graph neural network model in determining spatial features, at least one of the first graph neural network model and the second graph neural network model is a graph attention network model.

[0102] In one example, both the first and second graph neural network models are implemented as graph attention network models. They have the same model structure, but their input information differs. The input information for the first graph neural network model is model information, while the input information for the second graph neural network model is features derived from the local spatial features output by the first graph neural network model. Furthermore, the attention weights and information aggregation levels of the first and second graph neural network models differ, allowing them to extract local and global spatial features respectively.

[0103] In one implementation, the local spatial features output by the first graph neural network model are directly input into the second graph neural network model to determine the global spatial features of the spiking neurons.

[0104] In another implementation, the local spatial features output by the first graph neural network model are processed by other network layers and then input into the second graph neural network model to determine the global spatial features of the spiking neurons.

[0105] For example, a Dropout layer is set between the first and second graph neural network models during the training phase. In this embodiment, a certain proportion of neurons in the graph neural network model are randomly dropped by the Dropout layer during training, so that the model does not overly rely on certain specific neurons. This can prevent overfitting during model training and improve the model training effect.

[0106] The Dropout layer can be used in the actual prediction process of the graph neural network model, or it can be deleted in the actual prediction process of the graph neural network model. This embodiment does not limit the use of the Dropout layer in the actual prediction process.

[0107] If the Dropout layer is used in the actual prediction process of the graph neural network model, the local spatial features are processed by the Dropout layer and then input into the second graph neural network model.

[0108] For example, an activation layer is placed between the first and second neural network models to introduce non-linear features and improve model performance. In this case, the local spatial features are processed by the activation layer and then input into the second neural network model.

[0109] In other embodiments, the first neural network model and the second neural network model may also be configured with other network layers to improve model performance, which will not be listed here.

[0110] In other embodiments, the number of graph neural network models can be set to more as needed. In this case, the aforementioned Dropout layer and / or activation layer can be set between two adjacent graph neural network models. This embodiment does not limit the number of graph neural network models.

[0111] Optionally, the graph neural network model is further connected to a vector merging layer to merge the spatial features of different spiking neurons output by the graph neural network model into a single output representation of spatial features. Illustratively, the vector merging layer can be a Reduce layer; in other embodiments, the vector merging layer can also be a network layer with the same function. This embodiment does not limit the implementation method of the vector merging layer.

[0112] Step 103: Obtain the temporal characteristics of the neuromorphic application.

[0113] Temporal features are used to indicate the timing information of the pulses generated by spiking neurons.

[0114] In one example, obtaining the temporal features of a neuromorphic application includes: obtaining pre-trained temporal feature prediction models corresponding to different hardware platforms; determining the target temporal feature prediction model from among the various temporal feature prediction models based on the device information of the electronic device currently running the neuromorphic application; and inputting the model structure information and parameter information of the neuromorphic application into the target temporal feature prediction model to obtain the temporal features.

[0115] Since different hardware platforms generate different temporal features when running neuromorphic applications, this embodiment pre-trains temporal feature prediction models corresponding to different hardware platforms and selects the target temporal feature prediction model corresponding to the electronic device currently running the neuromorphic application to determine the temporal features of the neuromorphic application, which can ensure the accuracy of the obtained temporal features.

[0116] Optionally, the time-series feature prediction model is trained based on a regression model or a neural network model. This embodiment does not limit the construction method of the time-series feature prediction model. The training process of the time-series feature prediction model includes: for each hardware platform, acquiring training data when the hardware platform runs different neuromorphic applications. The training data includes the sample model structure information and sample parameter information of each neuromorphic application, as well as the time-series feature label corresponding to the neuromorphic application; iteratively training a pre-created mathematical model based on the training data to obtain the time-series feature prediction model. This training process is similar to the training process of traditional neural networks, and will not be described in detail here.

[0117] Schematic, based on the device information of the electronic device currently running the neuromorphic application, the target time-series feature prediction model is determined from various time-series feature prediction models, including: obtaining the platform information of the hardware platform corresponding to each time-series feature prediction model; matching the device information with the platform information, and determining the time-series feature prediction model corresponding to the platform information that is consistent with the device information as the target time-series feature prediction model.

[0118] To illustrate, the platform information and device information are of the same type, such as GPU model and / or number of GPUs. This embodiment does not limit the types of platform information and device information.

[0119] In one example, model structure information defines the SNN's model framework and the organization of individual spiking neurons. This structure determines how the SNN processes input data, transmits information, and generates output results. For example, model structure information includes the hierarchical structure between different spiking neurons and the synaptic connections. Parameter information defines the behavior of each spiking neuron and synaptic connection in the SNN; for example, parameters include the membrane potential and firing threshold of each spiking neuron.

[0120] Schematic, the methods for obtaining model structure information and parameter information include: determining model structure information and parameter information based on model information, for example: if the model information includes the inherent characteristics of each spiking neuron and the inherent characteristics of synaptic connections between different spiking neurons, then the model structure information may include the inherent characteristics of synaptic connections, and the parameter information may include the inherent characteristics of each spiking neuron; or, obtaining model structure information and parameter information by reading and parsing the organizational information of the brain-like application. This embodiment does not limit the method of obtaining model structure information and parameter information.

[0121] Optionally, step 103 can be executed after step 102, or before step 102, or simultaneously with step 102. This embodiment does not limit the execution order between steps 103 and 102.

[0122] Step 104: Based on temporal features, spatial features, and a pre-trained resource monitoring model, determine the computing resources occupied by the neuromorphic application during runtime.

[0123] The computing resources occupied by the neuromorphic application during runtime include, but are not limited to, parameters such as runtime, CPU usage, and / or memory access usage. This embodiment does not limit the type of computing resources.

[0124] In one example, the temporal features and spatial features of each spiking neuron are concatenated and input into a pre-trained resource monitoring model to calculate the computing resources used by the brain-like application during runtime.

[0125] In another example, the temporal features are concatenated with the spatial features merged by the vector merging layer and then input into a pre-trained resource monitoring model to calculate the computing resources used by the neuromorphic application during runtime.

[0126] Optionally, the resource monitoring model can be trained based on a traditional neural network model, such as a multilayer perceptron (MLP). In other embodiments, the resource monitoring model can also be trained based on other neural networks, such as a recurrent neural network (RNN) or a long short-term memory (LSTM). This embodiment does not limit the construction method of the resource monitoring model.

[0127] Optionally, after obtaining the computing resources occupied by the neuromorphic application during runtime, the electronic device may output the prediction results of those computing resources in response to an output command. And / or, the electronic device may allocate hardware resources based on those computing resources to avoid resource waste. And / or, the electronic device may output a first optimization suggestion for hardware configuration based on those computing resources, the first optimization suggestion being used to prompt the user to optimize the hardware configuration. And / or, the electronic device may output a second optimization suggestion for the neuromorphic application based on those computing resources, the second optimization suggestion being used to prompt developers to adjust the parameters of the neuromorphic application to improve its performance.

[0128] In summary, the graph neural network-based method for monitoring computational resources for brain-inspired applications provided in this embodiment acquires model information of the spiking neural network model implementing the brain-inspired application. This model information indicates the inherent characteristics of spiking neurons and synaptic connections in the spiking neural network model. The model information is then input into a pre-trained graph neural network model to determine the spatial characteristics of spiking neurons in the spiking neural network model. Temporal characteristics of the brain-inspired application are obtained. Based on these temporal characteristics, spatial characteristics, and the pre-trained resource monitoring model, the computational resources consumed by the brain-inspired application during runtime are determined. This method solves the problem that traditional machine learning methods cannot predict the computational resources consumed by brain-inspired applications. Because the spatial and temporal characteristics of the brain-inspired application can be extracted to calculate the computational resources consumed during runtime, the accuracy of computational resource monitoring can be guaranteed. Furthermore, compared to simulation methods for monitoring computational resources, since it does not require simulating the operation of spiking neurons, it can also improve the computational speed.

[0129] Furthermore, by setting up a second neural network model after the first neural network model, the second neural network model can further obtain the features of more distant neighbor nodes based on the local spatial features output by the first neural network model, thereby capturing global spatial features and improving the accuracy of spatial feature extraction.

[0130] Furthermore, by setting at least one of the first graph neural network model and the second graph neural network model as a graph attention network model, the accuracy of the graph neural network model in extracting spatial features can be further improved.

[0131] In addition, a Dropout layer is set between the first and second graph neural network models during the training phase. During training, a certain proportion of neurons in the graph neural network model are randomly dropped by the Dropout layer, so that the model does not over-rely on certain specific neurons. This can prevent overfitting during model training and improve the model training effect.

[0132] In addition, by classifying and aggregating spiking neurons into at least two neuron groups, and generating a node feature matrix based on the feature vectors of the neuron groups to calculate spatial features, the amount of data in the node feature matrix can be reduced, thereby improving the computational efficiency of spatial features.

[0133] In addition, by obtaining classification and aggregation rules based on the settings, the flexibility of classification and aggregation of spiking neurons can be improved.

[0134] To better understand the brain-inspired application computing resource monitoring method provided in this application, an example is given below. Referring to Figure 2, for any brain-inspired application 201, the electronic device acquires the model information, model structure information, and parameter information of the brain-inspired application 201. The model information includes a node feature matrix obtained based on the feature vectors of the neuron groups after classifying and aggregating spiking neurons to obtain neuron groups; an edge index matrix obtained based on the indexes of synaptic connections between different neuron groups; and an edge feature matrix obtained based on the connection features of synaptic connections between different neuron groups.

[0135] The model information is input into the first graph neural network model 202 to obtain local spatial features. These local spatial features are processed by the Rectified Linear Unit (ReLU) activation function 203 and then output to the second graph neural network model 205 via the Dropout layer 204. The second graph neural network model 205 calculates the local and global spatial features of the spiking neurons based on the features output by the Dropout layer 204. Then, the local and global spatial features of each spiking neuron are merged through the Reduce layer 206 to obtain the spatial features.

[0136] Input the model structure information and parameter information into the target time series feature prediction model 207 corresponding to the currently running electronic device to obtain the time series features.

[0137] Spatial and temporal features are concatenated and input into resource monitoring model 208 to obtain the computing resources occupied by the brain-like application during runtime.

[0138] In this embodiment, local and global spatial features of the neuromorphic application can be extracted to predict the computing resources occupied by the neuromorphic application during runtime, which can improve the accuracy of prediction.

[0139] In one example, the training process of the graph neural network model and resource monitoring model in the above embodiments includes: acquiring training data, which includes sample model information and the computational resource labels corresponding to the sample model information; inputting the sample model information into the graph neural network model to obtain a first result; inputting the model structure information and parameter information corresponding to the sample model information into a pre-trained temporal feature prediction model to obtain a second result; merging the first result and the second result and inputting them into a pre-created resource monitoring model to obtain a third result; comparing the third result with the computational resource labels, and iteratively training the graph neural network model and resource monitoring model according to the comparison result to obtain the trained graph neural network model and resource monitoring model.

[0140] In other embodiments, the graph neural network model and the resource monitoring model can also be trained separately. That is, the graph neural network model is trained using sample model information and the spatial feature labels corresponding to the sample model information, and the resource monitoring model is trained using sample temporal features, sample spatial features and corresponding computing resource labels. This embodiment does not limit the training methods of the graph neural network model and the resource monitoring model.

[0141] Optionally, during the training of the graph neural network model and the resource monitoring model, 70% of the original dataset can be used for training to obtain training data, 15% for validation to obtain validation data, and the remaining 15% for testing to obtain test data. An adaptive moment estimation (Adam) optimizer with a learning rate of 0.001 can be used to assist training. In other embodiments, the dataset partitioning method, the value of the learning rate, and the optimization algorithm can also be implemented in other ways, and this embodiment does not limit them.

[0142] The accuracy and efficiency of the brain-inspired application computing resource monitoring method provided in this application are explained below. Assume that by changing the spiking neuron parameters of monkey brain simulation programs, human brain simulation programs, and mouse brain simulation programs, multiple brain-inspired applications are obtained. These applications are run on GPUs of NVIDIA GPU 4090, A100, and P100 on different hardware platforms. The parameters of computing resources occupied by the brain-inspired applications on each hardware platform include runtime, CPU usage percentage, and memory access percentage.

[0143] Test results show that the average relative error rate of computing resources obtained through the above-mentioned computing resource monitoring method is 5.58%, which is close to the average error rate of 5.3% for predicting the computing resources occupied by traditional applications (i.e., non-neuro-inspired applications) using the general GraphCodeBERT network. In other words, the accuracy of the computing resource prediction for neuro-inspired applications provided in this application has reached the level of prediction accuracy in the general computing field, thus guaranteeing the accuracy of predicting the computing resources occupied by neuro-inspired applications.

[0144] Meanwhile, compared with simulation-based methods for monitoring computing resources, the resource monitoring method provided in this application can achieve a computing speed 200 times faster than traditional simulation methods, saving computing time and improving the processing efficiency of downstream tasks.

[0145] Figure 3 is a block diagram of a brain-like application computing resource monitoring device based on graph neural networks according to an embodiment of this application. The device includes: a model acquisition module 310, a first extraction module 320, a second extraction module 330, and a resource monitoring module 340.

[0146] The model acquisition module 310 is used to acquire model information of the spiking neural network model that implements the brain-like application; wherein, the spiking neural network model includes multiple spiking neurons and synaptic connections between different spiking neurons, and the model information is used to indicate the inherent characteristics of the spiking neurons and the inherent characteristics of the synaptic connections;

[0147] The first extraction module 320 is used to input the model information into a pre-trained graph neural network model to determine the spatial features of spiking neurons in the spiking neural network model;

[0148] The second extraction module 330 is used to obtain the temporal features of the neuromorphic application;

[0149] The resource monitoring module 340 is used to determine the computing resources occupied by the neuromorphic application during runtime based on the temporal features, the spatial features, and the pre-trained resource monitoring model.

[0150] For relevant details, please refer to the above method implementation examples.

[0151] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0152] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium can be volatile or non-volatile.

[0153] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing the instructions stored in the memory. An example of the electronic device can be seen in Figure 4.

[0154] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0155] Figure 4 is a block diagram of a brain-like application computing resource monitoring device or statistical information acquisition device based on graph neural networks according to an embodiment of this application. For example, device 1900 can be provided as a server or terminal device. Referring to Figure 4, device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions executable by processing component 1922, such as application programs. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, processing component 1922 is configured to execute instructions to perform the methods described above.

[0156] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output interface 1958 (I / O interface). Device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM Mac OS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0157] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of the device 1900 to perform the above-described method.

[0158] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for monitoring computing resources for brain-like applications based on graph neural networks, characterized in that, The method includes: Obtain model information of a spiking neural network model for implementing the brain-like application; wherein, the spiking neural network model includes multiple spiking neurons and synaptic connections between different spiking neurons, and the model information is used to indicate the inherent characteristics of the spiking neurons and the inherent characteristics of the synaptic connections; The model information is input into a pre-trained graph neural network model to determine the spatial features of spiking neurons in the spiking neural network model; Obtain the temporal characteristics of the neuromorphic application; Based on the temporal features, the spatial features, and the pre-trained resource monitoring model, the computing resources occupied by the neuromorphic application during runtime are determined.

2. The method according to claim 1, characterized in that, The graph neural network model includes a first graph neural network model and a second graph neural network model; correspondingly, the step of inputting the model information into the pre-trained graph neural network model to determine the spatial features of spiking neurons in the spiking neural network model includes: The model information is input into the first graph neural network model to determine the local spatial features of the spiking neuron based on the model information; Based on the local spatial features and the second graph neural network model, the global spatial features of the spiking neuron are determined; The spatial features include the local spatial features and the global spatial features.

3. The method according to claim 2, characterized in that, At least one of the first graph neural network model and the second graph neural network model is a graph attention network model.

4. The method according to claim 2, characterized in that, During the training phase, a Dropout layer is also set between the first neural network model and the second neural network model.

5. The method according to claim 1, characterized in that, The graph neural network model is also connected to a vector merging layer to merge the spatial features of different spiking neurons output by the graph neural network model into a single output representation of spatial features.

6. The method according to any one of claims 1 to 5, characterized in that, The step of obtaining model information for the spiking neural network model that implements the neuromorphic application includes: Obtain the intrinsic features of each spiking neuron; According to the preset classification and aggregation rules, each spiking neuron is classified and aggregated into at least two neuron groups; wherein each spiking neuron in the same neuron group has at least one common intrinsic feature; Generate a node feature matrix based on the feature vector of each neuron group; Generate an edge index matrix based on synaptic connections between different neuron groups; Generate an edge feature matrix based on the connection features of synaptic connections between different neuron groups; The model information includes the node feature matrix, the edge index matrix, and the edge feature matrix.

7. The method according to claim 6, characterized in that, Before classifying and aggregating each spiking neuron into at least two neuron groups according to a preset classification and aggregation rule, the method further includes: Receive settings instructions; Obtain the classification and aggregation rules of the setting operation instructions.

8. The method according to any one of claims 1 to 5, characterized in that, The acquisition of the temporal features of the neuromorphic application includes: Obtain pre-trained time-series feature prediction models corresponding to different hardware platforms; Based on the device information of the electronic device currently running the neuromorphic application, the target time-series feature prediction model is determined from various time-series feature prediction models; The model structure information and parameter information of the brain-like application are input into the target temporal feature prediction model to obtain the temporal features.

9. A data processing apparatus, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the brain-like application computing resource monitoring method based on graph neural networks as described in any one of claims 1 to 8 when executing instructions stored in the memory.

10. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the brain-like application computing resource monitoring method based on graph neural networks as described in any one of claims 1 to 8.

Images