Brain-like application computing resource monitoring method and device based on graph neural network
The spatial and timing characteristics of brain-like applications are extracted by a graph neural network method, and the problem of low monitoring accuracy and efficiency of brain-like applications in the existing technology is solved, and accurate prediction and efficient monitoring of SNN computing resources are achieved.
Patent Information
- Application Number
- CN202411215603.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-06
AI Technical Summary
The existing brain-like application computing resource monitoring methods have problems of low accuracy and low efficiency, especially the inability to effectively predict the computing resource occupation of SNN based on simulation and traditional machine learning methods, and the simulation methods require high-performance computing resources and long-term simulation. This makes monitoring difficult and costly.
The graph neural network-based method is adopted to obtain the model information of the pulse neural network model, extract the spatial and timing characteristics of the pulse neurons, and combine the pre-trained resource monitoring model to predict the computing resource occupation during the running of brain-like applications.
It improves the accuracy and efficiency of computing resource monitoring, avoids the operation process of simulated pulsed neurons in simulation methods, thereby accelerating the monitoring of computing resources and achieving accurate prediction of brain-like application computing resources.
Smart Images

Figure CN119089985A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for monitoring computing resources of brain-like applications based on graph neural networks. Background Art
[0002] Brain-like applications refer to applications represented by spiking neural networks (SNNs) composed of several spiking neurons and synaptic connections. In SNNs, the overall function of a given brain-like application is realized through the calculation and communication of spiking neurons and synaptic connections. Among them, SNN is the third generation of artificial neural networks. The network structure of SNN simulates the activity state of neurons and the way information is transmitted between neurons in biological nervous systems. SNN is different from traditional neural networks. SNN not only considers the activation state of spiking neurons, but also attaches importance to the time information of spiking neuron activation, which makes SNN have significant advantages in processing time series data and real-time signals. However, since the calculation of brain-like applications is relatively complex, it is particularly important to monitor the computing resources of brain-like applications, allocate and optimize hardware resources according to the monitoring results, or optimize algorithms and / or hardware configurations.
[0003] At present, the computing resource monitoring methods for brain-like applications include the following:
[0004] The first method is to implement computing resource monitoring based on simulation (or simulator). That is, the operation process of brain-like applications is simulated in detail through a specially designed SNN simulator, so as to capture the computing resources occupied by brain-like applications during operation. For example: through the Neural Simulation Tool (NEST), Brian Neural Network Simulator or Spiking Neural Network Architecture (SpiNNaker) Neural Network Simulator.
[0005] However, although NEST can accurately simulate the behavior of spiking neurons and synaptic connections, can be used for simulation of large-scale networks and can provide detailed performance data, NEST has a large computing overhead and requires high-performance computing resources. At the same time, NEST simulation time is relatively long, especially for large-scale networks, which leads to the problem of high difficulty and low efficiency in computing resource monitoring.
[0006] The simulation accuracy of the Brian neural network simulator is poor and it is not suitable for simulating large-scale networks, which leads to low accuracy and greater limitations in computing resource monitoring.
[0007] The SpiNNaker neural network simulator requires specialized hardware and technical support, which leads to the problem that the hardware implementation of computing resource monitoring is difficult and costly.
[0008] The second method is to mathematically analyze the structure and characteristics of SNNs for brain-like applications and establish performance prediction models to predict the computing resources of brain-like applications. These performance prediction models are usually established by considering the discharge rules of spiking neurons, the weight changes of synaptic connections, the topological structure of the network, and the use of computing resources.
[0009] Generally, performance prediction models can be established based on the following models: models based on mean field theory, models based on queue theory, and dynamic system models. Among them, mean field theory predicts computing resources by assuming that the behavior of spiking neurons tends to average in large-scale networks. Through this method, performance indicators such as the average discharge frequency of the network and the average change of synaptic weights can be derived to simplify the complexity of the network. Queuing theory is often used to analyze the use of computing resources, such as processor load and waiting time. Specifically, SNN can be regarded as a queue system to analyze the delay of spiking neuron discharge and synaptic connection signal transmission to analyze the use of computing resources. The dynamic system model uses differential equations to describe the discharge behavior of neurons and the weight change of synapses. By solving these differential equations, the dynamic behavior and performance of SNN can be predicted.
[0010] However, although mean field theory can simplify the analysis of large-scale networks, the output result is an average value, which cannot capture the behavioral differences of individual spiking neurons, resulting in inaccurate prediction of computing resources. The model of queue theory is highly complex and the parameters are difficult to set, which makes it difficult to predict computing resources. The computational complexity of the dynamic system model is high, which also makes it difficult to predict computing resources.
[0011] The third method is to use machine learning to predict the computing resources of brain-like applications through training models. Specifically, a large amount of training data is used to enable the machine learning model to learn the patterns and rules in the training data, and a performance prediction model is established to predict the computing resources occupied by brain-like applications.
[0012] At present, the use of machine learning to predict program performance is very mature in general fields, but no specific progress has been seen in predicting the computing resources occupied by brain-like applications based on SNN. The main reason is that SNN and traditional applications have significant differences in many aspects. These differences have brought many difficulties in the application of methods. The specific differences include:
[0013] First, SNN computing has high concurrency and asynchronous communication characteristics. In SNN, the pulse emission of spiking neurons and the weight update of synaptic connections are usually performed asynchronously, which is very different from the synchronous operation in the computing model of traditional applications. Traditional applications mostly rely on global clocks and synchronization mechanisms to achieve concurrency, while SNNs need to handle a large number of parallel events.
[0014] Secondly, the processing of timing information in SNN increases the complexity of computing resource prediction. SNN needs to consider not only the spatial distribution of spiking neurons, but also the temporal characteristics of spiking signals. This means that in the process of computing resource prediction, it is necessary to consider both the spatial allocation of computing resources and the temporal computing delay and timing dependency. This is completely different from the characteristics of traditional applications that mainly focus on static data and numerical calculations.
[0015] In addition, the high degree of customization and diversification of SNNs also makes it more difficult to predict computing resources. Different SNN models may use different neuron types, synaptic models, and network structures, all of which will affect the computing resources occupied by brain-like applications. Traditional applications can often rely on general prediction models, but for SNNs, personalized computing resource predictions must be made based on specific application scenarios and model structures, which increases the complexity and workload of predictions.
[0016] Based on the above differences, traditional machine learning methods are unable to predict the computing resources of brain-like applications. Summary of the invention
[0017] In view of this, the present disclosure proposes a method and device for monitoring computing resources of brain-like applications based on graph neural networks, which can realize the prediction of computing resources for brain-like applications. At the same time, compared with the simulation method to monitor computing resources, since there is no need to simulate the operation process of pulse neurons, the computing speed of computing resources can also be improved.
[0018] According to one aspect of the present disclosure, a method for monitoring computing resources of brain-like applications based on a graph neural network is provided, the method comprising:
[0019] Acquiring model information of a spiking neural network model for implementing the brain-like application; wherein the spiking neural network model includes a plurality of spiking neurons and synaptic connections between different spiking neurons, and the model information is used to indicate inherent characteristics of the spiking neurons and inherent characteristics of the synaptic connections;
[0020] Inputting the model information into a pre-trained graph neural network model to determine the spatial characteristics of spiking neurons in the spiking neural network model;
[0021] Acquiring the timing characteristics of the brain-like application;
[0022] Based on the temporal features, the spatial features and a pre-trained resource monitoring model, the computing resources occupied by the brain-inspired application when it is running are determined.
[0023] In a possible implementation, the graph neural network model includes a first graph neural network model and a second graph neural network model; accordingly, inputting the model information into a pre-trained graph neural network model to determine the spatial characteristics of spiking neurons in the spiking neural network model includes:
[0024] Inputting the model information into the first graph neural network model to determine the local spatial features of the spiking neuron based on the model information;
[0025] Determining a global spatial feature of the spiking neuron based on the local spatial feature and the second graph neural network model;
[0026] The spatial features include the local spatial features and the global spatial features.
[0027] In one possible implementation, at least one of the first graph neural network model and the second graph neural network model is a graph attention network model.
[0028] In a possible implementation, a Dropout layer is further provided between the first graph neural network model and the second graph neural network model during the training phase.
[0029] In one possible implementation, the graph neural network model is also connected to a vector merging layer to merge the spatial features of different pulse neurons output by the graph neural network model into spatial features represented by a single output.
[0030] In a possible implementation, the acquiring model information of the spiking neural network model for implementing the brain-like application includes:
[0031] Capture the intrinsic characteristics of each spiking neuron;
[0032] According to a preset classification and aggregation rule, each spiking neuron is classified and aggregated into at least two neuron groups; wherein each spiking neuron in the same neuron group has at least one same inherent characteristic;
[0033] Generate a node feature matrix based on the feature vector of each neuron group;
[0034] Generate an edge index matrix based on the synaptic connections between different neuronal groups;
[0035] Generate an edge feature matrix based on the connection features of synaptic connections between different neuron groups;
[0036] The model information includes the node feature matrix, the edge index matrix and the edge feature matrix.
[0037] In a possible implementation, before classifying and aggregating each spiking neuron into at least two neuron groups according to a preset classification and aggregation rule, the method further includes:
[0038] Receive setup operations;
[0039] Obtaining a classification aggregation rule for the setting operation indication.
[0040] In a possible implementation, the acquiring the timing characteristics of the brain-inspired application includes:
[0041] Obtain pre-trained time series feature prediction models corresponding to different hardware platforms;
[0042] Determining a target timing feature prediction model from various timing feature prediction models based on device information of the electronic device on which the brain-inspired application is currently running;
[0043] The model structure information and parameter information of the brain-like application are input into the target time series feature prediction model to obtain the time series feature.
[0044] According to another aspect of the present disclosure, there is provided a data processing device, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.
[0045] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.
[0046] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.
[0047] The present disclosure obtains model information of a pulse neural network model for implementing brain-like applications, and the model information is used to indicate the inherent characteristics of pulse neurons and the inherent characteristics of synaptic connections in the pulse neural network model; the model information is input into a pre-trained graph neural network model to determine the spatial characteristics of pulse neurons in the pulse neural network model; the timing characteristics of the brain-like application are obtained; based on the timing characteristics, spatial characteristics and a pre-trained resource monitoring model, the computing resources occupied by the brain-like application during operation are determined; the problem that traditional machine learning methods cannot predict the computing resources of brain-like applications can be solved; since the spatial characteristics and timing characteristics of brain-like applications can be extracted to calculate the computing resources occupied by brain-like applications during operation, the accuracy of computing resource monitoring can be guaranteed. At the same time, compared with the simulation method to realize the monitoring of computing resources, since there is no need to simulate the operation process of pulse neurons, the calculation speed of computing resources can also be improved.
[0048] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.
[0050] Figure 1 A flowchart of a method for monitoring computing resources of brain-like applications based on a graph neural network according to an embodiment of the present disclosure is shown;
[0051] Figure 2 A schematic diagram showing a process of monitoring brain-like application computing resources according to an embodiment of the present disclosure;
[0052] Figure 3 A block diagram of a brain-like application computing resource monitoring device based on a graph neural network according to an embodiment of the present disclosure is shown;
[0053] Figure 4 A block diagram of a brain-like application computing resource monitoring device based on a graph neural network according to another embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0054] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.
[0055] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
[0056] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.
[0057] First, several terms involved in this application are introduced.
[0058] Graph Neural Network (GNN) is a type of deep learning model for processing graph structure data. Graph structure data refers to a data structure composed of nodes and edges. In this application, graph structure data includes model structure data of SNN. Accordingly, nodes include pulse neurons of SNN, and edges include synaptic connections between different pulse neurons; or, nodes include neuron groups composed of multiple pulse neurons of SNN, and edges include synaptic connections between different neuron groups. GNN can effectively process the information of nodes and edges in graph structure data, thereby capturing complex graph structure features. GNN learns the representation of nodes by propagating information in the graph. The representation of each node depends not only on its own characteristics, but also on the characteristics of its neighboring nodes and their connection relationships.
[0059] For each node v, for its new representation The new representation of node v is updated by aggregating the representations of neighboring nodes and its own representation. It is expressed by the following formula:
[0060]
[0061] in, represents the set of neighbor nodes of node v; u represents a neighbor node of node v in the set of neighbor nodes; AGGREGATE represents an aggregation function, which is used to merge the representation of the node itself and the neighbor nodes; Represents the representation when node v is not updated; represents the representation of node u when it is not updated; W represents the weight matrix; σ represents the activation function.
[0062] Common GNN models include the Graph Attention Networks (GAT) model. The GAT model learns the weight relationship between nodes by introducing an attention mechanism. GAT calculates attention weights between each node and its neighbors to determine the priority of information propagation. This mechanism enables GAT to adaptively aggregate the information of important neighbors and adaptively allocate the information transfer weights between nodes and their neighbors, improving the flexibility and expressiveness of the model.
[0063] For example: For the edge (i, j) connecting node i and node j, the attention score e ij The calculation of is as follows:
[0064]
[0065] Among them, LeakyReLU represents a variant of the Rectified Linear Unit (ReLU), which is an activation function; || is a vector concatenation operation; W is a linear transformation matrix; h i represents the representation of node i; h j represents the representation of node j; Represents the transpose of the weight parameter matrix.
[0066] Afterwards, the softmax function is used to adjust the attention score e ij Normalize and get the attention weight α ij as follows:
[0067]
[0068] Among them, α ij It represents the attention weight to be multiplied when node j is transferred to node i. represents the set of neighbor nodes of node i, e ik Represents the attention score of the edge (i, k) connecting node i and its neighbor node k.
[0069] Finally, using the attention weight α ij Neighbor node set The representation of is weighted averaged, and the representation of the updated node is as follows:
[0070]
[0071] Among them, h′ i represents the updated representation of node i, α ij represents the attention weight of neighbor node j, h j represents the representation of neighbor node j, W represents the weight matrix, and σ represents the activation function.
[0072] The present application provides a method for monitoring computing resources of brain-like applications based on a graph neural network model. The graph neural network model is used to extract the spatial characteristics of brain-like applications, and the computing resources occupied by brain-like applications are predicted through a resource monitoring model in combination with the temporal characteristics of brain-like applications, so as to monitor the computing resources of brain-like applications.
[0073] The following is an introduction to the brain-like application computing resource monitoring method based on graph neural network provided in this application.
[0074] Figure 1 The flowchart of the method for monitoring computing resources of brain-like applications based on graph neural networks according to an embodiment of the present disclosure is shown. This embodiment takes the method used in an electronic device with computing capabilities as an example for explanation. The electronic device can be a user terminal or a server. The user terminal includes but is not limited to: a computer, a mobile phone, or a tablet computer, etc. This embodiment does not limit the types of electronic devices and user terminals. Figure 1 As shown, the method includes:
[0075] Step 101, obtaining model information of a pulse neural network model for implementing brain-like applications.
[0076] The spiking neural network model includes a plurality of spiking neurons and synaptic connections between different spiking neurons, and the model information is used to indicate the inherent characteristics of the spiking neurons and the inherent characteristics of the synaptic connections.
[0077] In the present application, brain-like application refers to an application program implemented based on a pulse neural network model. The brain-like application can provide certain functions through the computing process performed by SNN, such as: providing image and voice recognition functions, intelligent chat functions, etc. This embodiment does not limit the specific functions provided by the brain-like application.
[0078] In this embodiment, the model information is adapted to the input information of the graph neural network model. In other words, the model information is information that can be processed by the graph neural network model. Schematically, the model information includes a node feature matrix, an edge index matrix, and an edge feature matrix.
[0079] In one example, the node feature matrix includes a feature vector of each spiking neuron, which includes a feature vector of inherent features such as the membrane potential and / or firing rate of the spiking neuron. Intrinsic features are features possessed by the spiking neuron itself, which are generally set by developers when developing brain-like applications, that is, the inherent features will not change with the operation of brain-like applications. Among them, the membrane potential is used to simulate a potential value inside a neuron, which represents the activation state of the spiking neuron. The firing rate refers to the frequency at which a spiking neuron emits a pulse (Spike) per unit time, that is, the firing rate is a measure of the activity of the spiking neuron, which is used to reflect the number of action potentials generated by the spiking neuron within a certain period of time.
[0080] Edge index matrix: contains an indication of whether each spiking neuron is connected to an edge.
[0081] Edge feature matrix: A feature vector that includes the synaptic features of each edge, such as propagation delay, and / or synaptic weight, which are intrinsic features of synaptic connections. Propagation delay refers to the time required for a signal to pass from one spiking neuron to another through a synapse; synaptic weight is used to indicate the degree to which a synapse amplifies or inhibits a signal.
[0082] Since the SNN model of brain-like applications may be realized through millions of spiking neurons, if the model information is constructed according to the inherent characteristics of each spiking neuron and the inherent characteristics of each synaptic connection, the scale of the model information will be too large, which will lead to the problem of greater difficulty in calculating spatial features.
[0083] Based on the above problems, in another example, model information of a pulse neural network model for implementing brain-like applications is obtained, including: obtaining the inherent characteristics of each pulse neuron; classifying and aggregating each pulse neuron into at least two neuron groups according to preset classification and aggregation rules; generating a node feature matrix based on the feature vector of each neuron group; generating an edge index matrix based on the synaptic connections between different neuron groups; and generating an edge feature matrix based on the connection characteristics of the synaptic connections between different neuron groups.
[0084] Among them, each spiking neuron in the same neuron group has at least one same inherent characteristic.
[0085] Optionally, the type of inherent feature indicated by the classification aggregation rule is the same as or different from the type of the feature vector of the above-mentioned pulse neuron. For example, the inherent feature may also include the distribution position of the pulse neuron in the model (in this case, the type of inherent feature is different from the type of the feature vector of the pulse neuron); for example, the inherent feature includes the firing rate of the pulse neuron (in this case, the type of inherent feature is the same as the type of the feature vector of the pulse neuron). This embodiment does not limit the type of inherent feature.
[0086] Schematically, the classification aggregation rule is used to indicate the inherent features used in classification aggregation. For example, if the inherent feature indicated by the classification aggregation rule is the distribution position of the pulse neurons in the model, the electronic device will classify and aggregate the pulse neurons with the same distribution position into the same neuron group. For another example, if the inherent feature indicated by the classification aggregation rule is the firing rate, the electronic device will classify and aggregate the pulse neurons with the same firing rate into the same neuron group. This embodiment does not limit the type of inherent features indicated by the classification aggregation rule.
[0087] In one example, the classification and aggregation rules are pre-stored in the electronic device, and in this case, the inherent features indicated by the classification and aggregation rules remain unchanged.
[0088] In another example, the classification aggregation rule is set based on user needs. At this time, before classifying and aggregating each spiking neuron into at least two neuron groups according to the preset classification aggregation rule, it also includes: receiving a setting operation; and obtaining the classification aggregation rule indicated by the setting operation. Since the classification aggregation rule can be set based on user needs, it is possible to classify and aggregate spiking neurons according to user needs, thereby improving the flexibility of classification aggregation.
[0089] The electronic device receives a setting operation through a human-computer interaction control, where the setting operation is used to set a classification aggregation rule.
[0090] Optionally, the electronic device classifies and aggregates each pulse neuron into at least two neuron groups based on a classification aggregation algorithm. The classification aggregation algorithm may be a K-Nearest Neighbors (KNN) algorithm or a random forest algorithm, etc. This embodiment does not limit the implementation method of the classification aggregation algorithm.
[0091] In one example, after classification and aggregation, the node feature matrix includes: a feature vector of each neuron group, the feature vector of the neuron group includes but is not limited to: statistical features such as the average membrane potential and / or average firing rate of each spiking neuron in each neuron group, and since the statistical features are determined based on the inherent features of each spiking neuron, the node feature matrix is used to indicate the inherent features of each spiking neuron in each neuron group;
[0092] The edge index matrix includes: the indices of synaptic connections between different neuron groups;
[0093] The edge feature matrix includes: feature vectors of synaptic connections between different neuron groups. Since the synaptic connections between different neuron groups are determined based on the inherent features of the synaptic connections between spiking neurons in different neuron groups, the edge feature matrix is used to indicate the inherent features of the synaptic connections between spiking neurons in different neuron groups.
[0094] Optionally, the model information is determined based on the organizational information set by the developer when developing the brain-like application. For example: the brain-like application uses a development framework to describe the organizational information of the brain-like application, and the organizational information includes the feature vector of each spiking neuron, whether each spiking neuron is connected by edges, and the synaptic characteristics of each edge. At this time, the organizational information of the development framework is read to obtain the inherent characteristics of the spiking neurons and synaptic connections of the brain-like application to determine the model information. Among them, the development framework includes but is not limited to: SNN development framework (for example, PyNN framework) or deep SNN (ie D-SNN) development framework based on PyTorch (for example, SpikingJelly framework, Norse framework), etc.
[0095] Step 102, input the model information into a pre-trained graph neural network model to determine the spatial characteristics of the spiking neurons in the spiking neural network model.
[0096] In this embodiment, the graph neural network model is used to update the model information to the spatial features of the pulse neurons in the SNN. Specifically, the node feature initialization of the graph neural network model is determined based on the node feature matrix in the model information, and then the graph neural network model combines the edge index matrix and the edge feature matrix to update the node features of each node to obtain the spatial features.
[0097] The spatial characteristics of the spiking neuron are used to indicate the position and connection relationship of the spiking neuron in the SNN. Optionally, if the node feature matrix in the model information is the feature vector of each neuron group, the spatial characteristics of the spiking neuron are represented by the spatial characteristics of the neuron group to which the spiking neuron belongs.
[0098] The graph neural network model is pre-trained based on the training data. The specific training process is detailed in the following embodiment, and this embodiment will not be described in detail here.
[0099] In one example, an electronic device can determine the spatial features of a spike neuron through a graph neural network model. However, according to the calculation principle of the graph neural network model, the graph neural network model can only capture the spatial features between two adjacent nodes, that is, local spatial features.
[0100] Based on this, in another example, the graph neural network model includes a first graph neural network model and a second graph neural network model; accordingly, the model information is input into the pre-trained graph neural network model to determine the spatial features of the spiking neurons in the spiking neural network model, including: inputting the model information into the first graph neural network model to determine the local spatial features of the spiking neurons based on the model information; based on the local spatial features and the second graph neural network model, determining the global spatial features of the spiking neurons. At this time, the spatial features include local spatial features and global spatial features. In this embodiment, the spatial features output by the graph neural network model include local spatial features of spiking neurons and global spatial features of spiking neurons.
[0101] In this embodiment, a second graph neural network model is set after the first graph neural network model so that the second graph neural network model can further obtain features of more distant neighbor nodes based on the local spatial features output by the first graph neural network model, thereby capturing global spatial features.
[0102] In addition, in order for the second graph neural network model to calculate global spatial features, the edge index matrix and edge feature matrix in the model information will also be input into the second graph neural network model.
[0103] Optionally, in order to further improve the accuracy of the graph neural network model in determining spatial features, at least one of the first graph neural network model and the second graph neural network model is a graph attention network model.
[0104] In one example, the first graph neural network model and the second graph neural network model are both implemented as graph attention network models. The model structures of the first graph neural network model and the second graph neural network model are the same, but the input information of the two is different. The input information of the first graph neural network model is model information, and the input information of the second graph neural network model is features obtained based on the local spatial features output by the first graph neural network model. In addition, the attention weights and information aggregation levels of the first graph neural network model and the second graph neural network model are different to extract local spatial features and global spatial features respectively.
[0105] In one embodiment, the local spatial features output by the first graph neural network model are directly input into the second graph neural network model to determine the global spatial features of the spiking neurons.
[0106] In another embodiment, the local spatial features output by the first graph neural network model are processed by other network layers and then input into the second graph neural network model to determine the global spatial features of the pulse neurons.
[0107] For example, a Dropout layer is also provided between the first graph neural network model and the second neural network model during the training phase. In this embodiment, a certain proportion of neurons in the graph neural network model are randomly discarded through the Dropout layer during the training process, so that the model does not overly rely on certain specific neurons, which can prevent overfitting problems during model training and improve the model training effect.
[0108] The Dropout layer can be used in the actual prediction process of the graph neural network model, or deleted in the actual prediction process of the graph neural network model. This embodiment does not limit the use of the Dropout layer in the actual prediction process.
[0109] If the Dropout layer is used in the actual prediction process of the graph neural network model, the local spatial features are processed by the Dropout layer and then input into the second graph neural network model.
[0110] For example, an activation layer is set between the first graph neural network model and the second graph neural network model to introduce nonlinear features into the model and improve model performance. At this time, the local spatial features are processed by the activation layer and then input into the second graph neural network model.
[0111] In other embodiments, the first graph neural network model and the second neural network model may also be provided with other network layers for improving model performance, which are not listed one by one in this embodiment.
[0112] In addition, in other embodiments, the number of graph neural network models can be set to more as needed. In this case, the above-mentioned Dropout layer and / or activation layer can be set between two adjacent graph neural network models. This embodiment does not limit the number of graph neural network models.
[0113] Optionally, the graph neural network model is also connected to a vector merging layer to merge the spatial features of different pulse neurons output by the graph neural network model into spatial features represented by a single output. Schematically, the vector merging layer can be a Reduce layer. In other embodiments, the vector merging layer can also be a network layer with the same function. This embodiment does not limit the implementation method of the vector merging layer.
[0114] Step 103: Obtain the timing characteristics of the brain-like application.
[0115] The timing features are used to indicate the time information of the spikes generated by the spiking neurons.
[0116] In one example, obtaining timing features of a brain-inspired application includes: obtaining pre-trained timing feature prediction models corresponding to different hardware platforms; determining a target timing feature prediction model from various timing feature prediction models based on device information of an electronic device on which the brain-inspired application is currently running; and inputting model structure information and parameter information of the brain-inspired application into the target timing feature prediction model to obtain timing features.
[0117] Since different hardware platforms generate different timing characteristics when running brain-inspired applications, this embodiment pre-trains the timing characteristic prediction models corresponding to different hardware platforms, selects the target timing characteristic prediction model corresponding to the electronic device on which the brain-inspired application is currently running, to determine the timing characteristics of the brain-inspired application, thereby ensuring the accuracy of the acquired timing characteristics.
[0118] Optionally, the time series feature prediction model is obtained based on regression model training, or based on neural network model training. This embodiment does not limit the construction method of the time series feature prediction model. The training process of the time series feature prediction model includes: for each hardware platform, obtaining the training data when the hardware platform runs different brain-like applications, the training data includes the sample model structure information and sample parameter information of each brain-like application, and the time series feature label corresponding to the brain-like application; iteratively training the pre-created mathematical model based on the training data to obtain the time series feature prediction model. The training process is similar to the training process of the traditional neural network, and this embodiment will not be repeated here.
[0119] Schematically, based on the device information of the electronic device on which the brain-like application is currently running, a target timing feature prediction model is determined from various timing feature prediction models, including: obtaining platform information of the hardware platform corresponding to each timing feature prediction model; matching the device information with the platform information, and determining the timing feature prediction model corresponding to the platform information consistent with the device information as the target timing feature prediction model.
[0120] Illustratively, the platform information and the device information have the same information type, for example, the information type includes: GPU model, and / or GPU quantity, etc. This embodiment does not limit the information type of the platform information and the device information.
[0121] In one example, the model structure information is used to define the model framework of the SNN and the organization between each spiking neuron. The model structure information determines the process of the SNN processing input data, the process of information transmission, and the way the output results are generated. For example, the model structure information includes the hierarchical structure and synaptic connection between different spiking neurons. The parameter information is used to define the behavior of each spiking neuron and synaptic connection in the SNN, for example, the parameter information includes the membrane potential and discharge threshold of each spiking neuron.
[0122] Schematically, methods for obtaining model structure information and parameter information include: determining model structure information and parameter information based on model information, for example: the model information includes the inherent characteristics of each pulse neuron and the inherent characteristics of the synaptic connections between different pulse neurons, then the model structure information may include the inherent characteristics of the synaptic connections, and the parameter information may include the inherent characteristics of each pulse neuron; or, reading and parsing tissue information of brain-like applications to obtain model structure information and parameter information. This embodiment does not limit the method for obtaining model structure information and parameter information.
[0123] Optionally, step 103 may be executed after step 102, or may be executed before step 102, or may be executed simultaneously with step 102. This embodiment does not limit the execution order between steps 103 and 102.
[0124] Step 104 , based on the temporal features, spatial features and a pre-trained resource monitoring model, determine the computing resources occupied by the brain-inspired application when it is running.
[0125] Among them, the computing resources occupied by the brain-like application during operation include, but are not limited to, parameters such as running time, CPU share and / or memory access share. This embodiment does not limit the type of the computing resources.
[0126] In one example, the timing features and spatial features of each spiking neuron are concatenated and input into a pre-trained resource monitoring model to calculate the computing resources occupied by the brain-inspired application during operation.
[0127] In another example, the time series features are concatenated with the spatial features merged by the vector merging layer and input into a pre-trained resource monitoring model to calculate the computing resources occupied by the brain-like application during runtime.
[0128] Optionally, the resource monitoring model can be trained based on a traditional neural network model, for example, based on a multilayer perceptron (MLP) training. In other embodiments, the resource monitoring model can also be trained based on other neural networks, such as a recurrent neural network (RNN) training or a long short-term memory network (LSTM) training. This embodiment does not limit the construction method of the resource monitoring model.
[0129] Optionally, after obtaining the computing resources occupied by the brain-like application during operation, the electronic device can output the prediction result of the computing resources in response to the output instruction. And / or, the hardware resources of the electronic device are allocated based on the computing resources to avoid resource waste. And / or, the electronic device outputs a first optimization suggestion for the hardware configuration based on the computing resources, and the first optimization suggestion is used to prompt the user to optimize the hardware configuration. And / or, the electronic device outputs a second optimization suggestion for the brain-like application based on the computing resources, and the second optimization suggestion is used to prompt the developer to adjust the parameters of the brain-like application to improve the performance of the brain-like application.
[0130] In summary, the method for monitoring computing resources of brain-like applications based on graph neural networks provided in this embodiment obtains model information of a pulse neural network model that implements brain-like applications, and the model information is used to indicate the inherent characteristics of pulse neurons and the inherent characteristics of synaptic connections in the pulse neural network model; inputs the model information into a pre-trained graph neural network model to determine the spatial characteristics of pulse neurons in the pulse neural network model; obtains the timing characteristics of the brain-like application; based on the timing characteristics, spatial characteristics and pre-trained resource monitoring models, determines the computing resources occupied by the brain-like application when it is running; can solve the problem that traditional machine learning methods cannot predict the computing resources of brain-like applications; because the spatial characteristics and timing characteristics of brain-like applications can be extracted to calculate the computing resources occupied by brain-like applications when they are running, the accuracy of computing resource monitoring can be guaranteed. At the same time, compared with the simulation method to realize the monitoring of computing resources, since there is no need to simulate the operation process of pulse neurons, the calculation speed of computing resources can also be improved.
[0131] In addition, by setting the second graph neural network model after the first graph neural network model, the second graph neural network model can further obtain the features of more distant neighbor nodes based on the local spatial features output by the first graph neural network model, thereby capturing global spatial features and improving the accuracy of extracting spatial features.
[0132] In addition, by setting at least one of the first graph neural network model and the second graph neural network model as a graph attention network model, the accuracy of the graph neural network model in extracting spatial features can be further improved.
[0133] In addition, a Dropout layer is set between the first graph neural network model and the second neural network model during the training stage. During the training process, a certain proportion of neurons in the graph neural network model is randomly discarded through the Dropout layer, so that the model does not overly rely on certain specific neurons. This can prevent overfitting problems during model training and improve the model training effect.
[0134] In addition, by classifying and aggregating the spiking neurons into at least two neuron groups and generating a node feature matrix according to the feature vectors of the neuron groups to calculate the spatial features, the data volume of the node feature matrix can be reduced, thereby improving the calculation efficiency of the spatial features.
[0135] In addition, by obtaining classification aggregation rules according to setting operations, the flexibility of classification aggregation of spike neurons can be improved.
[0136] In order to more clearly understand the method for monitoring computing resources for brain-like applications provided by the present application, an example is given below to illustrate the method. Figure 2 For any brain-like application 201, the electronic device obtains the model information, model structure information and parameter information of the brain-like application 201. The model information includes a node feature matrix obtained based on the feature vector of the neuron group after the pulse neurons are classified and aggregated to obtain the neuron group, an edge index matrix obtained based on the index of the synaptic connection between different neuron groups, and an edge feature matrix obtained based on the connection characteristics of the synaptic connection between different neuron groups.
[0137] The model information is input into the first graph neural network model 202 to obtain local spatial features, which are processed by the activation function (Rectified Linear Unit, ReLU) 203 and then output to the second graph neural network model 205 through the Dropout layer 204. The second graph neural network model 205 calculates the local spatial features and global spatial features of the pulse neuron based on the features output by the Dropout layer 204. After that, the local spatial features and global spatial features of each pulse neuron are merged through the Reduce layer 206 to obtain the spatial features.
[0138] The model structure information and parameter information are input into the target time series feature prediction model 207 corresponding to the currently running electronic device to obtain the time series feature.
[0139] The spatial features and the temporal features are concatenated and input into the resource monitoring model 208 to obtain the computing resources occupied by the brain-like application when it is running.
[0140] In this embodiment, the local spatial features and global spatial features of the brain-inspired application can be extracted to predict the computing resources occupied by the brain-inspired application when it is running, which can improve the prediction accuracy.
[0141] In one example, the training process of the graph neural network model and the resource monitoring model in the above embodiment includes: obtaining training data, which includes sample model information and a computing resource label corresponding to the sample model information; inputting the sample model information into the graph neural network model to obtain a first result; inputting the model structure information and parameter information corresponding to the sample model information into a pre-trained time series feature prediction model to obtain a second result; merging the first result and the second result and inputting them into a pre-created resource monitoring model to obtain a third result; comparing the third result with the computing resource label, and iteratively training the graph neural network model and the resource monitoring model based on the comparison result to obtain the trained graph neural network model and the resource monitoring model.
[0142] In other embodiments, the graph neural network model and the resource monitoring model may also be trained separately, that is, the graph neural network model is trained using sample model information and the spatial feature labels corresponding to the sample model information, and the resource monitoring model is trained using sample timing features, sample spatial features and corresponding computing resource labels. This embodiment does not limit the training methods of the graph neural network model and the resource monitoring model.
[0143] Optionally, during the training process of the graph neural network model and the resource monitoring model, 70% of the original data set can be used for training to obtain training data, 15% can be used for verification to obtain verification data, and the remaining 15% can be used for testing to obtain test data. An Adaptive Moment Estimation (Adam) optimizer with a learning rate of 0.001 can be used to assist in training. In other embodiments, the data set division method, the learning rate value and the optimization algorithm can also be implemented in other ways, which are not limited in this embodiment.
[0144] The accuracy and efficiency of the brain-like application computing resource monitoring method provided by this application are described below. Assuming that multiple brain-like applications are obtained by changing the pulse neuron parameters of the monkey brain simulation program, the human brain simulation program, and the mouse brain simulation program, these brain-like applications are run on the GPU models of the hardware platforms NVIDIA GPU 4090, A100, and P100, respectively. The parameters of the computing resources occupied by the brain-like applications on each hardware platform include the running time, CPU share, and memory access share.
[0145] The test results show that the average relative error rate of computing resources obtained by the above computing resource monitoring method is 5.58%, which is close to the average error rate of 5.3% of the computing resources occupied by traditional applications (i.e., non-brain-like applications) using the general GraphCodeBERT network. That is, the accuracy of the prediction of computing resources for brain-like applications provided by this application has reached the level of prediction accuracy in the field of general computing, that is, the accuracy of predicting the computing resources occupied by brain-like applications is guaranteed.
[0146] At the same time, compared with the method of implementing computing resource monitoring based on simulation, the computing speed of the resource monitoring method provided by the present application can reach 200 times the speed of traditional simulation implementation, saving the computing time of computing resources to improve the processing efficiency of downstream tasks.
[0147] Figure 3 It is a block diagram of a brain-like application computing resource monitoring device based on graph neural network provided by an embodiment of the present application. The device includes: a model acquisition module 310, a first extraction module 320, a second extraction module 330 and a resource monitoring module 340.
[0148] A model acquisition module 310, used to acquire model information of a spiking neural network model for implementing the brain-like application; wherein the spiking neural network model includes a plurality of spiking neurons and synaptic connections between different spiking neurons, and the model information is used to indicate the inherent characteristics of the spiking neurons and the inherent characteristics of the synaptic connections;
[0149] A first extraction module 320, configured to input the model information into a pre-trained graph neural network model to determine the spatial characteristics of spiking neurons in the spiking neural network model;
[0150] A second extraction module 330, used to obtain the temporal characteristics of the brain-like application;
[0151] The resource monitoring module 340 is used to determine the computing resources occupied by the brain-inspired application when it is running based on the temporal features, the spatial features and a pre-trained resource monitoring model.
[0152] For relevant details, refer to the above method embodiment.
[0153] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0154] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.
[0155] The present disclosure also provides an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory. Figure 4 .
[0156] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.
[0157] Figure 4 A block diagram of a brain-like application computing resource monitoring device or a statistical information collection device based on a graph neural network provided by an embodiment of the present application. For example, the device 1900 can be provided as a server or a terminal device. Figure 4 , the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.
[0158] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.
[0159] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.
[0160] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A method for monitoring computing resources of brain-like applications based on graph neural network, characterized in that: The method comprises: Acquiring model information of a spiking neural network model for implementing the brain-like application; wherein the spiking neural network model includes a plurality of spiking neurons and synaptic connections between different spiking neurons, and the model information is used to indicate inherent characteristics of the spiking neurons and inherent characteristics of the synaptic connections; Inputting the model information into a pre-trained graph neural network model to determine the spatial characteristics of spiking neurons in the spiking neural network model; Acquiring the timing characteristics of the brain-like application; Based on the temporal features, the spatial features and a pre-trained resource monitoring model, the computing resources occupied by the brain-inspired application when it is running are determined.
2. The method according to claim 1, characterized in that The graph neural network model includes a first graph neural network model and a second graph neural network model; accordingly, inputting the model information into a pre-trained graph neural network model to determine the spatial characteristics of spiking neurons in the spiking neural network model includes: Inputting the model information into the first graph neural network model to determine the local spatial features of the spiking neuron based on the model information; Determining a global spatial feature of the spiking neuron based on the local spatial feature and the second graph neural network model; The spatial features include the local spatial features and the global spatial features.
3. The method according to claim 2, characterized in that At least one of the first graph neural network model and the second graph neural network model is a graph attention network model.
4. The method according to claim 2, characterized in that: A Dropout layer is also provided between the first graph neural network model and the second graph neural network model during the training phase.
5. The method according to claim 1, characterized in that The graph neural network model is also connected to a vector merging layer to merge the spatial features of different pulse neurons output by the graph neural network model into spatial features represented by a single output.
6. The method according to any one of claims 1 to 5, characterized in that: The obtaining of model information of the pulse neural network model for implementing the brain-like application includes: Capture the intrinsic characteristics of each spiking neuron; According to a preset classification and aggregation rule, each spiking neuron is classified and aggregated into at least two neuron groups; wherein each spiking neuron in the same neuron group has at least one same inherent characteristic; Generate a node feature matrix based on the feature vector of each neuron group; Generate an edge index matrix based on the synaptic connections between different neuronal groups; Generate an edge feature matrix based on the connection features of synaptic connections between different neuron groups; The model information includes the node feature matrix, the edge index matrix and the edge feature matrix.
7. The method according to claim 6, characterized in that Before classifying and aggregating each spiking neuron into at least two neuron groups according to a preset classification and aggregation rule, the method further includes: Receive setup operations; Obtaining a classification aggregation rule for the setting operation indication.
8. The method according to any one of claims 1 to 5, characterized in that: The obtaining of the timing characteristics of the brain-like application includes: Obtain pre-trained time series feature prediction models corresponding to different hardware platforms; Determining a target timing feature prediction model from various timing feature prediction models based on device information of the electronic device on which the brain-inspired application is currently running; The model structure information and parameter information of the brain-like application are input into the target time series feature prediction model to obtain the time series feature.
9. A data processing device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method for monitoring computing resources of brain-like applications based on graph neural networks as described in any one of claims 1 to 8 when executing the instructions stored in the memory.
10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method for monitoring computing resources of brain-like applications based on graph neural networks described in any one of claims 1 to 8 is implemented.
Citation Information
Patent Citations
Method and system for mapping spiking neural network to brain-like computing platform and medium
CN111882065A
Data processing system based on brain-like technology
CN114238707A
Spiking neural network application representation method and device of brain-like computer operating system
CN115392443A
Method for predicting dynamic network link by using spiking neural network
CN117035013A
