A CPU performance index prediction method based on a gated graph neural network

Abstract

The application discloses a CPU performance index prediction method based on a gated graph neural network, which obtains data of CPU performance bottleneck related indexes of the same architecture of multiple C language projects, and constructs a data set; C functions are extracted from the C language projects and an abstract syntax tree is constructed; data flow information, control flow information and code sequence information are integrated in the abstract syntax tree; code information graphs are obtained; then a deep learning model based on the gated graph neural network is built and trained, and finally, CPU performance bottleneck related indexes of the C functions are predicted under the same architecture as the data set. The method can statically evaluate CPU performance bottleneck indexes of the C functions without actually running the codes, and does not require developers to write tests, perform compilation and construction and the like for the codes, thereby avoiding time length overhead of a traditional dynamic program analysis method, and improving the efficiency of the developers in CPU performance bottleneck positioning and CPU performance optimization.

Description

Technical Field

[0001] This invention relates to the field of software performance evaluation, and in particular to a method for predicting CPU performance metrics based on gated graph neural networks. Background Technology

[0002] The purpose of this invention is to address the shortcomings of existing technologies by providing an automated static code performance evaluation method based on gated graph neural networks. This method can automatically predict the proportions of four CPU performance indicators, solving the problem of laborious and time-consuming traditional dynamic program analysis, and helping developers to identify and solve problems in a timely manner.

[0003] Improving software application performance is one of the most crucial tasks in modern software development. In computer architecture, the CPU is a key component responsible for executing all instructions and handling all computations. However, certain code patterns or algorithms can adversely affect the CPU pipeline, leading to performance degradation. For example, branching statements, loops, and frequent memory accesses can cause pipeline stalls or conflicts, thus reducing CPU efficiency. Therefore, analyzing how an application operates within the CPU pipeline is essential during software performance evaluation. By gaining a deep understanding of CPU architecture and its workings, software developers can pinpoint bottlenecks and optimize their code to maximize CPU resource utilization and improve application performance to the greatest extent possible.

[0004] Existing technologies primarily rely on dynamic program analysis, which involves monitoring and analyzing program behavior during runtime to uncover potential performance issues. To pinpoint CPU performance bottlenecks, developers need to write a sufficient number of comprehensive test functions, then compile, build the software, and run the tests to collect and analyze data. This process is time-consuming and labor-intensive. After successfully identifying the CPU performance bottleneck, developers typically need to optimize it. To measure the effectiveness of the optimization, developers still need to repeat the build, deployment, and run process.

[0005] Currently, there is no method to build a deep learning model using the static features of code and predict CPU performance metrics. Building a deep learning model using the static features of code is a code-based machine learning technique. It generates a code representation by analyzing the syntax, structure, and other static features of the code, and then feeds this representation into a deep learning model for training and prediction. Compared to dynamic program analysis methods, deep learning models built using the static features of code do not require actual code execution during application; they only need to extract the static features of the code to make relevant predictions. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by providing a static code performance evaluation method based on gated graph neural networks. This method can automatically predict the proportions of different indicators of CPU performance bottlenecks without actually running the code, and provides developers with real-time feedback during the development process. This method utilizes gated graph neural networks to learn the semantic and structural representations of C functions. Given a code snippet, we fuse different types of code structures, such as abstract syntax trees, data flow graphs, and control flow graphs, into a code information graph. A gated graph neural network and attention mechanism are used to extract the representation of the entire code information graph, and finally, a multilayer perceptron is used to predict relevant indicators of CPU performance bottlenecks.

[0007] This invention is achieved through the following technical solution: a method for predicting CPU performance indicators based on gated graph neural networks, the steps of which are as follows:

[0008] S1. Obtain data on CPU performance bottleneck related metrics of the same architecture from multiple C language projects and construct a dataset;

[0009] S2. Extract C functions from the C language project and construct an abstract syntax tree;

[0010] S3. Incorporate data flow information, control flow information, and code sequence information into the abstract syntax tree to obtain a code information graph;

[0011] S4. Build a deep learning model based on gated graph neural networks;

[0012] S5. Train deep learning models based on code information graphs;

[0013] S6. Predict CPU performance bottleneck metrics for C functions using the same architecture as the dataset.

[0014] Furthermore, the specific steps of S1 are as follows:

[0015] S1.1, Discovering open-source C language projects:

[0016] Select relevant C language projects that are popular, have clear build processes, and have extensive testing;

[0017] S1.2, Build the project: Build the selected C language project;

[0018] S1.3 Run the test case and perform performance analysis.

[0019] Run test cases for a C language project and collect relevant metrics for C functions; during the sampling and collection process, the average of multiple results is selected as the final predicted label.

[0020] S1.4, C Function Extraction and Dataset Partitioning

[0021] Functions are used to extract corresponding performance analysis results with relevant metrics from the source code, and data in the format of <C function, performance metric> is collected; the collected data is divided into a training set, a validation set, and a test set.

[0022] Further, the metrics related to the CPU performance bottleneck in S1 are the top-level metrics of four top-down microarchitecture analysis methods, including:

[0023] Retiring: Represents the proportion of the number of completed instructions, and is used to evaluate the relatively real efficiency of the program for the processor;

[0024] Back-End Bound: Represents the proportion of the processor being in a waiting idle state because the back end of the processor is too late to execute the micro-operations sent by the front end;

[0025] Front-End Bound: Represents the proportion of the processor being in a waiting idle state because the instruction fetching and decoding speeds of the front end of the processor are slow;

[0026] Bad Speculation: Represents the proportion of the analysis workload affected by incorrect execution paths.

[0027] Further, the construction of the abstract syntax tree is specifically as follows:

[0028] An abstract syntax tree is composed of nodes and edges. The nodes are further divided into terminal symbols and non-terminal symbols. Among them, terminal symbols refer to the symbols that cannot be expanded or further decomposed in the grammar rules, representing the specific lexical units or terminal symbols in the program. Non-terminal symbols refer to the symbols that can be further expanded or decomposed, representing the composite grammar structures in the program, denoted by the notation representing all the points on the abstract syntax tree; the edges of the abstract syntax tree represent the relationships between nodes, denoted by representing the original edges on the abstract syntax tree.

[0029] Further, the specific steps of incorporating data flow information into the abstract syntax tree are as follows:

[0030] Two types of edges representing data flow are added to the abstract syntax tree of the code so that the model can learn relevant information about the data flow. They are NextUse and ComputFrom respectively. Both sides of these two types of edges are terminal symbols representing variables in the abstract syntax tree;

[0031] The NextUse edge extracts the data flow relationship between the same variables, denoted by ; the ComputFrom edge extracts the data flow relationship between the same variable or different variables, denoted by representing.

[0032] Furthermore, the specific steps for incorporating control flow information into the abstract syntax tree are as follows:

[0033] Two types of edges representing control flow were added to the abstract syntax tree of the code so that the model could learn information related to control flow; namely, MustExe and MayExe. Both types of edges have non-terminal symbols of the abstract syntax tree on both sides.

[0034] MustExe edges represent edges that the program will definitely execute. This indicates that it is used to process while, goto, do_while, and compound statements in the C language. For while and do_while statements, it is necessary to check whether a certain conditional expression is satisfied, and the nodes of the while and do_while statements are connected to the nodes of the conditional expressions they are checking. For goto statements, the nodes of the goto statement and the target statement to which the goto statement jumps are connected. For compound statements, the nodes of all statements in the code block corresponding to a compound statement are connected in pairs from front to back in order.

[0035] MayExe edges represent edges that the program might execute, using... This indicates that it is used to process the switch and if statements in C language syntax. For the if statement, it means pointing from the node of the if statement to the node of the possible branches of the if statement; for the switch statement, it means pointing from the node of the switch statement to the node of the possible branches of each case statement.

[0036] Furthermore, NCS edges are introduced to represent the sequence information of the original code:

[0037] In an abstract syntax tree, the sequence information of the code is reflected in the terminal symbols. Therefore, a preorder traversal of the abstract syntax tree is performed to extract the positions of all terminal symbols, and they are connected in pairs from front to back in order.

[0038] Furthermore, the deep learning model based on the gated graph neural network specifically involves: initializing each node of the code information graph as a vector representation through the node initialization module, feeding it into the gated graph neural network module to learn the features of the code information graph, and then using an attention mechanism after the gated graph neural network module to... The vector representations of each node are compressed into a single vector, and this feature vector is then fed into a multilayer linear perceptron module for further learning. Finally, the Softmax function is used to predict the proportions of four CPU performance metrics. The gated graph neural network module uses a gated graph neural network as its basic module, and a total of [number missing] layers are stacked. layer.

[0039] Furthermore, in the node initialization module, two different initialization strategies are designed for the two types of nodes in the abstract syntax tree: terminal symbols and non-terminal symbols. Non-terminal symbols represent complex grammatical structures in the program and only have grammatical structure type information, while terminal symbols have lexical unit or terminal symbol information. For non-terminal symbols, the vector representation is randomly initialized according to the grammatical structure type. For terminal symbols, the pre-trained model Unixcoder is used to segment lexical units or terminal symbols and obtain the semantic vector representation of each word. Finally, the sum of the semantic vectors of multiple words represents the initialization.

[0040] Furthermore, the mini-batch stochastic gradient descent algorithm is used to minimize the value of the loss function: that is, a fixed number of samples are randomly selected in each iteration of each training round to calculate gradient updates and optimize the model; the defined loss function is the mean absolute error, using... Representing the true values ​​of the four performance metrics for each sample, using Let represent the predicted values ​​of the four performance metrics for each sample; then the loss for each sample is expressed by the following formula:

[0041]

[0042] in For the loss of this sample, The model's predicted value The first in One value, for The Values.

[0043] The beneficial effects of this invention are: currently, there is no static evaluation method for CPU performance bottleneck-related indicators. This invention, based on gated graph neural networks and incorporating the characteristics of CPU performance bottlenecks, implements a method that can statically evaluate CPU performance bottleneck indicators of C functions without actually running the code. The application of this invention does not require developers to write tests, perform compilation and building, etc., avoiding the time overhead of traditional dynamic program analysis methods, and improving the efficiency of developers in locating CPU performance bottlenecks and optimizing CPU performance. Attached Figure Description

[0044] Figure 1 is a flowchart of the data collection process;

[0045] Figure 2 is a schematic diagram of the abstract syntax tree for extracting C functions;

[0046] Figure 3 is a schematic diagram of incorporating data flow information into the abstract syntax tree;

[0047] Figure 4 is a schematic diagram of incorporating control flow information into the abstract syntax tree;

[0048] Figure 5 is a schematic diagram of incorporating code sequence information into the abstract syntax tree;

[0049] Figure 6 shows the model design architecture based on gated graph neural networks;

[0050] Figure 7 is a comparison diagram between the model of the present invention and the baseline model;

[0051] Figure 8 is a detailed comparison chart of the indices between the model of this invention and the baseline model;

[0052] Figure 9 is a comparison of time costs among different models and methods. Detailed Implementation

[0053] This invention mines data from open-source C language projects, uses the VTune Profiler tool to collect relevant performance metrics of C functions for model training, and employs deep learning methods to train, validate, and test the model, ultimately resulting in a usable deep learning model.

[0054] This invention can be applied to the actual development process of programmers, providing four indicators to predict CPU performance bottlenecks without actually running the code, helping programmers to identify and solve problems in a timely manner.

[0055] The specific working process of this invention is as follows:

[0056] Obtain data on CPU performance bottleneck-related metrics from multiple C language projects using the same architecture, and construct a dataset.

[0057] The four high-level metrics that this invention aims to predict for CPU performance bottlenecks are the highest-level metrics of a top-down microarchitecture analysis approach. Top-down microarchitecture analysis is a performance analysis method that helps developers understand microarchitecture-related performance data. This method collects specific hardware events, such as cache misses or branch error predictions, using on-chip performance monitoring units (PMUs) within the CPU core. By observing and combining these events, useful high-level metrics can be created to better understand application performance. The four high-level metrics that this invention aims to predict include:

[0058] Retiring: This represents the percentage of instructions completed, used to evaluate the relative efficiency of a program on the processor. A low retiring value indicates poor overall processor performance, requiring further analysis of other metrics.

[0059] Back-End Bound: This represents the percentage of the processor that remains idle because the back-end cannot execute micro-operations sent from the front-end in time. A high back-end Bound indicates that the processor's back-end execution units are the bottleneck.

[0060] Front-End Bound: This indicates the percentage of the processor that is idle due to slow instruction fetching and decoding speeds at the processor's front end. A high front-end execution unit indicates that the processor's front-end execution unit is the bottleneck.

[0061] Bad Speculation: This indicates the proportion of analytical workload affected by faulty execution paths. A high value for this metric indicates that the processor is executing a lot of code along faulty paths, thus wasting processor resources.

[0062] It is worth noting that the sum of the four indicators above is fixed at 1. This invention is based on deep learning, therefore a certain amount of data needs to be collected for model training. The data collection process of this invention consists of five steps, as follows: Figure 1 As shown:

[0063] Discovering open source C language projects

[0064] Through manual inspection, relevant C language projects with high popularity, clear build processes, and abundant tests were selected. Since VTune Profiler requires running test cases of projects to collect data, C language projects that do not contain tests were removed in this embodiment of the invention.

[0065] Build Project

[0066] For each open-source C language project, this embodiment of the invention follows the official build process to manually compile and build, using a variety of build tools, including CMake, Meson, etc., and removing projects that failed to build.

[0067] Run test cases and perform performance analysis

[0068] This step involves collecting four relevant metrics for C functions by running test cases from C language projects. In this embodiment, test cases are executed continuously until the metrics show minimal fluctuation, ensuring data reliability. Each sample is spaced at least 10 seconds apart. Each project is run ten times, and a modified Z-score is used to remove noise from the data. This embodiment uses the average of these scores as the final predicted label.

[0069] C function extraction

[0070] Then, the embodiment of the present invention extracts functions with corresponding performance analysis results having relevant metrics from the source code. The tool for extracting functions is Tree-sitter, which can parse the source file into an abstract syntax tree and locate the positions of specific functions therein. So far, by associating C functions with their corresponding performance metrics, the embodiment of the present invention can collect data in the format of <C function, performance metric>.

[0071] Dataset construction

[0072] After the fourth step, the embodiment of the present invention can obtain a total of 2,471 pieces of data in the format of <C function, performance metric>, where the performance metrics are the 4 aforementioned high-level performance metrics. The embodiment of the present invention divides the constructed data into 3 groups: 80% (1,976) of the samples are used for model training; 10% (247) of the samples are used for model tuning; and the last 10% (248) of the samples are used for model testing, which are respectively called the training set, the validation set, and the test set.

[0073] Extracting the abstract syntax tree representation of C functions

[0074] The present invention constructs a neural network model based on a gated graph neural network. First, a C function needs to be represented by a graph. The present invention uses Tree-sitter to perform lexical analysis and syntax analysis according to the syntax of the C language and extracts the abstract syntax tree representation, as Figure 2 shown.

[0075] An abstract syntax tree is a tree structure used to describe the code structure in a programming language, where each node represents a syntactic element in the code. It is usually used in software development fields such as compilers and static analysis tools. An abstract syntax tree consists of nodes and edges. Nodes can be further divided into terminals and non-terminals. Among them, terminals represent the most basic lexical units or terminal symbols in the program, such as identifiers, constants, literals, etc., corresponding to the Figure 2 circular nodes of the abstract syntax tree on the right side in ; non-terminals represent composite syntactic structures in the program, such as statements, expressions, function definitions, etc., corresponding to the Figure 2 elliptical nodes of the abstract syntax tree on the right side in. Later, use the notation to represent all the points on the abstract syntax tree. The edges of the abstract syntax tree represent the relationships between nodes. For example, one node may be the parent node or child node of another node. In Figure 2 the solid lines are used to represent the parent-child relationships between nodes, pointing from the parent node to the child node. Later, use to represent the original edges on the abstract syntax tree.

[0076] Integrating data flow information into the abstract syntax tree:

[0077] The abstract syntax tree (AST) of code only contains the structured information of the program itself. However, the performance of a program is also related to data flow information. Data flow information refers to the transfer of data between different statements in the program, describing how data flows from one variable or data structure to another. Therefore, this invention adds two types of edges representing data flow to the AST of code so that the model can learn relevant information about data flow, namely: NextUse and ComputFrom, as shown in Figure 3. Both sides of these two types of edges are terminal symbols representing variables in the AST.

[0078] NextUse extracts the data flow relationship between the same variable, using... This indicates that the corresponding is Figure 3 The dashed lines in the diagram. Specifically, extracting NextUse edges requires first performing a preorder traversal of the abstract syntax tree to extract the position of each variable within the abstract syntax tree, and then connecting nodes with different positions of the same variable in sequence from front to back. For example, consider the variable... Suppose it appears in the terminal symbol of the abstract syntax tree. The location of occurrence is divided into Therefore, for this variable, there exists the following NextUse edge.

[0079]

[0080] ComputFrom extracts the data flow relationship between the same variable or different variables. This indicates that the corresponding is Figure 3 The short dashed line. Specifically, ComputFrom targets variable initialization and assignment statements within C functions, and its form is: By using Tree-sitter to analyze the abstract syntax tree, this embodiment of the invention was able to locate all variable initialization and assignment statements in the C function, and obtain... Connect all the nodes corresponding to all variables in the list. The corresponding node, such an edge is called a ComputFrom edge. For example, in Figure 3 In the code, the assignment statement `int b = a * 2;` will produce... The ComputFrom edge.

[0081] Incorporating control flow information into the abstract syntax tree

[0082] Besides the structured information and data flow information of the abstract syntax tree, the performance of a C function is also related to control flow information. Control flow information refers to the execution order of statements and conditional branches in the program, describing the execution path. Therefore, this invention adds two types of edges representing control flow to the abstract syntax tree of the code so that the model can learn relevant information about data flow. These are called MustExe and MayExe, respectively. Figure 4 As shown, both types of edges have non-terminal symbols of the abstract syntax tree on both sides.

[0083] MustExe edges represent edges that the program will definitely execute. This indicates that the corresponding is Figure 4 The middle line is a short dashed horizontal line with a single point. Specifically, this type of edge mainly handles the while, goto, do_while, and compound statements in the C language, all of which have a certain order concept. Therefore, this embodiment of the invention extracts the positions of these four types of statements by traversing the abstract syntax tree and adds corresponding edges according to different cases. For while and do_while statements, they both need to check whether a certain conditional expression is satisfied. This embodiment of the invention connects the nodes of while and do_while statements to the nodes of the conditional expressions they need to check; for goto statements, it connects the goto statement to the node of the target statement to which the goto statement jumps; for compound statements, it connects the nodes of all statements in the code block corresponding to a compound statement in order from front to back. Figure 4 This demonstrates the use of the compound statement.

[0084] MayExe edges represent edges that the program might execute, using... This corresponds to the long dashed line in Figure 4. Specifically, this edge processing involves the `switch` and `if` statements in C language syntax. The main characteristic of these two statements is that they both have the concept of branches, meaning the program execution may choose either one. By traversing the abstract syntax tree, the positions of the `if` and `switch` statements are extracted, and then corresponding edges are added according to C language syntax. For the `if` statement, this means pointing from the node of the `if` statement to the possible branches of the `if` statement; for the `switch` statement, this means pointing from the node of the `switch` statement to the possible branches of each `case` statement. Figure 4 This demonstrates the case of an if statement.

[0085] Incorporating code sequence information into the abstract syntax tree

[0086] Research shows that incorporating code sequence information into the abstract syntax tree (API) is helpful for model learning. Therefore, in addition to the edges inherent in the API and the previously introduced data flow and control flow edges, this embodiment of the invention also introduces another type of edge to extract the sequence information of the source code. This edge is called NCS (Natural Code Sequence). Figure 5 The double-dot short dashed line in the middle.

[0087] In an abstract syntax tree (AST), the sequence information of the code is embodied in terminal symbols. Therefore, this embodiment of the invention performs a preorder traversal of the AST to extract the positions of all terminal symbols and connects them pairwise from front to back in order. For example, suppose there are a total of Each of the terminal symbols corresponds to a terminal symbol. ,So

[0088]

[0089] Finally, the code information diagram of the C function generated using the above method can be summarized as follows:

[0090]

[0091] in Let the graph representation of the program contain all nodes, assuming a total of There are nodes, and all edges in the graph are represented by [variable name]. Symbols are used to represent this.

[0092] Building a deep learning model based on gated graph neural networks

[0093] The model involved in this invention comprises the following modules: a node initialization module; a gated graph neural network module; and a multilayer perceptron module. Specifically, the node initialization module initializes each node of the code information graph as a vector representation, which is then fed into the gated graph neural network module to learn the features of the code information graph. An attention mechanism is then used after the gated graph neural network module to further refine the graph. The vector representations of each node are compressed into a single vector, and this feature vector is then fed into a multilayer linear perceptron module for further learning. Finally, the Softmax function is used to predict the proportions of four CPU performance metrics. The gated graph neural network module uses a gated graph neural network as its basic module, and a total of [number missing] layers are stacked. Layers. The architecture of the entire model is as follows: Figure 6 As shown below. Each module will be explained in detail below:

[0094] a) Node initialization module

[0095] The node initialization module takes the code information graph mentioned earlier as its input. It is responsible for initializing a vector representation for each node, upon which the subsequent gated graph neural network module and multilayer perceptron module can be trained. This invention designs two different initialization strategies for the two types of nodes in the abstract syntax tree—terminal symbols and non-terminal symbols. Non-terminal symbols represent complex grammatical structures in the program, possessing only grammatical structure type information, while terminal symbols possess lexical unit or terminal symbol information. For non-terminal symbols, the vector representation is randomly initialized according to the grammatical structure type. For terminal symbols, a pre-trained model, Unixcoder, is used to segment lexical units or terminal symbols to obtain the semantic vector representation of each word. Finally, the sum of the semantic vectors of multiple words represents the initialization. In implementation, the node initialization module is an embedding layer responsible for mapping the index labels of words and grammatical structure types to vectors. Therefore, the graph... The initial vectors of the nodes can be represented by the following vector matrix:

[0096]

[0097] This invention considers the node initialization module as the zeroth layer, wherein For nodes The initialization vector, with a length of 768, is the input to the first layer of the gated graph neural network in the gated graph neural network module. (Superscript) These respectively represent the vector as the first... The first layer There are several time steps. Note that the zeroth layer is an embedding layer and does not involve the concept of time steps. This form is used here for consistency with subsequent notation. The concept of time steps will be explained in the gated graph neural network module.

[0098] b) Gated graph neural network module

[0099] The gated graph neural network module consists of A gated graph neural network is composed of layers. In a graph neural network, the update of the vector representation of a node is achieved through a message passing mechanism. Building upon this, the gated graph neural network further utilizes a gating mechanism to update the state vectors of nodes, thereby capturing the information transmission and structural relationships between nodes.

[0100] No. The node update process of a layer is as follows: Consider a vertex on the graph. Assuming all its neighbors are At each time step It receives information from all its neighbors; this information is the node's own vector representation. All messages from neighbors will undergo a linear transformation and aggregation, expressed by the following formula:

[0101]

[0102] in This is the result after aggregation. Because the code information graph representation of the C function defined in this embodiment of the invention has multiple edge types, This indicates the node when considering a certain edge type. Neighbors Learnable weight parameters for the edge type corresponding to this edge type. The vector representation of a neighboring node at time step t in layer k;

[0103] After aggregating information from different neighbors, the node It needs to be represented by its own vector. Updates are performed. Gated graph neural networks use gated recurrent units (GRUs) for updates, expressed by the following formula:

[0104]

[0105] in The result of an additional time step is expanded for the gated loop unit; this expansion occurs at each layer. At each time step, GRU stands for Gated Recurrent Unit, a variant of Recurrent Neural Network (RNN) commonly used for processing sequential data. It performs better than traditional RNN structures in handling long-term dependency problems. GRU controls the flow of information by introducing two gates: the update gate and the reset gate. The above formula can be expanded as follows:

[0106]

[0107]

[0108]

[0109]

[0110] in This indicates the output of the update gate in the GRU. This indicates that the output of the reset door is being reset. This represents a candidate hidden state. For the sigmoid function, This represents element-wise product, while , , To update the parameters that the gate can learn, , , These are the parameters that the refactoring gate can learn. , , Other learnable parameters for GRU.

[0111] The previous section explained the first part in detail. Layer nodes The update process for this node is similar to the update process for other nodes in this layer. In the [missing information - likely a specific node or node]... After all nodes in a layer have been updated, the vector representations of all nodes in that layer will be used to initialize the nodes of the next layer of the gated graph neural network.

[0112]

[0113] in Indicates the first The initialization vector representation of all nodes in the layer. The gated graph neural network of the next layer is expanded using gated recurrent units. The result is updated after each time step.

[0114] In graph neural networks, different graphs have different numbers of nodes. Therefore, a mechanism is needed to compress the vector representations of all nodes into a single feature vector so that it can be used by subsequent multilayer perceptron modules. This process is called graph readout. Because different vertices in the code information graph representation of C functions have different levels of importance, this embodiment of the invention uses an attention mechanism for graph readout, expressed by the following formula:

[0115]

[0116] in This is the vector representation of the entire C function code information graph. Implemented using a linear transformation layer to transform each node at the 1st... The last vector of the layer This is represented as an attention score. The normalized exponential function is used to normalize the graph. The attention scores of each vertex are normalized to ensure that the sum of these attention scores is 1. The final vector representation of the entire C function code information graph is obtained by multiplying the attention coefficient of each node by its own vector representation and then summing all the node attention coefficients.

[0117] Multilayer perceptron module

[0118] In the gated graph neural network module, each code information graph ends with a feature vector representation. To further extract To enhance the latent features and semantic representation of the gated graph neural network, this embodiment of the invention adds a multilayer perceptron module after the gated graph neural network module, as expressed by the following formula:

[0119]

[0120]

[0121] in For the activation function, the embodiment of this invention uses the Rectified Linear Unit (ReLU). For a and Intermediate vectors of the same size , , , These are the learnable parameters of a multilayer linear perceptron. This is a vector of length 4, corresponding to the four advanced metrics of CPU performance bottlenecks to be predicted in this embodiment of the invention. Retiring, Indicates Front-End Bound. Indicates Back-End Bound. This indicates BadSpeculation.

[0122] Training deep learning models based on code infographics

[0123] Deep learning model training typically uses gradient descent to minimize the loss function. Gradient descent is an iterative algorithm that updates the model parameters by calculating the gradient of the loss function with respect to the model parameters, causing the loss function value to continuously decrease. This process is repeated multiple times until the loss function value reaches an acceptable threshold or converges. In this invention, embodiments employ mini-batch stochastic gradient descent, where a fixed number of random samples are used in each iteration of each training round to calculate gradients and update the optimized model. The defined loss function is Mean Absolute Error (MAE), using... Let the four performance metrics represent the true values ​​for each sample. Then, the loss for each sample can be expressed by the following formula:

[0124]

[0125] in For the loss of this sample, The model's predicted value The first in One value, for The Values.

[0126] This invention uses Python for coding and employs the PyTorch and DGL deep learning frameworks. The vector representation of nodes in the graph is 768-dimensional. During training, the batch size is set to 16, and the gradient accumulation step is 2. The model undergoes a maximum of 200 training rounds. To prevent overfitting, this invention employs early stopping, evaluating the model's performance twice on the validation set during each training round. If the model shows no improvement for 16 consecutive rounds, training is stopped early. The dropout rate and learning rate are discussed in detail below. and stacking layers This invention utilizes the Optuna tool for automatic hyperparameter adjustment. This invention also employs techniques such as learning rate warm-up and learning rate decay to help the model converge faster.

[0127] Predicting CPU performance bottleneck metrics for C functions within the same architecture as the dataset.

[0128] Prior to this invention, there was no static code prediction method for CPU performance bottleneck indicators. In order to verify and evaluate the effectiveness of this invention, this embodiment selects three baseline models from the field of intelligent software engineering for comparison with the model of this invention. These models are widely used in the field of intelligent software engineering, such as vulnerability detection and defect prediction.

[0129] The first baseline model is a random forest model built on machine learning. Random forest models are widely used in software engineering research, showing good results in defect prediction and vulnerability detection applications. This embodiment of the invention utilizes the Bag-of-Words Model to extract features from C function source code. The Bag-of-Words Model is a text feature representation method. This model assumes that all words in a text are independent, and the occurrence of each word in the text depends only on the frequency of that word's occurrences. Therefore, the Bag-of-Words Model represents a text as a vector, where each dimension represents a word, and the dimension value is the frequency of that word's occurrences in the text. This embodiment of the invention treats the code as text and feeds it into the Bag-of-Words Model to extract vector representations. Then, these vectors are used as feature vectors in the random forest. Each decision tree in the random forest learns to predict four metrics related to CPO performance bottlenecks. The predictions of all decision trees in the random forest are averaged to obtain the final prediction.

[0130] The second baseline model is a Long Short-Term Memory (LSTM) network built using deep learning. LSTM is one of the most popular recurrent neural network models. It is also widely used in software engineering research and can effectively learn the sequence representations of code syntax units. This embodiment of the invention uses LSTM to learn the sequence of code syntax units and predict the corresponding CPU performance bottleneck metrics. For fair comparison, the same vocabulary as the model in this invention is used, and the dimension of the hidden layer states is also 768. This embodiment of the invention extracts the last hidden layer state of the LSTM output as the feature vector of the code and uses a multilayer perceptron of the same size to predict the CPU performance bottleneck metrics.

[0131] The third baseline model is a Transformer-based model built on deep learning. The Transformer is an encoder-decoder architecture widely used in machine translation. In this invention, the embodiments use the Transformer's encoder to learn code representations and predict CPU distribution based on the code's vector representation. Similar to LSTM, the embodiments employ the same vocabulary and multilayer perceptron layers to ensure fair comparison between models. The Transformer is based on a self-attention mechanism, learning the sequence representation of each lexical unit by calculating the attention between each pair of lexical units. To obtain the representation of the input sequence, the embodiments take the maximum value element-wise from the representations of all lexical units output by the Transformer as the final vector representation of the input code.

[0132] To measure the learning performance of the model, the following metrics are defined to evaluate the predictive quality of the model.

[0133] The first evaluation metric is the mean absolute error over the entire dataset, which can be expressed by the following formula. A lower value for this metric is better:

[0134]

[0135] in This represents the number of samples in the corresponding dataset.

[0136] The second evaluation indicator is defined as The prediction error of the model was measured to fall within the range of The accuracy rate can be expressed by the following formula, and a higher value for this metric is better:

[0137]

[0138] In this embodiment of the invention, the model of the present invention and three baseline models were trained on the collected data. The model that performed best on the validation set was retained. Subsequently, this embodiment of the invention evaluated the capabilities of different models on the test set using evaluation metrics. Note that the test set consists of CPU performance bottleneck-related metrics for C functions under the same architecture; this is data the models have not encountered before, and therefore can be used to measure the model's capabilities. The results are as follows: Figure 7 As shown.

[0139] As can be seen, the model of this invention outperforms the baseline models on all evaluation metrics, with LSTM performing the worst among all baseline models. Regarding mean absolute error, the error rates of all models are within 10%. The model of this invention performs best, controlling the mean absolute error rate to within 7%, i.e., its mean absolute error is 0.067. This result demonstrates that the model of this invention can predict the relevant metrics of CPU performance bottlenecks with low variance. The model of the present invention is in and The optimal results were achieved in both cases, at 49.90% and 79.44% respectively, with improvements of 12.0-16.2% and 7.1%-12.9% compared to the baseline model.

[0140] To further verify the capabilities of the model of this invention, embodiments of this invention further compared the mean absolute error of Retiring, Front-End Bound, Back-End Bound, and Bad Speculation. The result is as follows Figure 8 As shown. In summary, the model of this invention has significant advantages over the baseline model.

[0141] To verify the beneficial effects of this invention—namely, its ability to improve developers' efficiency in identifying CPU performance bottlenecks and optimizing CPU performance—the embodiments of this invention also compared the time cost, parameter count, and other aspects of the present invention, baseline models, and dynamic programming methods. The results are as follows: Figure 9 As shown.

[0142] This invention is based on a gated graph neural network model, which requires training before being used to predict CPU performance bottleneck metrics for C functions. Training is performed offline, and predictions can be performed online after training. Embodiments of this invention record the start and end times of their training and testing processes. For fair comparison, all models are trained and their training times are recorded on the same machine. Results show that both the baseline model and the model of this invention significantly outperform VTune Profiler in prediction speed. Although the model of this invention is not the fastest, considering the preceding discussion and comparisons, it achieves a good balance in model capability, prediction speed, and the number of model parameters. This improves the efficiency of developers in locating CPU performance bottlenecks and optimizing CPU performance, eliminating the time-consuming process of writing tests, compiling, building, and then dynamically reprogramming the code.

[0143] Predicting CPU performance bottleneck metrics for C functions within the same architecture as the dataset.

[0144] Prior to this invention, there were no static code prediction methods for CPU performance bottleneck metrics. To verify and evaluate the effectiveness of this invention, the embodiments of this invention first define the following metrics for model prediction:

[0145] In this embodiment of the invention, the best-performing model is stored on the validation set. Subsequently, the capabilities of different models are tested on the test set using evaluation metrics. Note that the test set consists of CPU performance bottleneck-related metrics for C functions under the same architecture; this data is not previously seen by the model and can therefore be used to measure its capabilities. The mean absolute error of the model in this invention is 0.067, meaning it can control the mean absolute error rate to within 7%. The model of the present invention is in and The values ​​are 49.90% and 79.44%, respectively. This indicates that the model of the present invention can effectively learn the structural and semantic information of the code.

[0146] To verify the beneficial effects of this invention—namely, its ability to improve developers' efficiency in locating CPU performance bottlenecks and optimizing CPU performance—this embodiment of the invention also compares the time cost between this invention and dynamic programming methods. This invention is based on a gated graph neural network model, which requires training before being used to predict CPU performance bottleneck metrics for C functions. The training process is conducted offline, and predictions can be performed online after training is complete. This embodiment of the invention records the start and end times of both the training and testing processes. For a fair comparison, the time consumption was statistically analyzed on the same machine. The results show that the model of this invention only requires 0.009 seconds to predict CPU performance bottleneck-related metrics for a C function, significantly exceeding the 10 seconds of VTune Profiler. It should be noted that these 10 seconds do not include the time spent writing tests and compiling the code. Therefore, this invention can improve developers' efficiency in locating CPU performance bottlenecks and optimizing CPU performance, eliminating the time-consuming process of writing tests, compiling, and then performing dynamic programming.

[0147] The above embodiments are used to explain and illustrate the present invention, but not to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A method for CPU performance indicator prediction based on a gated graph neural network, characterized in that, The method steps are as follows: S1. Obtain data of indicators related to the CPU performance bottleneck of the same architecture for multiple C language projects, and construct a dataset; S2. Extract C functions from the C language projects and construct an abstract syntax tree; S3. Incorporate data flow information, control flow information, and code sequence information into the abstract syntax tree; obtain a code information graph; The specific steps for incorporating control flow information into the abstract syntax tree are as follows: Two types of edges representing control flow are added to the abstract syntax tree of the code so that the model can learn relevant information about the control flow; namely: MustExe and MayExe. Both sides of these two types of edges are non-terminal symbols of the abstract syntax tree; MustExe edges represent edges that the program will definitely execute. This indicates that it is used to process while, goto, do_while, and compound statements in the C language. For while and do_while statements, it is necessary to check whether a certain conditional expression is satisfied, and the nodes of the while and do_while statements are connected to the nodes of the conditional expressions they are checking. For goto statements, the nodes of the goto statement and the target statement to which the goto statement jumps are connected. For compound statements, the nodes of all statements in the code block corresponding to a compound statement are connected in pairs from front to back in order. MayExe edges represent edges that the program might execute, using... This indicates that it is used to process the switch and if statements in C language syntax. For the if statement, it is the node pointing from the node of the if statement to the node of the possible branches of the if statement. For switch statements, it is from the node of the switch statement to the nodes of the possible branches of each case statement; S4. Construct a deep learning model based on a gated graph neural network; specifically, the deep learning model based on the gated graph neural network is as follows: Each node of the code information graph is initialized as a vector representation through a node initialization module, and then fed into the gated graph neural network module to learn the features of the code information graph. An attention mechanism is then used after the gated graph neural network module to... The vector representations of each node are compressed into a single vector, and this feature vector is then fed into a multilayer linear perceptron module to learn the vector representation. Finally, the Softmax function is used to predict the proportions of four CPU performance metrics. The gated graph neural network module uses a gated graph neural network as its basic module, and a total of [number missing] modules are stacked. layer; In the node initialization module, two different initialization strategies are designed for two types of nodes in the abstract syntax tree: terminal symbols and non-terminal symbols; Non-terminal symbols represent composite syntax structures in the program, and only have syntax structure type information, while terminal symbols have lexical unit or terminal symbol information; For non-terminal symbols, randomly initialize the vector representation according to the syntax structure type; For terminal symbols, use the pre-trained model Unixcoder to tokenize the lexical unit or terminal symbol and obtain the semantic vector representation of each token, and finally the sum of the semantic vectors of multiple tokens represents the initialization; S5. Train the deep learning model based on the code information graph; S6. Predict the indicators related to the CPU performance bottleneck of C functions under the same architecture as the dataset.

2. The method for predicting CPU performance metrics based on a gated graph neural network according to claim 1, characterized in that, The specific steps of S1 are as follows: S1.

1. Mine open-source C language projects: Select relevant C language projects with high popularity, clear construction processes, and rich tests; S1.

2. Build the projects: Build the selected C language projects; S1.

3. Run test cases and perform performance analysis Run the test cases of the C language project and collect the relevant indicators of C functions; During the sampling collection process, select the average value of multiple results as the final prediction label; S1.

4. C function extraction and dataset division Extract functions with corresponding performance analysis results of relevant indicators from the source code, and collect data in the format of <C function, performance indicator>; Divide the collected data into a training set, a validation set, and a test set.

3. The method for predicting CPU performance indicators based on a gated graph neural network according to claim 2, characterized in that, The indicators related to the CPU performance bottleneck in S1 are the highest-level indicators of four top-down microarchitecture analysis methods, including: Retiring: Represents the proportion of the number of completed instructions, Back-End Bound: Represents the proportion of the processor in the waiting idle state due to the back end of the processor being unable to execute the micro-operations sent by the front end in time; Front-End Bound: Represents the proportion of the processor in the waiting idle state due to the slow instruction fetching and decoding speed of the front end of the processor; Bad Speculation: Represents the proportion of the analysis workload affected by incorrect execution paths.

4. The method for predicting CPU performance indicators based on gated graph neural networks according to claim 1, characterized in that, The specific construction of the abstract syntax tree is as follows: An abstract syntax tree (AST) consists of nodes and edges. Nodes are further subdivided into terminal symbols and non-terminal symbols. Terminal symbols are symbols in the syntax rules that cannot be further expanded or decomposed, representing specific lexical units or terminal symbols in the program. Non-terminal symbols are symbols that can be further expanded or decomposed, representing complex syntactic structures in the program, represented by notations. The abstract syntax tree (API) represents all the nodes in the abstract syntax tree; the edges of the API represent the relationships between the nodes. This represents the original edge on the abstract syntax tree.

5. The method for predicting CPU performance metrics based on a gated graph neural network according to claim 1, characterized in that, The specific steps for incorporating data flow information into the abstract syntax tree are as follows: Two types of edges representing data flow were added to the abstract syntax tree of the code so that the model could learn relevant information about the data flow: NextUse and ComputFrom. Both types of edges have terminal symbols representing variables in the abstract syntax tree on both sides. NextUse extracts the data flow relationship between the same variable. The ComputFrom method extracts the data flow relationship between the same variable or different variables. express.

6. The method for predicting CPU performance metrics based on a gated graph neural network according to claim 1, characterized in that, Introducing NCS edges to represent the sequence information of the original code: In an abstract syntax tree, the sequence information of the code is reflected in the terminal symbols. Therefore, a preorder traversal of the abstract syntax tree is performed to extract the positions of all terminal symbols, and they are connected in pairs from front to back in order.

7. The method for predicting CPU performance metrics based on a gated graph neural network according to claim 1, characterized in that, Mini-batch stochastic gradient descent is used to minimize the loss function: a fixed number of samples are randomly selected in each iteration of the training round to compute gradients and update the model; the defined loss function is the mean absolute error. Representing the true values ​​of the four performance metrics for each sample, using Let represent the predicted values ​​of the four performance metrics for each sample; then the loss for each sample is expressed by the following formula: in For the loss of this sample, The model's predicted value The first in One value, for The Values.

Images