A Method and System for Predicting the Performance of Convolutional Neural Networks Based on Hardware Interaction Features

Abstract

This invention discloses a method and system for predicting the performance of convolutional neural networks (CNNs) based on hardware interaction features, relating to the fields of artificial intelligence and computer system performance optimization. The method involves obtaining hierarchical features of a CNN model; calculating interaction features based on these hierarchical features and the hardware specifications of the target hardware platform; using a Transformer model with a bias matrix for prediction; generating a hardware-aware bias matrix after processing the interaction features by a hardware-aware attention bias module, and then inputting this matrix into the Transformer encoder; and finally, after sequential processing by an input encoder, a Transformer encoder, and a linear decoding layer, predicting the runtime of the CNN model. This invention predicts the inference and training time of a CNN model through static analysis of its architecture and target hardware specifications.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and computer system performance optimization technology, and in particular to a method and system for predicting the performance of convolutional neural networks based on hardware interaction features. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] With the widespread application of artificial intelligence in industries such as manufacturing, healthcare, and finance, neural network models, especially convolutional neural networks (CNNs), are being deployed extensively in servers, edge devices, and embedded systems. In practical deployments, the inference and training times of these models become key indicators for measuring system response speed and hardware load capacity. Due to the increasingly complex structure of CNN models and their enormous computational resource consumption, inference and training times directly impact user experience, system throughput, and resource scheduling strategies. Therefore, accurately predicting the runtime of a model on the target platform before deployment has become a core issue in model development and system design.

[0004] Currently, common methods for evaluating the runtime of neural networks mainly rely on actual runtime testing or simulation analysis. These methods are not only time-consuming but also require access to the deployment environment or specific hardware platform, limiting their application in the design phase. Furthermore, since the runtime of neural networks is affected by various factors, including model structure, input size, hierarchical features, and platform scheduling behavior, relying solely on traditional model hierarchy metrics is insufficient to accurately characterize runtime characteristics.

[0005] Most existing convolutional neural network performance prediction methods rely on hierarchical features that are independent of hardware specifications, which cannot reveal the complex performance bottlenecks caused by the interaction between model architecture and hardware resources, thus resulting in limited prediction accuracy. Furthermore, existing methods mostly use black-box machine learning models, which lack effective prior knowledge mechanisms and cannot fully capture the nonlinear relationship between time and input features and target platform hardware specifications, leading to low model learning efficiency and insufficient stability in complex scenarios. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides a method and system for predicting the performance of convolutional neural networks based on hardware interaction features, which predicts the inference and training time of the convolutional neural network model by statically analyzing the model architecture and target hardware specifications.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] In a first aspect, the present invention provides a method for predicting the performance of convolutional neural networks based on hardware interaction features, comprising:

[0009] Identify the target hardware platform, select a convolutional neural network model, and obtain the hierarchical features of the convolutional neural network model;

[0010] Based on the aforementioned hierarchical features, and combined with the hardware specifications of the target hardware platform, the interaction features are calculated through joint analysis.

[0011] The hierarchical features are input into a Transformer model with a bias matrix for processing, and the interaction features are input into the hardware-aware attention bias module of the model for processing. Finally, the model predicts the running time of the convolutional neural network model.

[0012] The construction process of the biased attention mechanism model includes:

[0013] The interaction features are processed by the hardware-aware attention bias module to generate a hardware-aware bias matrix, which is then input into the Transformer encoder. The hierarchical features are processed sequentially by the input encoder, the Transformer encoder, and the linear decoding layer to obtain the predicted runtime.

[0014] A further technical solution is that the hierarchical features include the total number of output activation elements, the total number of input tensor elements, the total number of floating-point operations, the total number of parameters, the total number of layers, the interaction complexity, and the maximum channel complexity.

[0015] Further technical solutions describe interactive features including computational intensity, model parameter quantity and GPU memory matching degree, and parallel computation granularity.

[0016] A further technical solution defines the computational intensity as the ratio of the total number of floating-point operations to the total number of memory accesses in one forward propagation of the convolutional neural network model, expressed as:

[0017]

[0018] in, This indicates the number of floating-point operations required by the model to process each byte of memory data. This represents the total number of floating-point operations required for the model to perform one forward propagation. This represents the total number of parameters in the model. This represents the total number of input tensor elements across all computational layers of the model. This represents the total number of output activation elements across all computational layers of the model. This indicates the data precision factor.

[0019] In a further technical solution, the hardware-aware attention bias module calculates the bias intensity scalar of the interaction features, multiplies the bias intensity scalar with the corresponding bias matrix to obtain their respective final bias matrices, and sums the final bias matrices to obtain the hardware-aware bias matrix.

[0020] A further technical solution describes the formula for calculating the bias intensity scalar of the model parameter quantity and memory matching degree in the interaction features:

[0021]

[0022] in, This represents a scalar value indicating the bias strength corresponding to the memory matching degree. Indicates input features The standardized value This represents the Sigmoid activation function. and Represents the learnable parameters. and This represents the learnable linear transformation parameters.

[0023] A further technical solution is that the score calculation process for multi-head self-attention in the Transformer encoder is expressed as follows:

[0024]

[0025] in, , These represent the query matrix and key matrix in the self-attention mechanism, respectively. Indicates transpose. Let B represent the dimension of the key vector, and let B represent the hardware-aware bias matrix.

[0026] Secondly, the present invention provides a convolutional neural network performance prediction system based on hardware interaction features, comprising:

[0027] The data acquisition module is configured to: determine the target hardware platform, select a convolutional neural network model, and obtain the hierarchical features of the convolutional neural network model;

[0028] The hardware interaction module is configured to perform joint analysis based on the hierarchical features and the hardware specifications of the target hardware platform to calculate the interaction features.

[0029] The model prediction module is configured to: input the hierarchical features into a Transformer model with a bias matrix for processing, and simultaneously input the interaction features into the hardware-aware attention bias module of the model for processing, and finally predict the running time of the convolutional neural network model.

[0030] The construction process of the biased attention mechanism model includes:

[0031] The interaction features are processed by the hardware-aware attention bias module to generate a hardware-aware bias matrix, which is then input into the Transformer encoder. The hierarchical features are processed sequentially by the input encoder, the Transformer encoder, and the linear decoding layer to obtain the predicted runtime.

[0032] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the convolutional neural network performance prediction method based on hardware interaction features as described in the first aspect.

[0033] Fourthly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the convolutional neural network performance prediction method based on hardware interaction features as described in the first aspect.

[0034] The above one or more technical solutions have the following beneficial effects:

[0035] This invention designs interaction features between the convolutional neural network model and the hardware, using both hierarchical features and interaction features as inputs to the prediction model. This overcomes the limitation of relying solely on isolated hierarchical features, enabling the prediction model to understand the physical root causes of performance bottlenecks at a more fundamental level and improving the accuracy of convolutional neural network performance prediction. Furthermore, the prediction model employs a Transformer model with a bias matrix. By introducing the bias matrix, dynamic modulation of attention weights between hierarchical features is achieved, further enhancing prediction accuracy.

[0036] This invention designs interactive features, using both hierarchical and interactive features of the convolutional neural network model as input to the prediction model—a key design element. This invention overcomes the limitations of relying solely on isolated hierarchical features. By introducing computational intensity, model parameter quantity and memory matching degree, and parallel computation granularity, it scientifically quantifies the interaction between the computational memory access patterns of the convolutional neural network model and hardware specifications. This allows the prediction model to move beyond simply learning shallow correlations on the data surface, instead understanding the physical root causes of performance bottlenecks at a more fundamental level. This enables the model to capture the nonlinear performance changes dominated by these complex factors, ultimately achieving a qualitative improvement in both prediction accuracy and physical interpretability.

[0037] This invention designs a Transformer model with a bias matrix, which introduces a physical prior-based bias matrix into the standard self-attention mechanism. This bias matrix is ​​directly superimposed on the standard attention score matrix, thereby realizing the dynamic modulation of attention weights between hierarchical features. This enables the prediction model to adaptively enhance its sensitivity to key features while suppressing the influence of irrelevant or secondary features when performing feature fusion. This design can improve the convergence efficiency, prediction accuracy, and robustness of the prediction model in complex scenarios.

[0038] All input features of the prediction model in this invention can be obtained through purely static analysis, without the need for any time-consuming dynamic profiling on the target hardware. This characteristic makes it a highly efficient and low-cost tool, and the introduction of a bias matrix further enhances the stability of the prediction model. More importantly, because it learns the more universal physical interaction between the model and the hardware, the prediction model of this invention also exhibits excellent generalization ability for novel network architectures not seen in the training set. Attached Figure Description

[0039] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0040] Figure 1 This is a flowchart of the convolutional neural network performance prediction method based on hardware interaction features according to an embodiment of the present invention;

[0041] Figure 2 This is a flowchart illustrating the calculation of the hardware-aware bias matrix according to an embodiment of the present invention;

[0042] Figure 3 This is a diagram showing the experimental results of an embodiment of the present invention. Detailed Implementation

[0043] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0044] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0045] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0046] Example 1

[0047] like Figure 1 As shown, this embodiment discloses a method for predicting the performance of a convolutional neural network based on hardware interaction features. The method includes the following steps:

[0048] S1: Determine the target hardware platform, select a convolutional neural network model, and obtain the hierarchical features of the convolutional neural network model;

[0049] In this embodiment, a systematic data acquisition method is employed to obtain the forward execution time and structural indicators (hierarchical features) of the neural network model under different input scales, providing high-quality training and testing samples for subsequent interactive feature design and transformer modeling. This achieves data acquisition for both the inference and training stages of the convolutional neural network, providing training and testing samples for subsequent prediction models, thus enabling accurate modeling of convolutional neural networks of different models and scales.

[0050] The data acquisition process begins by identifying a target hardware platform and selecting a set of representative convolutional neural network models of varying complexity. Specifically, a set of mainstream network models is loaded sequentially from a standard model library containing pre-trained models. The standard model library uses torchvision, a computer vision toolkit provided by PyTorch that includes pre-trained mainstream neural network models.

[0051] To ensure data diversity and generalization ability, automated scripts were used to iteratively test each selected model under various preset runtime configurations. These configurations primarily included a series of varying batch sizes and different input image sizes to comprehensively simulate different load conditions in real-world application scenarios. Specifically, different batch sizes and image resolutions were set for each model to simulate actual deployment scenarios. Furthermore, when running neural networks on a GPU, the first inference run typically takes a long time because the GPU's cache and memory management are not yet fully utilized. Running multiple inference processes in advance warms up the GPU, thereby reducing timing errors during the first inference run.

[0052] A high-precision GPU native timing tool is used to accurately record the time taken for multiple repetitions of the forward propagation, and the median is taken as the stable and reliable actual running time under this configuration. Specifically, the GPU's native high-precision timing tool (cuda.Event tool is used in this embodiment) is used to insert time markers at the entry and exit points of the GPU's forward inference execution, and the median is taken as a more stable inference time indicator through multiple repetitions.

[0053] For the hierarchical features of convolutional neural networks, the model architecture is analyzed using standard libraries and hook functions. These two techniques are employed to obtain the model's hierarchical features. Hook functions are callback functions registered on neural network layers, automatically triggered during forward or backward propagation, used to capture intermediate data or perform specific operations. By registering hook functions and using the standard library ptflops, the input and output feature tensors of all convolutional layers in the model are captured, and key structural features are statistically analyzed and calculated. The collected hierarchical features include:

[0054] (1) Output the total number of active elements: Activations This represents the total number of output tensor elements of all convolutional layers in the neural network.

[0055] (2) Input the total number of tensor elements: This represents the total number of elements in the input tensor across all convolutional layers of the neural network. For a given model architecture, the size of the input tensor is typically fixed, represented as:

[0056]

[0057]

[0058] in, It is the set of target computational layers, typically all convolutional layers; It is the first The input and output tensors received by the layer; `C` is a function that calculates the total number of elements in a tensor. For example, for a four-dimensional tensor `[BatchSize, Channels, Height, Width]`, `B` (BatchSize) represents the batch size, indicating the number of samples processed simultaneously in one forward propagation; `C` (Channels) represents the number of feature maps per sample; for an input RGB image, `C` is 3; for intermediate layers of the network, `C` is the depth of the feature map of that layer; `H` (Height) represents the number of pixels in the vertical dimension of the feature map; and `W` (Width) represents the number of pixels in the horizontal dimension of the feature map. The calculation method is as follows .

[0059] (3) Total floating-point operations (FLOPs): The total number of floating-point operations in all layers is obtained by traversing all layers that perform calculations in the model and summing the FLOPs of each layer. It is expressed as:

[0060]

[0061] in, It represents the total number of floating-point operations performed by the model. This represents a computational layer in the model. It is the set of all layers in the model that perform computational operations (the target computational layer set), mainly including convolutional layers and fully connected layers; It is the first The number of floating-point operations performed by the layer. For example, for a standard two-dimensional convolutional layer with an output feature map size of HH, the number of its input channels... Number of output channels kernel size The height and width of the output feature map from the convolution operation. FLOPs can be approximated as:

[0062] .

[0063] (4) Total number of parameters: The total number of trainable parameters in the model. The total number of parameters can be obtained by iterating through all layers containing trainable parameters and summing the number of parameters in each layer, as follows:

[0064]

[0065] in, This represents the total number of parameters in a convolutional neural network; This represents a specific layer in the model; It is the first The number of parameters in a layer. For example, the number of parameters in a convolutional layer can be calculated as follows:

[0066]

[0067] in, This indicates the number of parameters in the convolutional layer.

[0068] (5) Total number of layers: The number of layers in the model The larger the value, the more complex the network structure and the higher the network depth.

[0069] (6) Interaction Complexity Mixed: represents the sum of the product of the input tensor size and the number of output tensor channels of the convolutional layer, and characterizes the interaction complexity between the input and output feature maps.

[0070] (7) MaxChannels: represents the maximum number of input and output channels of each convolution layer multiplied by the sum of the input tensor size, characterizing the impact of channel dimension on memory and computation.

[0071] For data collection during the training phase, a similar approach to the inference phase was used to measure the forward propagation, backpropagation, and gradient update processes. The execution time and hierarchical features of the Forward, Backward, and Optimizer phases were recorded separately, and combined to derive the complete training step duration. This process was also executed multiple times to ensure the representativeness and reliability of the time data.

[0072] Through the above process, data records with actual time and hierarchical features are generated for each configuration of the convolutional neural network. The operation of the convolutional neural network is divided into an inference phase and a training phase. Data acquisition during the inference phase includes data acquisition for the forward propagation part. A similar systematic approach is used for data acquisition during the training phase, with precise timing and feature acquisition performed for independent stages such as forward propagation, back propagation, and optimizer updates.

[0073] S2: Based on hierarchical features, combined with the hardware specifications of the target hardware platform, a joint analysis is performed to calculate the interaction features;

[0074] In this embodiment, to address the problem of insufficient prediction accuracy caused by relying solely on the hierarchical structure features of the convolutional neural network model in existing technologies, the present invention mathematically couples the hierarchical structure of the convolutional neural network model with the hardware parameters of the platform to generate three derived features that better reveal performance bottlenecks.

[0075] By jointly analyzing model features (hierarchical features) and the hardware specifications of the target hardware platform, derived features (interaction features) that deeply reflect the interaction between the two and have greater physical meaning are designed and calculated. These derived features enable the predictive model to understand the root cause of performance bottlenecks. The design and calculation of the following three key interaction features are included:

[0076] (1) Calculate the strength

[0077] Computational intensity is represented by the number of floating-point operations required for the model to process each byte of memory data, enabling the predictive model to understand performance bottlenecks at a more fundamental level. Computational intensity is defined as the ratio of the total number of floating-point operations to the total number of bytes accessed in one forward propagation of a convolutional neural network model, expressed as:

[0078]

[0079] in, This represents the number of floating-point operations required for the model to process each byte of memory data, expressed in FLOPs / Byte. This represents the total number of floating-point operations (FLOPs) required for the model to perform one forward propagation. This represents the total number of parameters in the model. Inputs represents the total number of input tensor elements across all computational layers of the model. Activations represents the total number of output activation elements across all computational layers of the model. This represents the data precision factor, which is the number of bytes occupied by each numerical element in the model, for standard 32-bit floating-point numbers (FP32). The value of is 4.

[0080] (2) Matching degree between model parameters and video memory

[0081] The static volume of the quantization model impacts the GPU memory usage of the target hardware platform. The model parameter count-to-memory fit is defined as the ratio of the total number of bytes occupied by all model parameters to the total GPU memory capacity of the target hardware, expressed as:

[0082]

[0083] in, This represents the proportion of model parameters to the total GPU memory. This indicates the total number of parameters in the model; This represents the data precision factor, which is the number of bytes occupied by each parameter; it is usually set to 4. This represents the total video memory capacity of the target hardware platform's GPU, in bytes, and is a predetermined hardware specification constant.

[0084] (3) Parallel computing granularity

[0085] Parallel computing granularity is the ratio of the total number of floating-point operations in the model to the number of layers and the number of hardware parallel units. It represents the average number of floating-point operations that each layer of the model is allocated to on each streaming multiprocessor (SM) of the GPU.

[0086] GPUs achieve high-performance computing through a large number of computing cores, such as streaming multiprocessors (SMs). If a model's computational tasks at any given time can be fully parallelized across all SMs, its overall execution efficiency will be significantly improved. This characteristic measures whether the workload of each computational layer of the model can fully utilize the parallel units of the GPU, and is expressed as:

[0087]

[0088] in, This indicates the granularity of parallel computing, representing the average amount of floating-point operations distributed across each layer of the model on each streaming multiprocessor (SM) of the GPU. This represents the total number of streaming multiprocessors in the target hardware GPU. This is a hardware-specific constant and must be determined in advance.

[0089] S3: The hierarchical features are input into the Transformer model with a bias matrix for processing, and the interaction features are input into the hardware-aware attention bias module of the model for processing. Finally, the model predicts the running time of the convolutional neural network model. The construction process of the biased attention mechanism model includes: the interaction features are processed by the hardware-aware attention bias module to generate a hardware-aware bias matrix, which is then input into the Transformer encoder; the hierarchical features are processed sequentially by the input encoder, the Transformer encoder, and the linear decoding layer to obtain the predicted running time.

[0090] In this embodiment, to address the technical pain point of traditional machine learning models being black boxes and unable to effectively incorporate domain prior knowledge, as well as the problem of unstable prediction results, this invention further proposes an attention calculation method based on hardware-aware bias. This invention directly injects the interaction characteristics between the model and hardware into the Transformer attention calculation process in the form of a bias matrix, guiding the model to capture performance bottlenecks more efficiently and accurately. The main idea of ​​this design is to introduce a hardware-aware bias matrix into the score calculation process of the standard Transformer self-attention mechanism. The hardware-aware attention bias module calculates the bias matrix based on the input interaction features. The interaction features are used to calculate the bias matrix, and the hierarchical features are used as input to the prediction model. The bias matrix is ​​then superimposed on the original attention score. By using a prediction model with a bias matrix, the accuracy and stability of the model's predictions can be further improved.

[0091] To address the issue of unstable prediction accuracy in prediction models and to achieve accurate prediction of hardware interaction features and hierarchical features from model to runtime, this invention designs a hardware-aware Transformer model (prediction model) by adding an attention bias matrix, which can inject physical prior knowledge into the standard self-attention mechanism.

[0092] (1) Attention bias matrix

[0093] like Figure 2 As shown, in the hardware-aware attention bias module, the input data is the interaction features. Based on the current input interaction feature values, a bias matrix is ​​generated to correct the attention score. First, for the three interaction features—model parameter count and GPU memory matching degree—[the following steps are taken]. Calculation intensity and parallel computing granularity Calculate a bias intensity scalar for each. Taking the model parameter quantity and memory matching degree as an example, the formula for calculating its intensity scalar is as follows:

[0094]

[0095] in, It is the scalar value of the bias intensity corresponding to the memory matching degree; Input features The standardized value; It is the Sigmoid activation function, which maps the input to the (0, 1) interval; and These are learnable parameters that control the slope and threshold of the Sigmoid function, respectively. and These are learnable linear transformation parameters used to scale and translate the output of the Sigmoid function to control the magnitude and range of the final bias intensity. The bias intensity scalar is used to calculate the intensity. and the bias intensity scalar of parallel computation granularity It is also calculated using a formula with the same structure but independent parameters, expressed as:

[0096]

[0097] .

[0098] Each bias strength scalar is multiplied by a learnable, feature-specific bias matrix. For example, the final bias matrix corresponds to the model parameter count and memory matching. The calculation is as follows:

[0099]

[0100] in, It is a learnable bias matrix with the shape (n_heads, n_features, n_features), where n_heads is the number of attention heads in the Transformer model, and n_features is the total number of input features. The bias matrix is ​​used to calculate the intensity. and the bias matrix of parallel computing granularity The structures are identical, but the formulas are independent. Ultimately, the total hardware-aware bias matrix B is obtained by adding the three independent bias matrices:

[0101]

[0102] The shape of matrix B is ([batch_size, n_heads, n_features, n_features]), where batch_size is the batch size of the transformer prediction model.

[0103] (2) Transformer model with bias matrix (biased Transformer)

[0104] The overall prediction model architecture of this invention adopts a feature self-attention mode, that is, treating N input features as a sequence of length N for processing. The Transformer model includes a hardware-aware attention bias module and an input encoder in parallel. After the input encoder, a Transformer encoder and a linear decoding layer are set in sequence. The output of the hardware-aware attention bias module, namely the hardware-aware bias matrix B, is injected into the calculation process of the attention score of the Transformer encoder.

[0105] The input is a sample vector containing N features. (Hierarchical features) are first mapped and reshaped into a feature sequence through an input encoder (linear layer). ,in This is the hidden dimension of the model. The processed feature sequence. It is fed into a custom Transformer encoder containing L layers. In each layer of the encoder, the core multi-head self-attention score calculation process is modified as follows:

[0106]

[0107] in, , These are the query and key matrices in the self-attention mechanism, respectively. Indicates transpose; B is the dimension of the key vector, preventing gradient explosion and gradient vanishing problems; B is the hardware-aware bias matrix.

[0108] After processing by the L-layer Transformer encoder, a new feature sequence is obtained by fusing contextual information. This sequence is flattened and fed into a final linear decoding layer, mapped to a single scalar value, which is the predicted runtime.

[0109] By directly superimposing B on In addition, this invention enables learnable guidance of attention weights between features based on the physical meaning of the input features.

[0110] (3) Training and prediction of Transformer model with bias matrix

[0111] Two independent prediction models are used for the inference and training phases, each with its own dataset. First, the N features obtained from the datasets are explicitly divided into two logically independent subsets: hierarchical feature vectors and hardware interaction feature vectors. The hierarchical feature vectors contain fundamental features describing the model's size and structure; these vectors serve as input for context encoding in the transformer model. The hardware interaction feature vectors contain three interaction features used to generate the bias matrix, guiding the attention mechanism. Each dataset is then divided into training, validation, and test sets, and the hierarchical and interaction feature sets are standardized separately.

[0112] In each training batch of the prediction model, the model's forward function simultaneously receives standardized hierarchical features and hardware interaction features. The hardware interaction features are fed separately into a hardware-aware attention bias module. This module dynamically calculates and generates a final attention bias matrix B based on the real-time values ​​of these three features using an internal learnable function. Simultaneously, the hierarchical features are fed into the model's input encoder and processed into a feature sequence, serving as the backbone input to the Transformer encoder. During self-attention computation within the Transformer encoder, the generated bias matrix B is injected into the attention score calculation process, ensuring that the model's calculation of the correlations between major hierarchical features is guided by the attention score with the bias matrix. After processing through the biased attention mechanism, the model outputs the final runtime prediction.

[0113] The loss between the predicted runtime and the actual runtime is calculated and backpropagated. This gradient simultaneously updates all learnable parameters in the Transformer backbone and bias generation module, thereby optimizing the entire hardware-aware prediction system. To ensure the model's prediction accuracy, four evaluation metrics are used to assess its performance. Through comprehensive evaluation of these metrics, the final selected model is ensured to have high prediction accuracy and stability.

[0114] Experimental instructions

[0115] (1) Experimental environment

[0116] On the NVIDIA A100-SXM4-40GB, the number of streaming multiprocessors (SM) is 108, using PyTorch with CUDA, PyTorch version 2.5.1, and Python version 3.10.

[0117] (2) Experimental Case

[0118] Nine typical CNN models were tested, covering lightweight networks, large residual networks, and classic architectures. Image sizes were 64 and 128 pixels, batch sizes ranged from 16 to 2048, and stride was 16.

[0119] (3) Experimental steps

[0120] 1) Collect basic data. On the target GPU, with 9 representative neural network models, different scale configurations were traversed through using automated scripts. Static analysis tools were used to obtain the hierarchical features (such as FLOPs, number of parameters, activation values, etc.) and actual runtime of each model configuration.

[0121] 2) Hardware interaction features of the computational model. Based on the collected hierarchical features and combined with known hardware specifications (such as VRAM capacity and number of SMs), three interaction features are calculated: “computational intensity”, “matching degree between model parameters and video memory”, and “parallel computation granularity”.

[0122] 3) Training the Transformer model. Hierarchical features and interaction features are used as input. Interaction features are used to calculate the bias matrix, while hierarchical features are used as input to the prediction model. After standardizing the data, a Transformer model with an attention mechanism and bias matrix is ​​used to model both the inference and training phases of the neural network to predict time.

[0123] 4) Performance evaluation and verification. Compare the predicted time with the actual time on an independent test set and calculate the error.

[0124] (4) Experimental results

[0125] Table 1 shows the prediction error values ​​for the four models in the inference and training phases. `inf` represents the inference phase, where the average MapE of the nine convolutional neural network predictions is only about 3.8%. `train` represents the training phase, where the average MapE of the nine convolutional neural network predictions is only about 4.9%.

[0126] Table 1. Prediction error values ​​for four specific models during the inference and training phases.

[0127]

[0128] The coefficient of determination is RMSE, the root mean square error is NRMSE, the normalized root mean square error is MAPE, and the mean absolute percentage error is MAPE.

[0129] like Figure 3As shown, the average mape was measured after 50 repeated runs, and the errors of the prediction model using the bias matrix and the prediction model without the bias matrix were compared. The blue line at the top represents the prediction model without the bias matrix, and the yellow line at the bottom represents the prediction model using the bias matrix. It can be seen that the prediction model using the bias matrix has an error range of 2.9%-3.9%, while the prediction model without the bias matrix has an error range of 5.8%-18.3%. The prediction model with the bias matrix not only has a smaller error but also improved stability.

[0130] In summary, this invention aims to address the technical problem of insufficient prediction accuracy in existing convolutional neural network model performance prediction methods, which rely on macroscopic hierarchical features and fail to fully consider the interaction between model architecture and hardware resources. To this end, this invention first constructs a feature space containing three types of interactive features: computational intensity, parameter quantity and memory capacity matching degree, and parallel computing intensity. Second, it generates an attention bias matrix based on this feature space and incorporates it into the attention score calculation process. Finally, it uses a Transformer model integrating this bias matrix attention calculation method for time prediction, thereby achieving high-precision prediction of the neural network's runtime on specific hardware. The parameters required for the prediction process of this invention are obtained statically. When predicting the performance of a new convolutional neural network model, only its architecture definition needs to be statically analyzed; no actual operation or dynamic profiling on the target hardware is required.

[0131] Example 2

[0132] This embodiment discloses a hardware-interactive convolutional neural network performance prediction system, including:

[0133] The data acquisition module is configured to: determine the target hardware platform, select a convolutional neural network model, and obtain the hierarchical features of the convolutional neural network model;

[0134] The hardware interaction module is configured to perform joint analysis based on the hierarchical features and the hardware specifications of the target hardware platform to calculate the interaction features.

[0135] The model prediction module is configured to: input the hierarchical features into a Transformer model with a bias matrix for processing, and simultaneously input the interaction features into the hardware-aware attention bias module of the model for processing, and finally predict the running time of the convolutional neural network model.

[0136] The construction process of the biased attention mechanism model includes:

[0137] The interaction features are processed by the hardware-aware attention bias module to generate a hardware-aware bias matrix, which is then input into the Transformer encoder. The hierarchical features are processed sequentially by the input encoder, the Transformer encoder, and the linear decoding layer to obtain the predicted runtime.

[0138] Example 3

[0139] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method of Embodiment 1.

[0140] Example 4

[0141] The purpose of this embodiment is to provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method of Embodiment 1.

[0142] The steps and methods involved in the apparatuses of Embodiments 3 and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0143] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0144] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0145] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for predicting performance of a convolutional neural network based on hardware interaction characteristics, the method comprising: The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. 2.The hardware interaction feature based convolutional neural network performance prediction method of claim 1, wherein, The application relates to a convolutional neural network model running time prediction method and device. 3.The hardware interaction feature based convolutional neural network performance prediction method of claim 1, wherein, The application relates to a convolutional neural network model running time prediction method and device. wherein, represents the number of floating point operations performed by the model per byte of memory data, represents the total number of floating point operations required by the model to perform one forward pass, represents the total number of parameters of the model, represents the total number of input tensor elements of all computational layers of the model, represents the total number of output activation elements of all computational layers of the model, represents the data precision factor. 4.The hardware interaction feature based convolutional neural network performance prediction method of claim 1, wherein, The application relates to a convolutional neural network model running time prediction method and device. wherein, represents the bias strength scalar corresponding to the memory match degree, represents the value after standardization of the input feature , represents the Sigmoid activation function, and represents the learnable parameter, and represents the learnable linear transformation parameter. 5.The hardware interaction feature based convolutional neural network performance prediction method of claim 1, wherein, The application relates to a convolutional neural network model running time prediction method and device. wherein, , denote the query matrix and the key matrix in the self-attention mechanism, respectively, denotes the transpose, denotes the dimension of the key vector, and B denotes a hardware-aware bias matrix.

6. A system for predicting performance of a convolutional neural network based on hardware interaction characteristics, the system comprising: The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural network model running time prediction method and device. The application relates to a convolutional neural The interaction feature is processed by a hardware-aware attention biasing module to generate a hardware-aware biasing matrix, which is input into a Transformer encoder; and the hierarchical feature is processed by an input encoder, a Transformer encoder and a linear decoding layer in sequence to obtain a predicted running time.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements the steps in the hardware interaction feature-based convolutional neural network performance prediction method of any one of claims 1-5.

8. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor, when executing the program, implements the steps in the hardware interaction feature-based convolutional neural network performance prediction method of any one of claims 1-5.

Images