Deep learning model reasoning stage computing power resource demand prediction method and system

Abstract

The invention provides a deep learning model reasoning stage computing power resource demand prediction method and system, and belongs to the field of artificial intelligence, and the method comprises the steps: S1, analyzing a deep learning model file, and extracting node features and node connection information of a computational graph corresponding to the model file; s2, constructing and training a computing power resource demand prediction model based on a convolution attention mechanism and data smoothing processing; and S3, testing the performance of the model by using the samples in the test set, and storing a predicted performance index result in a document. According to the method, on the basis of hyper-parameter features of a deep learning model, a deep learning model feature extraction method based on an inductive computational graph is selected and used, a feature aggregation layer based on a convolutional attention mechanism is designed, and deep learning model features which contribute more to computing power resource demands are focused on; and data smoothing operation is adopted, so that the influence of the problems of data distribution imbalance and numerical value missing on the prediction precision is effectively relieved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of artificial intelligence, and particularly relates to a deep learning model inference stage computing resource demand prediction method and system. BACKGROUND

[0002] As a representative of artificial intelligence technology, deep learning has been widely applied to many different fields. With the continuous evolution of technology and the increasing diversification and complexity of practical application scenarios, various types of deep learning models designed and developed for specific scenarios are emerging. Different deep learning models have significant differences in the degree of hardware resource consumption during actual operation. The computing resource demand of a deep learning model includes multiple key indicators during the operation of the model on a hardware platform. Memory consumption is the main research object of computing resource demand, and directly affects the efficiency of the model's consumption of system memory. Model parameter quantity, intermediate data size, and temporary variables during calculation will significantly restrict the memory usage efficiency during operation. In cloud server application scenarios, when the user's local computing power cannot support large-scale model deployment, service providers need to dynamically provide heterogeneous virtual computing resources - different virtual machines have inherent differences in performance configuration, cost structure, and resource allocation. This not only brings cost optimization problems to users, but also puts higher requirements on the resource scheduling capabilities of service providers.

[0003] The inference delay of the model is another key indicator of computing resource demand. In real-time sensitive scenarios such as autonomous driving, microsecond-level delays in object detection and recognition tasks can cause serious consequences such as control failure, and the execution time must be compressed within a strict threshold to ensure the timeliness and reliability of system decision-making. In addition, hardware utilization rate and energy consumption ratio are also important categories of computing resource research.

[0004] Accurate prediction of the computing resource demand of a deep learning model is the premise and foundation for solving the above problems. Therefore, the prediction of the computing resource demand of a deep learning model is of great significance for the deployment and application of the model, and has been a research hotspot in recent years.

[0005] For the memory consumption of the model, the computing resource demand prediction technology can help users accurately evaluate the memory consumption of the model, provide key basis for hardware selection, and thus scientifically select suitable hardware devices, and in-depth analyze the performance of the model in actual operation, effectively reducing the training cost. For computing service providers, this technology can guide resource allocation strategies, optimize the scheduling and utilization of computing resources, significantly improve resource utilization, reduce the construction and operation investment of computing infrastructure, and ultimately maximize operational efficiency.

[0006] For the inference delay of the model, the computing resource demand prediction technology can accurately estimate the runtime delay of the model on the mobile edge server. For example, in an autonomous driving system, the technology can verify in advance whether the model meets the strict time delay requirements of real-time detection and decision-making, providing reliable technical support for traffic safety. In addition, for the performance requirements of different application scenarios, such as high-throughput computing tasks or low-latency inference tasks, the technology can also assist developers in optimizing model architecture and deployment schemes to ensure that the system can still run stably in resource-constrained environments.

[0007] Due to the limitations of the use scenarios, the prediction accuracy is low in some cases, and more universal and general computing resource demand prediction methods need to be studied. At present, the deep learning model computing resource demand prediction method has transitioned from the traditional model based on model super parameter features to the artificial intelligence related method represented by machine learning and deep learning. In recent years, deep learning has developed rapidly. It not only shines in the fields of computer vision and natural language processing, but also opens up more possibilities, thereby expanding its application range, especially in the analysis of deep learning model computing resource demand.

[0008] Compared with machine learning algorithms, the advantage of deep learning algorithms is that they can integrate the features of each layer of the input model file and the interlayer connection information, and are more suitable for diversified model structure computing resource demand prediction scenarios. At present, although many deep learning-based computing resource demand prediction methods can initially realize the prediction of the memory consumption and inference time of the model, they lack cross-task general prediction ability and cannot accurately predict the computing resource demand of deep learning models of different sizes, and the prediction accuracy needs to be improved. SUMMARY

[0009] To solve the above technical problems, the present application provides a deep learning model inference stage computing resource demand prediction method, comprising the following steps:

[0010] Step S1: parse the deep learning model file, extract the node features and node connection information of the computation graph corresponding to the model file;

[0011] Step S2: build and train a computing resource demand prediction model based on convolution attention mechanism and data smoothing processing, including node embedding layer, aggregation layer, data smoothing processing layer and prediction layer, based on the node features and node connection information, build a node embedding vector containing node features and model structure features, then perform dimensionality reduction processing to obtain a one-dimensional graph embedding; then perform smoothing processing through the data smoothing processing layer, and finally output the computing resource demand prediction value;

[0012] Step S3: test the performance of the model using samples in the test set, and save the predicted performance index results to the document.

[0013] Advantages:

[0014] 1. The application provides a deep learning model inference stage computing resource demand prediction method, which parses the deep learning model into a computation graph, and uses an inductive graph neural network to embed the node features of the deep learning model to be predicted.

[0015] 2. In the aspect of deep learning model graph embedding representation, the application uses a convolution attention mechanism to perform two-dimensional learning and aggregation on the extracted computation graph node vector and FLOPs, parameter quantity and other model information, so that the graph embedding representation effectively focuses on the part that has more influence on runtime performance, and improves the feature representation ability of the graph embedding.

[0016] 3. The application adopts data smoothing processing operation to process the aggregated graph embedding data, so as to alleviate the model prediction result deviation problem caused by uneven data distribution. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a deep learning model inference stage computing resource demand prediction method flowchart of the application;

[0018] Figure 2 is a structure diagram of the computing resource demand prediction model based on the convolution attention mechanism and data smoothing processing;

[0019] Figure 3 is a structure block diagram of a deep learning model inference stage computing resource demand prediction system of the application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and not to limit the application. In addition, the technical features involved in each embodiment of the application described below can be combined with each other as long as they do not conflict with each other.

[0021] Example 1

[0022] As shown in Figure 1 , the deep learning model inference stage computing resource demand prediction method provided by the application embodiment comprises the following steps:

[0023] Step S1: Parse the deep learning model file and extract the node features and node connection information of the corresponding computation graph of the model file;

[0024] Step S2: Construct and train a computing resource demand prediction model based on convolutional attention mechanism and data smoothing processing, including: node embedding layer, aggregation layer, data smoothing processing layer and prediction layer. Based on the node features and node connection information, construct a node embedding vector containing node features and model structure features, and then perform dimensionality reduction processing to obtain a one-dimensional graph embedding; then perform smoothing processing through the data smoothing processing layer, and finally output the predicted value of computing resource demand.

[0025] Step S3: Test the model's performance using samples from the test set, and save the predicted performance metrics results to a document.

[0026] In one embodiment, step S1 above: parsing the deep learning model file and extracting the node features and node connection information of the computation graph corresponding to the model file, specifically includes:

[0027] Step S11: Input and parse the initial serialization file of the deep learning model, and extract the corresponding computation graph;

[0028] The initial serialized file of the input deep learning model, such as a file in the form of pth or pb, is parsed to extract the feature information of each layer of the deep learning model and the connection method between layers, thereby obtaining the node features and node connection information of the corresponding computation graph.

[0029] Step S12: Extract the node features and node connection information of the computation graph. The node features include: node type, node hyperparameters (e.g., kernel size, stride, padding, convolution nodes, and dilation coefficient), size of node input tensor, and size of node output tensor.

[0030] Let deep learning models have a total of Each node ,use The vector represents the initial node information, where N represents the feature dimension of the node. It is a one-dimensional vector of length N:

[0031] ;

[0032] in, This indicates vector concatenation; Indicates the node type. Indicates node hyperparameters. and These represent the dimensions of the node's input and output tensors, respectively.

[0033] In one embodiment, step S2 above: constructing and training a computing resource demand prediction model based on convolutional attention mechanism and data smoothing processing, including: a node embedding layer, an aggregation layer, a data smoothing processing layer, and a prediction layer; constructing a node embedding vector containing node features and model structure features based on the node features and node connection information, then performing dimensionality reduction processing to obtain a one-dimensional graph embedding; then performing smoothing processing through the data smoothing processing layer, and finally outputting the predicted computing resource demand value, specifically including:

[0034] Step S21: Construct a computing resource demand prediction model based on convolutional attention mechanism and data smoothing processing, including: node embedding layer, aggregation layer, data smoothing processing layer and prediction layer. The aggregation layer includes: 1 convolutional layer, 1 max pooling layer and 4 Res-CAM modules. The Res-CAM module includes: a residual block and a convolutional attention mechanism module. The convolutional attention mechanism module includes: a channel attention module and a spatial attention module.

[0035] Step S22: The node embedding layer combines the node features and node connection information of the computation graph to generate a node embedding vector that simultaneously contains node features and model structure features. Specifically, it includes:

[0036] Step S221: To minimize computational complexity, this embodiment of the invention selects the classic inductive graph neural network GraphSAGE model as the node embedding layer; firstly, the current node is randomly selected. Sampling is performed on neighboring nodes, with nodes as the basis. Sampling from the center outwards Layer, the number of samples in each layer is equal to the number of nodes in the previous layer. If the number of neighboring nodes is insufficient, the sampling number can be supplemented by repeatedly sampling existing neighboring nodes. ; and nodes Nodes with direct or indirect connections are considered its neighbors; assuming the first... The set of neighbor nodes of the layer is , For the first The first layer There are neighbor nodes; the set of neighbor nodes obtained after sampling is . ;

[0037] Step S222: After sampling is completed, based on the node The characteristics of aggregating neighboring nodes from the center outwards: For the ... Layer neighbor nodes, using aggregation functions to aggregate the first-level neighbor nodes. The features of neighboring nodes sampled from the layer are calculated as follows:

[0038] ;

[0039] in, Indicates aggregation operation; Represents a node The embedded representation of the k-th layer neighbor nodes after the aggregation operation;

[0040] In GraphSAGE, different aggregation methods are used to combine the representations of neighboring nodes. The aggregation method determines how the information of neighboring nodes is integrated into the representation of the current node, thus reflecting the context information of the node in the graph. The aggregation method selected in this embodiment of the invention is GCN aggregation.

[0041] Step S223: Concatenate the aggregated neighbor embeddings of each layer with the embeddings of the previous layer, and update the node embeddings using linear transformations and activation functions. The calculation process is as follows: ;

[0042] in, Indicates the first Layered embedding representation, combining nodes The contextual information of itself and its neighbors contains higher-level graph structure features; This represents the Sigmoid activation function; This indicates a concatenation operation for embedded representations; This represents the weight matrix of the k-th layer;

[0043] Step S224: For Modulo operation and standardization result in a length of The final node embedding vector:

[0044] ;

[0045] in, Represents the second-order norm operation. This represents the updated node embedding vector;

[0046] Step S23: The one-dimensional row vector containing the parameters and FLOPs information of the current network is stacked to form a two-channel tensor, which is then input into the aggregation layer. After convolution, max pooling, and attention operations, a one-dimensional graph embedding is obtained. FLOPs information describes the number of floating-point operations that the deep learning model needs to perform during computation, and is directly related to the features of the input feature map. Specifically, it includes:

[0047] Step S231: ... The input information of the first channel is stacked with the one-dimensional row vector of the current network's parameters and FLOPs information to form a two-channel tensor; where the first channel contains the input information. The first channel contains node embedding vectors that require significant computational resources, where n is the number of node embedding vectors in the channel and d is the length of the node embedding vector; the second channel contains the input information. It consists of node embedding vectors with relatively low computational resource requirements, as well as information on the FLOPs and parameter count of the deep learning model to be predicted.

[0048] Step S232: ... and The input aggregation layer needs to undergo convolution and max pooling operations to obtain a tensor. :

[0049] ;

[0050] in, This represents the convolution operation. This represents the max pooling operation;

[0051] Step S233: ... The input to the Res-CAM module first passes through the residual module, and the output is... ;

[0052] ;

[0053] Step S234: After processing by the channel attention module, the output is: :

[0054] ;

[0055] in, This represents the average pooling operation. This represents the operation of a multilayer perceptron;

[0056] Step S235: ... and Element-wise multiplication is performed to obtain the input for spatial attention. :

[0057] ;

[0058] Step S236: ... Input spatial attention module, output is Similarly, it needs to be with Perform element-wise multiplication:

[0059] ;

[0060] ;

[0061] Step S237: ... Repeat steps S233~S236 three more times, using... This indicates that subsequent operations will ultimately result in a one-dimensional graph embedding of length 256. M represents the length of the graph embedding:

[0062] .

[0063] One-dimensional graph embedding This not only includes node features and node connection features of the computational graph, but also helps the model effectively identify which information in the node embeddings has a greater impact on computational resource requirements, thereby improving overall prediction performance. In step S23, this invention uses a novel aggregation layer based on a convolutional attention mechanism to further enhance the embedding features. After generating node embeddings, it is necessary to aggregate the embedding vectors of all nodes, the parameter count of the deep learning model, and FLOPs values ​​to generate the graph embedding representation of the deep learning model. Furthermore, the generation of graph embeddings not only needs to focus on information that has a greater impact on the computational resource requirements of the model, but also needs to mine the features inside the node embeddings to obtain a more complete and refined computational graph embedding representation. The convolutional attention mechanism includes channel attention and spatial attention. Channel attention enhances the ability to extract complex features by adjusting the weights of the maximum and average values. Spatial attention learns the importance weights of each spatial location, automatically focusing on the features most important to the current task, enhancing the embedding representation's ability to learn the overall understanding of the input, adaptively refining features, and increasing generalization. The channel attention module and the spatial attention module are sequentially connected in series between the convolutional parts of each residual block to further improve the robustness and convergence of the convolutional attention mechanism and avoid problems such as gradient vanishing.

[0064] Since GPU memory consumption prediction and latency prediction are regression prediction problems involving continuous values, traditional upsampling and downsampling operations are only effective for existing discrete samples. Therefore, after the aggregation layer, this invention uses a data smoothing layer to address the problems of imbalanced sample distribution and missing samples within the dataset, thereby mitigating the prediction bias caused by the distribution problems of continuous data.

[0065] Data smoothing includes feature distribution smoothing and label distribution smoothing. The main idea is that if data is evenly distributed in a continuous space, then data in neighboring areas share similar characteristics. Feature distribution smoothing transforms data near missing values, allowing approximate samples to replace the features of the missing regions. After all data has been transformed, the data is redistributed, and the original missing regions are approximated by data from their neighboring regions. Simultaneously, label distribution smoothing updates the probability density distribution of the data, using the updated probability density distribution to weight the loss function, improving the model's perception of data distribution within each interval.

[0066] Step S24: The data smoothing layer processes the imbalanced sample distribution and missing samples in the dataset to obtain the smoothed graph embedding. Specifically, it includes:

[0067] Step S241: Based on the distribution of sample labels within the dataset, divide the labels into several equally spaced intervals and calculate... The mean of the current label interval ,variance :

[0068] ;

[0069]

[0070] in, The first of the intervals containing the current label Bar chart embedding data, This represents the number of samples within the current label's range.

[0071] Step S242: After obtaining the mean and variance of each interval, based on the feature distribution smoothing algorithm, a symmetric kernel function is used. and Perform convolution operations:

[0072] ;

[0073] ;

[0074] in, For symmetric kernel functions, the Gaussian kernel function, which has the best performance, is selected as the symmetric kernel function in this embodiment of the invention; This represents the number of neighboring intervals involved in updating the features; This indicates the interval currently being updated. The label values ​​of the samples within the current interval. The sample label values ​​within the nearest interval;

[0075] Step S243: ... The mean and variance of the updated data within the same interval are standardized to obtain a smoothed graph embedding vector. :

[0076] ;

[0077] Until all After the update is complete, you will get ,Right now, for A set;

[0078] Step S244: Use the label distribution smoothing algorithm to convolve the symmetric kernel function with the empirical label density distribution to obtain the updated label distribution density.

[0079] ;

[0080] in, The original label distribution density of the data in the current interval is the number of deep learning models in this interval divided by the total number of models in the dataset. This represents the updated label distribution density.

[0081] The data smoothing layer is only enabled during the training phase of the prediction model, enabling the prediction model to learn a mapping relationship between the feature distribution of the deep learning model and the computing resource requirements during the inference phase that is more consistent with reality. Therefore, during the testing phase of the computing resource requirement prediction model based on the convolutional attention mechanism and data smoothing, the data does not need to be processed by the data smoothing layer.

[0082] Step S25: In the prediction layer, The final output, mapped to specific computing resource requirements, includes:

[0083] Step S251: Use a multilayer perceptron (MLP) as the prediction layer. The calculation process is as follows:

[0084] ;

[0085] ;

[0086] Among them, As the first hidden layer Input MLP; This is the predicted value for the current input sample; For the first The input of each hidden layer For the first The weight matrix of each hidden layer For the first The bias matrix of each hidden layer; L is the total number of hidden layers;

[0087] Step S252: Calculate the average relative loss function With weighted loss function :

[0088] ;

[0089] ;

[0090] Where Num is the total number of samples. Let be the predicted value of the prediction model for the i-th sample. Let be the label value of the i-th sample. The updated label distribution density for the interval containing the i-th sample;

[0091] Step S253: Perform backpropagation based on the weighted loss function, and update and optimize the parameters using gradient descent until the loss is minimized or a set threshold is reached.

[0092] The node embedding layer constructed by the method of this invention has two purposes: First, it merges the layer information and inter-layer connection information of the model into node embeddings that contain model structural information. Second, it maps low-dimensional graph feature data to a high-dimensional space, mining more complex feature information of the model. Then, the high-dimensional node embedding represents the part following the input: including an aggregation layer, a data smoothing layer, and a prediction layer. The purpose of the aggregation layer is to further extract and reduce the dimensionality of useful high-dimensional information related to computing resource requirements in the computation graph, outputting simple low-dimensional graph embedding information. In other words, it is to further extract important features of computing resource requirements. Then, the graph embedding features are further learned by the data smoothing layer to achieve accurate prediction of neural networks of various sizes. Finally, the prediction layer is used to output the corresponding predicted values ​​of computing resource requirements.

[0093] Figure 2 This is a schematic diagram of the computing resource demand prediction model based on convolutional attention mechanism and data smoothing processing.

[0094] In one embodiment, step S3 above involves testing the model's performance using samples from the test set and saving the predicted performance metrics results to a document (e.g., txt).

[0095] This invention uses the open-source dataset NAS-bench201 for experiments. By comparing it with existing schemes that use deep learning for predicting computing resource requirements, the invention reduces the mean absolute percentage error of the prediction. Tables 1 and 2 show the performance metrics of the proposed method and existing classical methods in latency prediction and memory consumption prediction.

[0096] Table 1. Specific performance metrics of the present invention and classical methods for latency prediction on the NAS-bench201 dataset.

[0097] Table 2. Specific performance metrics of memory consumption prediction for the present invention and classical methods on the NAS-bench201 dataset.

[0098] Example 2

[0099] like Figure 3 As shown in the figure, this embodiment of the invention provides a system for predicting computing resource requirements during the inference stage of a deep learning model, comprising the following modules:

[0100] Preprocessing module 41 is used to parse deep learning model files and extract node features and node connection information of the computation graph corresponding to the model file;

[0101] Model building and training module 42 is used to build and train a computing resource demand prediction model based on convolutional attention mechanism and data smoothing processing;

[0102] The performance testing module 43 is used to test the performance of the model using samples in the test set and save the predicted performance metrics results to a document.

[0103] A device for predicting computing resource requirements during the inference phase of a deep learning model includes one or more electronic devices, wherein the one or more electronic devices are used to implement a method for predicting computing resource requirements during the inference phase of a deep learning model.

[0104] An electronic device includes: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a method for predicting computing resource requirements during the inference phase of a deep learning model.

[0105] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for predicting computing resource requirements during the inference stage of a deep learning model, characterized in that, include: Step S1: Parse the deep learning model file and extract the node features and node connection information of the corresponding computation graph of the model file; Step S2: Construct and train a computing resource demand prediction model based on convolutional attention mechanism and data smoothing processing, including: node embedding layer, aggregation layer, data smoothing processing layer and prediction layer. Based on the node features and node connection information, construct a node embedding vector containing node features and model structure features, and then perform dimensionality reduction processing to obtain a one-dimensional graph embedding; then perform smoothing processing through the data smoothing processing layer, and finally output the predicted value of computing resource demand. Step S3: Test the model's performance using samples from the test set, and save the predicted performance metrics results to a document.

2. The method for predicting computing resource requirements during the inference stage of a deep learning model according to claim 1, characterized in that, Step S1: Parse the deep learning model file and extract the node features and node connection information of the computation graph corresponding to the model file, specifically including: Step S11: Input and parse the initial serialization file of the deep learning model, and extract the corresponding computation graph; Step S12: Extract the node features and node connection information of the computation graph. The node features include: node type, node hyperparameters, size of node input tensor, and size of node output tensor. Let the deep learning model have a total of Each node ,use The vector represents the initial node information, where N represents the feature dimension of the node. It is a one-dimensional vector of length N: ； in, This indicates vector concatenation; Indicates the node type. Indicates node hyperparameters. and These represent the dimensions of the node's input and output tensors, respectively.

3. The method for predicting computing resource requirements during the inference stage of a deep learning model according to claim 2, characterized in that, Step S2: Construct and train a computing resource demand prediction model based on convolutional attention mechanism and data smoothing processing, including: a node embedding layer, an aggregation layer, a data smoothing processing layer, and a prediction layer. Based on the node features and node connection information, construct a node embedding vector containing node features and model structure features, then perform dimensionality reduction processing to obtain a one-dimensional graph embedding; then perform smoothing processing through the data smoothing processing layer, and finally output the predicted computing resource demand value, specifically including: Step S21: Construct a computing resource demand prediction model based on convolutional attention mechanism and data smoothing processing, including: node embedding layer, aggregation layer, data smoothing processing layer and prediction layer, wherein the aggregation layer includes: 1 convolutional layer, 1 max pooling layer and 4 Res-CAM modules, wherein the Res-CAM module includes: a residual block and a convolutional attention mechanism module, and the convolutional attention mechanism module includes: a channel attention module and a spatial attention module; Step S22: The node embedding layer combines the node features of the computation graph with the node connection information to generate a node embedding vector that simultaneously contains node features and model structure features. ; Step S23: The one-dimensional row vector containing the parameters and FLOPs information of the current network is stacked to form a two-channel tensor, which is then input into the aggregation layer. After convolution, max pooling, and attention operations, a one-dimensional graph embedding is obtained. The FLOPs information describes the number of floating-point operations that the deep learning model needs to perform during the computation process, and is directly related to the features of the input feature map. Step S24: The data smoothing layer processes the imbalanced sample distribution and missing samples in the dataset to obtain a smoothed graph embedding. ; Step S25: In the prediction layer, This is mapped to specific computing resource requirements as the final output.

4. The method for predicting computing resource requirements during the inference stage of a deep learning model according to claim 3, characterized in that, Step S22: The node embedding layer combines the node features of the computation graph and the node connection information to generate a node embedding vector that simultaneously contains node features and model structure features. Specifically, it includes: Step S221: First, randomly select the current node. Sampling is performed on neighboring nodes, with nodes as the basis. Sampling from the center outwards Layer, the number of samples in each layer is equal to the number of nodes in the previous layer. If the number of neighboring nodes is insufficient, the sampling number can be supplemented by repeatedly sampling existing neighboring nodes. ; and nodes Nodes with direct or indirect connections are considered its neighbors; assuming the first... The set of neighbor nodes of the layer is , For the first The first layer There are neighbor nodes; the set of neighbor nodes obtained after sampling is . ; Step S222: For the first Layer neighbor nodes, using aggregation functions to aggregate the first-level neighbor nodes. The features of neighboring nodes sampled from the layer are calculated as follows: ； in, Indicates aggregation operation; Represents a node The embedded representation of the k-th layer neighbor nodes after the aggregation operation; Step S223: Concatenate the aggregated neighbor embeddings of each layer with the embeddings of the previous layer, and update the node embeddings using linear transformations and activation functions. The calculation process is as follows: ； in, Indicates the first Layered embedding representation, combining nodes The contextual information of itself and its neighbors contains higher-level graph structure features; This represents the Sigmoid activation function; This indicates a concatenation operation for embedded representations; This represents the weight matrix of the k-th layer; Step S224: For Modulo operation and standardization result in a length of The final node embedding vector: ； in, Represents the second-order norm operation. This represents the updated node embedding vector.

5. The method for predicting computing resource requirements during the inference stage of a deep learning model according to claim 4, characterized in that, Step S23: ... The one-dimensional row vector containing the parameters and FLOPs information of the current network is stacked to form a two-channel tensor, which is then input into the aggregation layer. After convolution, max pooling, and attention operations, a one-dimensional graph embedding is obtained. Specifically, it includes: Step S231: ... The input information of the first channel is stacked with the one-dimensional row vector of the current network's parameters and FLOPs information to form a two-channel tensor; where the first channel contains the input information. The first channel contains node embedding vectors that require significant computational resources, where n is the number of node embedding vectors in the channel and d is the length of the node embedding vector; the second channel contains the input information. It consists of node embedding vectors with relatively low computational resource requirements, as well as information on the FLOPs and parameter count of the deep learning model to be predicted. Step S232: ... and The input to the aggregation layer needs to undergo convolution and max pooling operations to obtain a tensor. : ； in, This represents the convolution operation. This represents the max pooling operation; Step S233: ... The input to the Res-CAM module first passes through the residual module, and the output is... ; ； Step S234: After processing by the channel attention module, the output is: : ； in, This represents the average pooling operation. This represents the operation of a multilayer perceptron; Step S235: ... and Element-wise multiplication is performed to obtain the input for spatial attention. : ； Step S236: ... Input spatial attention module, output is Similarly, it needs to be with Perform element-wise multiplication: ； ； Step S237: ... Repeat steps S233~S236 three more times, using... This indicates that subsequent operations will ultimately result in a one-dimensional graph embedding of length 256. M represents the length of the graph embedding: 。 6. The method for predicting computing resource requirements during the inference stage of a deep learning model according to claim 5, characterized in that, Step S24: The data smoothing layer processes the imbalanced sample distribution and missing sample issues within the dataset to obtain a smoothed graph embedding. Specifically, it includes: Step S241: Based on the distribution of sample labels within the dataset, divide the labels into several equally spaced intervals and calculate... The mean of the current label interval ,variance : ； ； in, The first of the intervals containing the current label Bar chart embedding data, This represents the number of samples within the current label's range. Step S242: Based on the feature distribution smoothing algorithm, use a symmetric kernel function and and Perform convolution operations: ； ； in, It is a symmetric kernel function; This represents the number of neighboring intervals involved in updating the features; This indicates the interval currently being updated. The label values ​​of the samples within the current interval. The sample label values ​​within the nearest interval; Step S243: ... The mean and variance of the updated data within the same interval are standardized to obtain a smoothed graph embedding vector. : ； Until all After the update is complete, you will get ,Right now, for A set; Step S244: Using a label distribution smoothing algorithm, convolve the symmetric kernel function with the empirical label density distribution to obtain the updated label distribution density. ； in, The original label distribution density of the data in the current interval is the number of deep learning models in this interval divided by the total number of models in the dataset. This represents the updated label distribution density.

7. The method for predicting computing resource requirements during the inference stage of a deep learning model according to claim 6, characterized in that, Step S25: In the prediction layer, The final output, mapped to specific computing resource requirements, includes: Step S251: Use a multilayer perceptron (MLP) as the prediction layer. The calculation process is as follows: ； ； Among them, As the first hidden layer Input MLP; This is the predicted value for the current input sample; For the first The input of each hidden layer For the first The weight matrix of each hidden layer For the first The bias matrix of each hidden layer; L is the total number of hidden layers; Step S252: Calculate the average relative loss function With weighted loss function : ； ； Where Num is the total number of samples. Let be the predicted value of the prediction model for the i-th sample. Let be the label value of the i-th sample. The updated label distribution density for the interval containing the i-th sample; Step S253: Perform backpropagation according to the weighted loss function, and update and optimize the parameters using gradient descent until the loss is minimized or a set threshold is reached.

8. A system for predicting computing resource requirements during the inference stage of a deep learning model, characterized in that, Includes the following modules: The preprocessing module is used to parse deep learning model files and extract node features and node connection information of the corresponding computation graph of the model file; The model building and training module is used to build and train a computing resource demand prediction model based on convolutional attention mechanism and data smoothing processing. The performance testing module is used to test the model's performance using samples from the test set and save the predicted performance metrics results to a document.

9. A device for predicting computing resource requirements during the inference stage of a deep learning model, characterized in that, It includes one or more electronic devices, wherein the one or more electronic devices are used to implement the method of any one of claims 1 to 7.

10. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1 to 7.

Images