Model performance prediction method and device, electronic equipment and medium

Abstract

The embodiment of the invention relates to the technical field of artificial intelligence and high-performance computing, and provides a model performance prediction method and device, electronic equipment and a medium, and the method comprises the steps: obtaining a prediction application parameter and a constraint target of a to-be-tested model inputted by a user; determining a target reasoning mode based on the prediction application parameters; and based on the prediction application parameter, the constraint target and the target reasoning mode, performing performance prediction on the to-be-tested model to obtain a performance prediction result, the performance prediction result including a performance index prediction value, a hardware type selection recommendation result or a model structure recommendation result. Therefore, accurate prediction of the performance of the model is realized, and a user is assisted in performing optimal resource scheduling and model selection on a specific task.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and high-performance computing technology, and in particular to a method, apparatus, electronic device and medium for predicting the performance of a model. Background Technology

[0002] With the rapid development of deep learning technology, the complexity of model structures and the size of datasets are growing exponentially, leading to an increasing demand for hardware computing power (such as GPUs and TPUs). In the actual engineering implementation of artificial intelligence, users typically face the following challenges: Time cost assessment is difficult: It is difficult to accurately predict the time, memory, and computing resources required before training or deployment begins. Estimation usually relies on human experience or by running a few trial epochs, which is prone to error and inefficient.

[0003] Hardware selection is challenging: Faced with different models of computing cards (such as NVIDIA V100, A100, T4, etc.), users do not know which hardware offers the best cost performance or can meet specific latency requirements, often leading to wasted resources or substandard performance, and making it difficult to make optimal decisions in hardware procurement.

[0004] Blindly selecting models: In scenarios with hardware or time constraints (such as edge device deployment), it is difficult to quickly determine which model structure can complete the calculation and achieve the target accuracy within the specified time. This often requires significant hardware and human resources investment in preliminary testing.

[0005] Current performance evaluation primarily relies on simple benchmark tests or general theoretical computing power formulas (such as FLOPs divided by peak hardware performance). This approach achieves high accuracy on some mainstream model architectures (such as ResNet and MobileNet) and mainstream datasets (such as ImageNet and COCO). However, this approach fails to distill and extend the understanding of complex factors such as dataset characteristics (e.g., dataset size, image resolution), differences in model operators (different operators have varying utilization rates on different architectures), and bandwidth bottlenecks in actual hardware operation. Existing technologies lack a unified performance awareness and recommendation mechanism that comprehensively considers the coupling relationship between datasets, models, and hardware, and therefore cannot support flexible, multi-dimensional constrained recommendations. Summary of the Invention

[0006] This invention provides a method, apparatus, electronic device, and medium for predicting model performance, which addresses the shortcomings of existing technologies that lack a unified performance perception and recommendation mechanism that comprehensively considers the coupling relationship between datasets, models, and hardware, and cannot support flexible multi-dimensional constraint recommendations. It enables accurate prediction of model performance and assists users in optimal resource scheduling and model selection for specific tasks.

[0007] This invention provides a method for predicting the performance of a model, comprising: Obtain the prediction application parameters and constraint objectives of the model under test, as input by the user; The target inference pattern is determined based on the predicted application parameters; Based on the predicted application parameters, constraint objectives, and objective reasoning patterns, the performance of the model under test is predicted to obtain performance prediction results, wherein the performance prediction results include predicted performance index values, hardware selection recommendation results, or model structure recommendation results.

[0008] In one possible implementation, the method further includes: The target reasoning modes include positive performance awareness mode, hardware selection recommendation mode, and model structure recommendation mode; When the prediction application parameters include dataset features, model structure features, and hardware type features, the target inference mode is determined to be a positive performance-aware mode. When the prediction application parameters include dataset features, model structure features, and expected time, the target inference mode is determined to be a hardware selection recommendation mode. When the prediction application parameters include dataset features, hardware type features, and expected time, the target inference mode is determined to be the model structure recommendation mode.

[0009] In one possible implementation, the method further includes: Obtain the raw performance data of the sample deep learning model; The raw performance data is cleaned, and a performance knowledge base is constructed based on the cleaned performance data. The performance knowledge base includes datasets, model structures, hardware parameters, and performance metrics.

[0010] In one possible implementation, the method further includes: If the target inference mode is determined to be a positive performance-aware mode, then the first triplet information is generated based on the dataset features, model structure features, and hardware type features in the prediction application parameters. Retrieve target performance records in the performance knowledge base that are exactly or isomorphically matched with the information of the first triplet; If the target performance record is found, the corresponding performance metric is returned directly.

[0011] In one possible implementation, the method further includes: If the target reasoning mode is determined to be a hardware selection recommendation mode, then the hardware candidate set is traversed, and a second triplet is generated for each hardware in the hardware candidate set by combining the hardware information of the dataset features and model structure features. Based on the constraint target and the second triplet information corresponding to each hardware, calculate the performance index corresponding to each hardware; Based on the performance indicators corresponding to each hardware component, the hardware configuration that meets the constraint objectives and has the lowest cost or highest utilization rate is selected from the hardware candidate set.

[0012] In one possible implementation, the method further includes: If the target reasoning mode is determined to be a model structure recommendation mode, then the first set of candidate model structures for similar tasks is extracted from the performance knowledge base. Based on the dataset features and hardware type features in the prediction application parameters, calculate the performance index corresponding to each model in the first candidate model structure set. Based on the performance indicators and constraint objectives corresponding to each model, a second set of candidate model structures that meet the constraint objectives is selected. Query the average accuracy index of each target model structure in the second candidate model structure set recorded in the performance knowledge base; Based on the average accuracy index, the target model structure that meets the time requirements and has the best accuracy on the specified hardware is selected.

[0013] In one possible implementation, the method further includes: Based on the predicted application parameters, constraint objectives, and objective reasoning patterns, the performance of the model under test is predicted through a hierarchical hybrid architecture to obtain the performance prediction results. The hierarchical hybrid architecture includes a first layer of table lookup matching, a second layer of linear inference, and a third layer of machine learning fitting.

[0014] The present invention also provides a performance prediction device for a model, comprising the following modules: The acquisition module is used to acquire the prediction application parameters and constraint objectives of the model under test, which are input by the user. The determination module is used to determine the target inference mode based on the prediction application parameters; The prediction module is used to perform performance prediction on the model under test based on the prediction application parameters, constraint objectives, and target inference mode, and obtain performance prediction results, wherein the performance prediction results include predicted values ​​of performance indicators, hardware selection recommendation results, or model structure recommendation results.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a performance prediction method for the model as described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the performance prediction method of the model as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a performance prediction method for the model as described above.

[0018] The present invention provides a model performance prediction method, apparatus, electronic device, and medium that acquires the prediction application parameters and constraint objectives of the model under test input by the user; determines the target inference mode based on the prediction application parameters; and performs performance prediction on the model under test based on the prediction application parameters, constraint objectives, and target inference mode to obtain performance prediction results. The performance prediction results include predicted performance index values, hardware selection recommendation results, or model structure recommendation results. Compared to existing technologies that lack a unified performance perception and recommendation mechanism that comprehensively considers the coupling relationship between datasets, models, and hardware, and cannot support flexible multi-dimensional constraint recommendations, this solution achieves accurate prediction of model performance, assisting users in optimal resource scheduling and model selection for specific tasks. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is an overall architecture diagram of the performance prediction system for the model provided by this invention.

[0021] Figure 2 This is one of the flowcharts illustrating the performance prediction method for the model provided by this invention.

[0022] Figure 3 This is the performance knowledge base architecture diagram provided by the present invention.

[0023] Figure 4 This is a database architecture diagram of the performance knowledge base provided by this invention.

[0024] Figure 5 This is the second flowchart illustrating the performance prediction method for the model provided by this invention.

[0025] Figure 6 This is a schematic diagram of the intelligent recommendation process provided by the present invention.

[0026] Figure 7This is a diagram of the two-model structure provided by the present invention.

[0027] Figure 8 This is a schematic diagram of the performance prediction device for the model provided by the present invention.

[0028] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0030] To facilitate understanding of the embodiments of the present invention, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0031] Figure 1 This is an overall architecture diagram of the performance prediction system for the model provided by this invention, such as... Figure 1 As shown, the overall architecture of this system includes: a data feature extraction module, a performance knowledge base construction module, a performance prediction engine, and a multi-dimensional recommendation decision module.

[0032] Among them, the data feature extraction module: the system is based on triples. Establish mapping relationship ,in For dataset features, For model structural features, Hardware type characteristics, For performance metrics (time, memory usage, power consumption, etc.), the formula is expressed as: Performance knowledge base building module: Uses a relational database (such as PostgreSQL) to store all... Triplet and its corresponding performance indicators .

[0033] Performance prediction engine: Uses table lookup (when hitting the knowledge base), simple linear interpolation (when the variable changes linearly), multinomial fitting, or machine learning methods (such as XGBoost, random forest, multilayer perceptron) to predict the given data. predict .

[0034] Multidimensional Recommendation Decision Module: Based on a performance prediction engine, it provides decision suggestions according to the user's actual needs. In a given task (dataset)... Under these conditions, the required time can be predicted based on a given model and hardware, suitable hardware can be recommended based on a given model and expected time, and the best model can be recommended based on existing hardware and expected time.

[0035] Data feature extraction module: Core function: Responsible for processing triples Perform multidimensional feature quantization.

[0036] Interaction relationships: This is the foundation for other modules, and all other modules need to call this module.

[0037] Performance knowledge base building module: Physical entities: built on a relational database (such as PostgreSQL), storing historically cleaned data. Structured data.

[0038] Interaction relationship: The data feature extraction module is called to obtain features, which are then stored and used by the performance prediction engine.

[0039] Performance prediction engine: Core function: As the computational core of the system, it is responsible for processing the input... Predictive performance metrics .

[0040] Interaction relationship: The data feature extraction module is called to obtain features, and the performance knowledge base is used for prediction.

[0041] Multidimensional recommendation decision module: Core functionality: Based on the output of the prediction engine, provide intelligent decision support according to user constraints.

[0042] Interaction relationship: depends on the results of all the above modules.

[0043] Figure 2 This is one of the flowcharts illustrating the performance prediction method for the model provided by this invention, such as... Figure 2 As shown, the method includes the following: S21. Obtain the prediction application parameters and constraint objectives of the model to be tested, as input by the user.

[0044] The embodiments of the present invention are combined with Figure 5 The second flowchart illustrating the performance prediction method for the model is explained in detail.

[0045] First, obtain the prediction application parameters and constraint objectives of the model under test, as input by the user. The prediction application parameters include: Dataset features The features considered in this method include input shape (input data dimension) and data type. For vision tasks, the input shape is... or For natural language processing tasks, the input shape is... Data types include Float32, Float16, Int8, etc.

[0046] Model structural features The features considered in this method include total number of parameters, theoretical computational cost (FLOPs), and operator type and number (commonly used ones such as Conv2d, Linear, Attention).

[0047] Hardware type characteristics This method only considers NVIDIA graphics cards that support CUDA and TensorRT acceleration technologies. It extracts the hardware's theoretical peak computing power (TFLOPS), memory size and bandwidth, CUDA core count and Tensor unit count, and NVIDIA graphics card computing capability.

[0048] Constraint objectives include: user-expected training or inference time, meeting time requirements on specified hardware, or achieving optimal accuracy.

[0049] S22. Determine the target reasoning mode based on the predicted application parameters.

[0050] The target reasoning modes include positive performance awareness mode, hardware selection recommendation mode, and model structure recommendation mode.

[0051] When the predicted application parameters include dataset features, model structure features, and hardware type features, the target inference mode is determined to be a positive performance-aware mode. When the predicted application parameters include dataset features, model structure features, and expected time, the target inference mode is determined to be the hardware selection recommendation mode. When the predicted application parameters include dataset features, hardware type features, and expected time, the target inference mode is determined to be the model structure recommendation mode.

[0052] Specifically, Mode A (Positive Performance Awareness Mode): User input The system outputs the predicted training / inference time and GPU memory usage.

[0053] Mode B (Hardware Selection Recommendation Mode): User Input ,in This represents the user's desired training or inference time. The system iterates through the available hardware in the hardware library. ,calculate We recommend hardware to users that meets their expected inference time and has the lowest cost or highest utilization rate.

[0054] Pattern C (Model Structure Recommendation Pattern): User Input The system traverses models in the model library. ,calculate The model structure that meets the time requirements and has the best accuracy on the specified hardware is selected.

[0055] S23. Based on the predicted application parameters, constraint objectives, and target inference mode, perform performance prediction on the model under test to obtain performance prediction results. These performance prediction results include predicted performance index values, hardware selection recommendations, or model structure recommendations.

[0056] In this embodiment of the invention, a performance knowledge base must first be constructed to store historical cleaned data. Structured data is stored and then used by the performance prediction engine.

[0057] Specifically, Figure 3 This is the performance knowledge base architecture diagram provided by the present invention, such as... Figure 3 As shown, firstly, multi-source data collection is performed to obtain the raw performance data of the deep learning model. The raw performance data is then cleaned, and a performance knowledge base is constructed based on the cleaned data. This knowledge base includes the dataset, model structure, hardware parameters, and performance metrics.

[0058] Multi-source data includes: Public results can be found on NVIDIA's official website, specifically on the CUDA or TensorRT project pages, where you can find training and inference speed data for many mainstream models on different hardware across mainstream datasets. This data is also publicly available in some open-source projects such as MLPerf.

[0059] User upload: The system built in this invention can support users to upload their own historical real operation logs.

[0060] Batch simulation: Both of the above methods mainly obtain the end-to-end performance of the overall model, but lack the actual running speed of the unit operators. Data can be obtained by batch simulating commonly used operators on different hardware.

[0061] Log cleaning: Clean and organize all the logs obtained above. The data is combined and stored in the database to build a performance knowledge base.

[0062] S1 Multi-source Data Acquisition and Knowledge Base Initialization To elaborate on the specific implementation process of multi-source data acquisition and knowledge base initialization in the embodiments of the present invention, it is broken down into three main stages: data source acquisition, feature parsing and standardization, and performance knowledge base construction. The examples described below are intended to explain the embodiments of the present invention and should not be construed as limiting them. For ease of description, the embodiments of the present invention assume that the training time for deep learning is based on the PyTorch 2.7 framework, and the inference time is based on the PyTorch 2.7 or TensorRT 10 framework.

[0063] S1.1 Constructing a multi-level data acquisition channel To ensure data comprehensiveness, the system obtains raw performance data through the following three complementary approaches: Public benchmark data scraping: Using automated scripts or interfaces, and in compliance with regulations, scrape performance reports of mainstream deep learning models (such as ResNet, MobileNet, etc.) on mainstream datasets (such as ImageNet, COCO, etc.) from NVIDIA's official website (such as CUDA or TensorRT project pages) and performance benchmark websites similar to MLPerf.

[0064] Data collection: The focus is on collecting macro-level metrics such as end-to-end training speed and inference latency under different hardware environments.

[0065] Advantages and disadvantages: The advantage of this data set is the comprehensive range of hardware and the accuracy of the test data. However, the disadvantages are that test data is only available for mainstream deep learning models and datasets, while real-world datasets in production applications often differ in characteristics from mainstream datasets. Furthermore, it only provides overall model performance data and lacks data at the internal operator level.

[0066] User historical log access: The system provides an open log upload interface, supporting users to upload real operation logs generated in actual business operations in the past.

[0067] Data collected includes: analyzing user logs of runtime environment configurations, model parameter snapshots, and actual recorded runtime timestamps and peak memory usage. This data reflects the true performance in a production environment.

[0068] Advantages and disadvantages: This part of the data is more in line with real-world production environment usage scenarios, but it depends on the quality of user-uploaded data, and there is a risk of insufficient data volume. Furthermore, it can only provide information on the overall model performance.

[0069] Operator-level batch simulation: Addressing the limitations of the two methods mentioned above that lack access to detailed internal data, this system implements proactive operator simulation testing. Test scripts are written to independently run batch simulations of commonly used unit operators in deep learning (such as Conv2d, Linear, Attention, etc.) on different hardware devices. It is important to note that the performance differences between different generations of NVIDIA GPUs are difficult to predict, while the performance of different GPU models within the same generation generally exhibits numerical relationships in terms of memory bandwidth, CUDA computing units, etc.

[0070] Data collection: Accurately record the actual running speed and resource consumption of a single operator under different input dimensions and parameter configurations.

[0071] Advantages and disadvantages: This data can accurately predict operator-level computational performance and has strong scalability. The disadvantage is that it requires different models of GPUs for simulation in the early stages, resulting in high computational costs.

[0072] S1.2 Multidimensional Feature Extraction and Analysis The system utilizes a data feature extraction module to uniformly map the collected raw unstructured or semi-structured data into standard triple features. .

[0073] Dataset feature parsing: Extracting geometric and type information from logs or configurations of input data.

[0074] Shape features: For visual tasks, extract (Number of channels, height, width); For natural language processing tasks, extract... (Sequence length, dimension).

[0075] Type characteristics: Data precision type. In this example, FP32 and FP16 are considered. Currently, the mainstream precision types also include TF32, BF16, INT8, INT4, FP4, etc.

[0076] Model structure feature analysis: Static analysis of the model architecture Macro metrics: Calculate the total number of parameters and theoretical computational cost (FLOPs / MACs) of the model using libraries similar to, but not limited to, ultralytics-thop.

[0077] Micro-level metrics: The types and quantities of basic operators included in the statistical model. Specifically, in this example, basic operators refer to all operators included in the torch.nn library. For simplicity, this example only considers the Conv2d, Linear, and BatchNorm operators.

[0078] Hardware Type Characteristics Analysis: Hardware characteristics determine the upper limit of computing power. This example primarily focuses on NVIDIA graphics cards with CUDA and TensorRT.

[0079] Compute Capability: This is a key identifier that distinguishes different generations of hardware architecture. The performance of GPUs from different generations cannot be directly compared numerically; a table lookup is necessary. Table 1 lists some mainstream generational architectures, their compute capability codes, and representative graphics cards.

[0080] Table 1 Architecture Name computing power Represents graphics card Turing sm_75 Tesla T4, RTX 20 series Ampere sm_80 Tesla A100 Ampere sm_86 RTX 30 series Ada Lovelace sm_89 RTX 40 series Hopper sm_90 Tesla H100 Blackwell sm_120 RTX 50 series Video memory specifications: These include video memory size (GB) and video memory bandwidth (GB / s). Table 2 lists the video memory specifications of some mainstream graphics cards.

[0081] Table 2 Graphics card model Video memory size Video memory bandwidth RTX 2080 11GB 616GB / s RTX 3090 24GB 936GB / s RTX 4080 16 GB 717GB / s RTX 4090 24GB 1TB / s Tesla T4 16 GB 320GB / s Tesla A100 (40GB) 40GB 1.6TB / s Tesla H100 (PCIe) 80GB 2TB / s Computational units: These include the number of CUDA stream processors and the number of Tensor processors. The number of Tensor processors has a significant impact on FP16 and INT8 performance and is a key metric in deep learning. It also includes the theoretical peak computational power (TFLOPS). Table 3 lists computational unit data for some mainstream graphics cards. From this data, it's clear that the number of Tensor cores is closely related to FP32 computing power, but there is no theoretical numerical relationship between it and AI computing power.

[0082] Table 3 Graphics card model Tensor core count FP32 computing power AI computing power RTX 2080 544 (2nd generation) 13.4 TFLOPS 27 TFLOPS RTX 3090 328 (3rd generation) 35.6 TFLOPS 71 TFLOPS RTX 4080 304 (4th generation) 48.7 TFLOPS 97 TFLOPS RTX 4090 512 (4th generation) 82.6 TFLOPS 165 TFLOPS Tesla T4 320 (2nd generation) 8.1 TFLOPS 65 TFLOPS Tesla A100 (40GB) 432 (3rd generation) 19.5 TFLOPS 312 TFLOPS Tesla H100 (PCIe) 456 (4th generation) 51.2 TFLOPS 1513 TFLOPS S1.3 Definition of Performance Indicators Performance metrics are the target output of a prediction model, used to quantify the efficiency of training or inference. During training, the primary focus is generally on the throughput (number of samples processed per unit time); higher throughput results in shorter total training time. During inference, depending on the specific needs, both inference latency and throughput may be considered. Generally, as batch size increases, throughput gradually increases, while inference latency gradually increases; a balance needs to be found based on actual requirements. In this embodiment, for simplicity, inference latency is defined as the time required for the deep learning model to infer one sample. Throughput is the theoretical maximum throughput of the model under the given memory constraints.

[0083] Time indicators: Inference latency: the time taken for a single forward propagation (ms); Training throughput: The maximum throughput during training within the limits of available GPU memory; Inference Throughput: The maximum throughput during inference within the limits of available video memory.

[0084] Resource indicators: Training memory usage: Peak training memory usage (MB / GB) when Batchsize is 1; Inference memory usage: Peak inference memory usage (MB / GB) when Batchsize is 1.

[0085] S1.4 Data Cleaning and Knowledge Base Storage This step performs quality control and structured persistent storage on the various source data obtained in step S1.1, completing the initialization of the knowledge base.

[0086] User log data cleaning and association: Clean all user-uploaded log data obtained in step S1.1, remove outliers (such as extreme latency caused by hardware failure), and associate and match the extracted dataset features, model features, hardware features with the actual observed performance indicators.

[0087] Structured storage: Using relational databases such as, but not limited to, PostgreSQL as the underlying storage medium to establish a performance knowledge base.

[0088] Table 4 (Preset Hardware Feature Table), Table 5 (Dataset Feature Table), Table 6 (Model Feature Table), and Table 7 (Performance Fact Table) disclose a table design for a performance knowledge database. The database architecture diagram of the performance knowledge base is shown below. Figure 4 As shown. This design does not limit the form of the performance knowledge database in this invention.

[0089] Table 4 Preset Hardware Features field name type illustrate Example hardware_id INT(PK) Primary key 1 device_name VARCHAR Graphics card name RTX 3090 compute_capability VARCHAR computing power sm_86 memory_sz_gb FLOAT Video memory size (GB) 24 memory_bw_gbps FLOAT Video memory bandwidth (GB / s) 936 tensor_core_count INT Number of Tensors 328 peak_tflops_fp32 FLOAT Single-precision floating-point computing power 35.6 peak_tflops_fp16 FLOAT Half-precision floating-point computing power 71 Table 5 Dataset Features field name type illustrate Example dataset_id INT(PK) Primary key 1 dataset_name VARCHAR Dataset Name ImageNet domain_type VARCHAR Task type CV input_shape JSON Input dimensions (3,224,224) Table 6 Model Feature Table field name type illustrate Example model_id INT(PK) Primary key 1 model_name VARCHAR Model Name ResNet50 total_params_m FLOAT Total number of parameters 25.6 gflops FLOAT Theoretical calculation amount 4.1 operators JSON Include operator {"conv2d_512c_3k":10} Table 7 Performance Fact Sheet field name type illustrate Example result_id BIGINT(PK) primary key 1 model_id INT(FK) Association Model 1 hardward_id INT(FK) Related hardware 1 dataset_id INT(FK) Related datasets 1 precision VARCHAR Operating accuracy FP16 infer_latency_ms FLOAT Inference latency 4 train_troughput FLOAT Training throughput 1200 infer_troughput FLOAT Inference throughput 5000 train_memory_gb FLOAT Training memory 1 infer_memory_gb FLOAT Inference memory 1 Furthermore, train the performance prediction model: The features extracted above can be categorized as follows: Simple linear relationship characteristics: For example, data types, the computation speed and memory usage of the Float16 type can be considered to be a constant multiple (slightly more than half) of that of the Float32 type. The computation speed and memory usage of cascading operators are approximately equal to... Operators of the same type times.

[0090] Non-simple linear relationships: For example, the number of channels in a convolution, assuming the input and output shapes of convolution A are... The input and output shapes of convolution B are Theoretically, B has four times the number of parameters and four times the computational cost of A, so A's computation speed should be four times that of B. However, if... When the size is small, due to poor parallelism, the computation speed of A will be less than 4 times that of B; while if When the size is large, convolution B may be limited by the size and bandwidth of the video memory, and the calculation speed of A will be more than 4 times that of B.

[0091] Unable to find numerical relationships: For example, the computing power of NVIDIA graphics cards is represented by version numbers, such as sm_86 for the RTX 30 series and sm_89 for the RTX 40 series. It can only be determined that the computing power of sm_89 is greater than that of sm_86, but there is no definite numerical relationship.

[0092] Therefore, this embodiment employs a step-by-step prediction model. First, for features where no numerical relationship can be found, a table lookup is used; if no match is found, the prediction fails. For example, to predict the computing power of a K80 graphics card (sm_35), if the performance knowledge base lacks data for that computing power, prediction is impossible. Next, linear interpolation prediction is performed on simple linear relationship features. Finally, machine learning methods (such as XGBoost, Random Forest, and Multilayer Perceptron) are used to predict features with non-simple linear relationships. The final prediction model input is then obtained. Corresponding performance indicators .

[0093] Specifically, S2. Hierarchical performance prediction model construction Based on the characteristic mentioned above that there is no numerical relationship between certain dimensions of features and the final performance, this embodiment of the invention does not use an end-to-end prediction model. Instead, it constructs a hierarchical prediction architecture based on "table lookup matching - linear interpolation deduction - machine learning fitting". This architecture aims to make full use of existing knowledge base data, prioritize the processing of features with high determinism, and then use statistical models to process complex nonlinear features.

[0094] The specific construction steps are as follows: S2.1 First Layer: Exact Matching and Approximate Retrieval Based on Knowledge Base This layer primarily handles discrete features without numerical relationships (such as GPU computing power version numbers sm_86 and sm_89) and known historical data. It is divided into three cases: Exact Hit: Query Triples If the performance knowledge base built in S1 contains completely consistent records, then the corresponding performance metrics from the storage will be returned directly. This is the most accurate way to predict.

[0095] Homogeneous Hardware Mapping: If a specific hardware model (such as RTX 3080) is not in the database, the system will search for hardware records with the same computing power. For example, if RTX 3080 data is missing from the database, but RTX 3070 and RTX 3090 data with the same architecture exist, they will be used as the base anchor point for linear scaling at the next layer. Similarly, if a specified model is not in the database, but data records exist for all the unit operators used by that model, it can also be used as the anchor point for linear scaling at the next layer.

[0096] Homogeneous hardware not found: If the specified hardware model is not in the library and there is no hardware record with the same computing power; or if the specified model is not in the database and not all of the unit operators used have data records, then the feature set cannot be effectively predicted, and in this case, it should be returned that it cannot be predicted.

[0097] S2.2 Second Layer: Linear Deduction Based on Physical Rules For some features with simple linear relationships, this layer establishes a linear regression model based on physical rules for rapid deduction. Among the feature dimensions described in S1, the following features can be linearly deduced: Total video memory Total number of parameters: The number of model parameters doubles, and the required video memory also doubles; Total video memory Operating precision: FP32 uses twice the video memory of FP16; Total video memory Number of operators: N operators occupy N times the video memory of a single operator; Overall latency Number of cascaded operators: The overall time delay of N cascaded operators is N times that of a single operator.

[0098] The remaining characteristics are all nonlinear or piecewise linear.

[0099] S2.3 Third Layer: Nonlinear Fitting Based on Machine Learning For complex scenarios that cannot be solved by table lookup or simple linear regression, this system constructs a regression prediction model based on supervised learning. This layer focuses on addressing the nonlinear impact of the number of convolution channels on parallelism and the limitations of video memory bandwidth.

[0100] S2.3.1 Multidimensional Feature Cross-Fusion In order for machine learning models to understand The coupling relationship between the three cannot be simply addressed by directly inputting the original parameters; feature engineering must be performed to extract derived features with clear physical meaning.

[0101] Input feature vector Defined as a cascade of the following three parts: Original feature encoding: Hardware Vector ( ): Normalized [FP32 TFLOPS, memory bandwidth, CUDA core count, Tensor Core count].

[0102] Model-Data Vector ( ): [Total number of parameters, theoretical FLOPs, input tensor volume ( ), BatchSize] Physical constraint-derived features: Arithmetic strength ( ): Used to determine whether a task is "computation-intensive" or "memory-intensive" in the model.

[0103] Theoretical parallelism ratio ( ): This feature is used to indicate whether the GPU is in a state of "not being able to run at full capacity due to insufficient workload".

[0104] Operator histogram features ( ): The frequency and proportion of different types of operators in the statistical model. Here, the operators must be pre-built operators in the performance knowledge base.

[0105] Finally, the input vector is obtained. .

[0106] S2.3.2 Selection of Predictive Model Architecture Based on the characteristics of the data, a fusion strategy of XGBoost (Extreme Gradient Boosting Tree) and MLP (Multilayer Perceptron) is adopted.

[0107] Main model (XGBoost Regressor): Reason for selection: Tree models excel at handling heterogeneous data (such as continuous FLOPs and discrete GPU generations) and non-linear interactions between features. They learn well when... The rule breakpoint is when the computation speed no longer increases linearly with FLOPs.

[0108] Input: Complete feature vector .

[0109] Output: Basic prediction performance Residual Correction Model (MLP): Reason for selection: Neural networks have a better fitting ability for continuous variables (such as smooth curves caused by changes in BatchSize).

[0110] Purpose: As a lightweight network, it is used to correct the shortcomings of tree models in numerical extrapolation.

[0111] S2.3.3 Training and Optimization Strategies Loss function design To simultaneously assess prediction accuracy and relative error, MAPE (Mean Absolute Percentage Error) is used as the primary loss function, supplemented by MSE (Mean Squared Error). in For actual measured performance metrics (such as time). These are predicted values.

[0112] Divide and conquer training strategy Scenario A (Bandwidth-Constrained Area): Filter out A separate sub-model is trained for samples with lower (memory-intensive) memory access, focusing on learning the clamping effect of memory bandwidth on speed.

[0113] Scenario B (Computationally Constrained Area): Filter out The sub-model is trained with a high (computationally intensive) sample size, focusing on learning the effects of TFLOPS and Tensor Cores.

[0114] During prediction: based on the input data The outputs of the two sub-models are dynamically weighted and fused.

[0115] Cold start and incremental learning In the initial stage, pre-training is performed using public datasets (such as MLPerf) and batch simulated data (obtained in S1).

[0116] Once users upload real logs (obtained in S1), the model is fine-tuned using this high-value "real domain" data to adapt to specific production environment deviations.

[0117] S2.3.4 Example: Prediction logic for the "convolution channel doubling" scenario Assuming convolution A input Convolution B input .

[0118] Feature extraction: The model identified that B has 4 times more FLOPs and 4 times more parameters than A.

[0119] Parallelism determination: like Smaller: Calculated features Extremely low. The XGBoost model judges that the GPU utilization is low at this time based on historical patterns, and predicts that the time taken for B may only be 1.5 times that of A (instead of 4 times), because a large number of CUDA Cores are idle when computing A, but are activated when computing B.

[0120] like Larger: Saturated, but A surge in bandwidth saturation occurs. The model predicts that B may take more than four times longer than A, due to memory swapping or bandwidth bottlenecks.

[0121] Through the detailed steps of S2.3 described above, this embodiment of the invention achieves "perception" of the performance of complex deep learning, rather than simple "computation".

[0122] Furthermore, S3. Multi-scenario Intelligent Decision-Making and Recommendation like Figure 6 As shown in the diagram of the intelligent recommendation process, the multi-dimensional recommendation decision module, based on the output of the performance prediction engine and combined with the constraints set by the user, solves three core problems in the engineering implementation of deep learning through search and optimization algorithms: prediction, selection, and scheduling.

[0123] Based on the type of parameters entered by the user, automatically switch to one of the following three working modes: S3.1 Mode A: Positive Performance Perception and Early Warning This mode primarily addresses the problem of "difficulty in assessing time costs" and is suitable for scenarios where users have already determined the model and hardware and only need to estimate the operational status.

[0124] Input definition: User input triples .

[0125] Dataset features (e.g., ImageNet) Model structural features (such as ResNet50) : Specify the hardware environment (e.g., RTX 3090, precision FP16) Other hyperparameters include precision and BatchSize. BatchSize can be specified by the user or adaptively found to find the maximum BatchSize that meets the video memory requirements. When the user does not specify a batch size, BatchSize is initially set to 1.

[0126] Execution process: Step 1: Call the feature extraction module in S1 to convert the input unstructured information into feature vectors.

[0127] Step 2: Call the hierarchical performance prediction model established by S2 to calculate the prediction results. .

[0128] Step 3: If the user does not specify a batch size, the maximum batch size is calculated based on the predicted video memory usage and the hardware's physical video memory limit. Otherwise, it checks whether the predicted video memory usage exceeds the hardware's physical video memory limit. If it exceeds the limit, an "OOM (Out Of Memory)" warning is issued, and a maximum allowed batch size is recommended based on the ratio.

[0129] Step 4: Output the training throughput (samples / second), inference latency (ms), and GPU memory usage for prediction.

[0130] S3.2 Mode B: Hardware Selection and Optimal Cost Recommendation This model primarily addresses the challenge of hardware selection, making it suitable for scenarios where users need to purchase hardware or apply for cloud instances and aim for the lowest cost while meeting time requirements. In this scenario, user constraints generally fall into three categories. The first is completing the training task within the expected timeframe, where the problem transforms into finding the maximum training throughput; the second is finding the maximum inference throughput; and the third is finding the minimum single-sample inference latency. These objectives can all be obtained from the performance prediction model in step S2.

[0131] Input definition: User input And the desired goals.

[0132] Execution process: Step 1 (Establishing the Candidate Set): The system loads all available candidate hardware sets from the hardware knowledge base. .

[0133] Step 2 (Traversal Prediction): For each hardware in the set... , combined Generate triples and call the prediction engine to get performance results. .

[0134] Step 3 (Constraint Filtering): Filter out the valid hardware subset that meets the time constraints. .

[0135] Step 4 (Multidimensional Ranking Decision): Based on user preferences... Sort them.

[0136] Strategy 1 (Cost First): Recommend the cheapest graphics card that meets the performance requirements (e.g., recommend the RTX 3080 instead of the A100).

[0137] Strategy 2 (Utilization Priority): Recommend hardware that can maximize computing power to avoid resource waste.

[0138] S3.3 Mode C: Adaptive Recommendation Based on Model Structure This mode primarily addresses the problem of "blindly selecting models" and is suitable for edge computing or real-time application scenarios (where hardware and latency are fixed, and the model with the highest accuracy needs to be found).

[0139] Input definition: User input And the desired goals.

[0140] Execution process: Step 1 (Model Library Retrieval): The system extracts all candidate model structures for similar tasks from the model library. (For example, a series of models from MobileNetV2 to ResNet152).

[0141] Step 2 (Reverse Traversal): For fixed hardware and dataset traverse and calculate each model Performance.

[0142] Step 3 (Boundary Filtering): Filter out all models that satisfy the constraints. .

[0143] Step 4 (Precision Optimization): In Query the average accuracy of models recorded in the knowledge base (such as Top-1 Accuracy) and recommend the model structure with the highest accuracy.

[0144] Example 1: An example of batch simulation of operators This embodiment aims to explain in detail how the system constructs a high-precision performance knowledge base through "operator-level batch simulation" (corresponding to the description in S1.1), especially the data acquisition process for nonlinear features (such as the difference in parallelism under different input sizes).

[0145] Simulation Environment Setup Hardware components: Two representative graphics cards were selected for comparative testing. Hardware A: NVIDIA Tesla T4 (16GB VRAM, 8.1 TFLOPS, computing power sm_75) Hardware B: NVIDIA RTX 3090 (24GB VRAM, 35.6 TFLOPS, sm_86 compute capability) Test operator: Select the most commonly used Conv2d (two-dimensional convolution) operator in deep learning.

[0146] Variable settings Fixed parameter: kernel size Step length ,filling FP32 precision Variation parameters: Input and output feature map sizes from by The step size is gradually increased to BatchSize from Increase to Simulated execution process The system automatically executes the script, performing the following steps for each of the above configuration groups: Warm-up: Run 10 idle runs to activate the GPU frequency to Boost state and avoid cold start errors; Timing: Perform 100 consecutive forward propagations, record the total time, and take the average value to obtain the inference latency; Video memory record: Use PyTorch's cuda.max_memory_allocated() to record peak video memory usage.

[0147] Data Collection Results and Analysis The data collected by the system is shown in Table 8 below: Table 8 ID hardware Input dimensions FLOPs Theoretical time Actual time consumption 1 Tesla T4 (1,256,32,32) 1.2G 0.149ms 0.318ms 2 Tesla T4 (32,256,32,32) 38.7G 4.77ms 4.91ms 3 RTX 3090 (1,256,32,32) 1.2G 0.034ms 0.229ms 4 RTX 3090 (128,512,32,32) 618G 17.37ms 24.18ms In Experiments 1 and 3, due to the small input, most of the CUDA cores were idle, and the startup overhead accounted for a large proportion. Therefore, although the theoretical peak computing power of the RTX 3090 graphics card is more than 4 times that of the Tesla T4, the actual time taken is less than 2 times.

[0148] In Experiment 2, increasing the BatchSize improved the parallelism, and the measured performance was closer to the theoretical value.

[0149] In Experiment 4, the excessive number of channels triggered the memory bandwidth bottleneck, resulting in a longer processing time than theoretically expected.

[0150] Through the aforementioned batch simulations, the system no longer relies solely on FLOPs for linear estimation. Instead, it has learned that when the input size is small, the prediction model should increase the "startup penalty coefficient," and when the input size is extremely large, it should increase the "bandwidth penalty coefficient." This data, after being cleaned, is stored in a performance knowledge base to support subsequent accurate predictions.

[0151] Example 2: Application of intelligent decision-making and recommendation in Go AI This embodiment demonstrates how the system of the present invention solves practical problems in specific AI engineering application scenarios through Mode B (hardware recommendation) and Mode C (model recommendation).

[0152] Scene background A Go AI team needs to develop a Go AI program with professional-level performance.

[0153] Dataset: Historical game data containing 50 million game records Model type: The constant-width, constant-depth residual network proposed in the AlphaGo paper (a ResNet without downsampling, such as...) Figure 7 (As shown) Constraints: The first round of iterations must be completed within 7 days to allow for subsequent testing of the search engine. The competition rules also strictly limit the thought time for each move to no more than 10 seconds, the provided hardware is a single RTX 4090, and MCTS needs to simulate 10,000 inferences, meaning each inference must take less than 1ms.

[0154] Application process Because the competition imposes strict constraints on inference time, the model structure was first selected based on the competition's hardware and constraints, and then the training hardware was chosen accordingly.

[0155] Competition Phase: Model Structure Adaptive Recommendation (Pattern C) User input: Hardware is fixed at RTX 4090, expected inference latency <1ms, candidate models are ResNets of different depths (each convolution input and output size is...). ), kernel size is ) System processing: Based on the inference performance of the pre-processed and simulated Conv2d operator on RTX 4090, the system found that the inference time of a single ResNet Block at FP16 accuracy is about 0.05ms.

[0156] Decision output: The system recommends using a 60-layer ResNet (inference time is about 0.9ms). This size can achieve high accuracy while allowing sufficient room for fluctuation.

[0157] Training Phase: Hardware Selection and Cost Optimization (Mode B) User input: The dataset contains 50 million samples, and the model is a 60-layer ResNet (approximately 60M parameters). The expected time to traverse the entire dataset is less than or equal to 7 days (i.e., training throughput is approximately >82 samples per second). System processing: The system traverses the hardware library and simulates and predicts the training throughput of the model on RTX 3090 and RTX 4090. The prediction results are that the training throughput on RTX 3090 is approximately 70 samples per second, and the training throughput on RTX 4090 is approximately 130 samples per second.

[0158] Decision output: The system recommends using an RTX 4090 for training. Although two RTX 3090s can also be used, considering that the cost of two RTX 3090s is close to that of one RTX 4090, but multi-GPU training incurs overhead and is more complex to implement, a single RTX 4090 is still recommended.

[0159] As can be seen from this embodiment, the system of the present invention can not only perform simple performance calculations, but also combine specific business constraints (such as deadlines and competition rules) to provide optimal hardware procurement suggestions and model architecture suggestions in two completely different scenarios of "training" and "inference", which greatly improves the decision-making efficiency and success rate of AI engineering.

[0160] The beneficial effects of this method include: High-precision prediction: Compared with theoretical estimation that relies solely on FLOPs, this invention comprehensively considers data characteristics, hardware bandwidth bottlenecks, and operator characteristics, resulting in a significant reduction in prediction error rate.

[0161] Multi-dimensional optimization support: For a given task, it can not only predict the time based on the given model and hardware, but also recommend models or hardware when time is limited, which greatly improves the efficiency of AI technology transformation in real-world problems.

[0162] Maximizing resource utilization: This avoids the situation of over-configuring expensive hardware to leave room for uncertainty, thus saving enterprises computing costs.

[0163] Automated processes: It can be integrated into cloud computing platforms to automatically allocate the most suitable computing nodes to new tasks without human intervention.

[0164] The performance prediction method for models provided by this invention obtains the prediction application parameters and constraint objectives of the model under test input by the user; determines the target inference mode based on the prediction application parameters; and performs performance prediction on the model under test based on the prediction application parameters, constraint objectives, and target inference mode to obtain performance prediction results. The performance prediction results include predicted performance index values, hardware selection recommendation results, or model structure recommendation results. Compared to existing technologies that lack a unified performance awareness and recommendation mechanism that comprehensively considers the coupling relationship between datasets, models, and hardware, and cannot support flexible multi-dimensional constraint recommendations, this method establishes a performance prediction model based on a (dataset, model structure, hardware parameters) triple, achieving accurate prediction of training or inference time and resource consumption. Based on this data, it provides forward prediction and reverse recommendation functions to assist users in optimal resource scheduling and model selection for specific tasks.

[0165] The performance prediction device for the model provided by the present invention is described below. The performance prediction device for the model described below can be referred to in correspondence with the performance prediction method for the model described above.

[0166] Figure 8 This is a schematic diagram of the performance prediction device for the model provided by the present invention, specifically including: The acquisition module 801 is used to acquire the prediction application parameters and constraint objectives of the model under test input by the user. For detailed explanations, please refer to the relevant descriptions in the above method embodiments; they will not be repeated here.

[0167] The determination module 802 is used to determine the target inference mode based on the predicted application parameters. For detailed explanations, please refer to the relevant descriptions in the above method embodiments; they will not be repeated here.

[0168] The prediction module 803 is used to perform performance prediction on the model under test based on the prediction application parameters, constraint objectives, and target inference mode, and obtain performance prediction results. The performance prediction results include predicted performance index values, hardware selection recommendation results, or model structure recommendation results. For detailed explanations, please refer to the relevant descriptions in the above method embodiments; they will not be repeated here.

[0169] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9As shown, the electronic device may include a processor 910, a communications interface 920, a memory 930, and a communication bus 940, wherein the processor 910, the communications interface 920, and the memory 930 communicate with each other via the communication bus 940. The processor 910 can call logical instructions in the memory 930 to execute a model performance prediction method. This method includes: acquiring prediction application parameters and constraint objectives of the model under test input by the user; determining a target inference mode based on the prediction application parameters; and performing performance prediction on the model under test based on the prediction application parameters, constraint objectives, and target inference mode to obtain a performance prediction result, wherein the performance prediction result includes predicted performance index values, hardware selection recommendation results, or model structure recommendation results.

[0170] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0171] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the performance prediction method for the model provided by the above methods. The method includes: obtaining prediction application parameters and constraint objectives of the model under test input by a user; determining a target inference mode based on the prediction application parameters; and performing performance prediction on the model under test based on the prediction application parameters, constraint objectives, and target inference mode to obtain a performance prediction result. The performance prediction result includes predicted values ​​of performance indicators, hardware selection recommendation results, or model structure recommendation results.

[0172] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a performance prediction method for the model provided by the methods described above. The method includes: acquiring predictive application parameters and constraint targets of a model under test input by a user; determining a target inference mode based on the predictive application parameters; and performing performance prediction on the model under test based on the predictive application parameters, constraint targets, and target inference mode to obtain a performance prediction result, wherein the performance prediction result includes predicted performance index values, hardware selection recommendation results, or model structure recommendation results.

[0173] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0174] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0175] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting the performance of a model, characterized in that, include: Obtain the prediction application parameters and constraint objectives of the model under test, as input by the user; The target inference pattern is determined based on the predicted application parameters; Based on the predicted application parameters, constraint objectives, and objective reasoning patterns, the performance of the model under test is predicted to obtain performance prediction results, wherein the performance prediction results include predicted performance index values, hardware selection recommendation results, or model structure recommendation results.

2. The method according to claim 1, characterized in that, The target reasoning modes include a positive performance perception mode, a hardware selection recommendation mode, and a model structure recommendation mode; The process of determining the target inference pattern based on the predicted application parameters includes: When the prediction application parameters include dataset features, model structure features, and hardware type features, the target inference mode is determined to be a positive performance-aware mode. When the prediction application parameters include dataset features, model structure features, and expected time, the target inference mode is determined to be a hardware selection recommendation mode. When the prediction application parameters include dataset features, hardware type features, and expected time, the target inference mode is determined to be the model structure recommendation mode.

3. The method according to claim 1, characterized in that, Before obtaining the prediction application parameters and constraint objectives of the model under test input by the user, the following steps are included: Obtain the raw performance data of the sample deep learning model; The raw performance data is cleaned, and a performance knowledge base is constructed based on the cleaned performance data. The performance knowledge base includes datasets, model structures, hardware parameters, and performance metrics.

4. The method according to any one of claims 1-3, characterized in that, The process of predicting the performance of the model under test based on the predicted application parameters, constraint objectives, and objective reasoning patterns to obtain performance prediction results includes: If the target inference mode is determined to be a positive performance-aware mode, then the first triplet information is generated based on the dataset features, model structure features, and hardware type features in the prediction application parameters. Retrieve target performance records in the performance knowledge base that are exactly or isomorphically matched with the information of the first triplet; If the target performance record is found, the corresponding performance metric is returned directly.

5. The method according to any one of claims 1-3, characterized in that, The process of predicting the performance of the model under test based on the predicted application parameters, constraint objectives, and objective reasoning patterns to obtain performance prediction results includes: If the target reasoning mode is determined to be a hardware selection recommendation mode, then the hardware candidate set is traversed, and a second triplet is generated for each hardware in the hardware candidate set by combining the hardware information of the dataset features and model structure features. Based on the constraint target and the second triplet information corresponding to each hardware, calculate the performance index corresponding to each hardware; Based on the performance indicators corresponding to each hardware component, the hardware configuration that meets the constraint objectives and has the lowest cost or highest utilization rate is selected from the hardware candidate set.

6. The method according to any one of claims 1-3, characterized in that, The process of predicting the performance of the model under test based on the predicted application parameters, constraint objectives, and objective reasoning patterns to obtain performance prediction results includes: If the target reasoning mode is determined to be a model structure recommendation mode, then the first set of candidate model structures for similar tasks is extracted from the performance knowledge base. Based on the dataset features and hardware type features in the prediction application parameters, calculate the performance index corresponding to each model in the first candidate model structure set. Based on the performance indicators and constraint objectives corresponding to each model, a second set of candidate model structures that meet the constraint objectives is selected. Query the average accuracy index of each target model structure in the second candidate model structure set recorded in the performance knowledge base; Based on the average accuracy index, the target model structure that meets the time requirements and has the best accuracy on the specified hardware is selected.

7. The method according to claim 1, characterized in that, The method further includes: Based on the predicted application parameters, constraint objectives, and objective reasoning patterns, the performance of the model under test is predicted through a hierarchical hybrid architecture to obtain the performance prediction results. The hierarchical hybrid architecture includes a first layer of table lookup matching, a second layer of linear inference, and a third layer of machine learning fitting.

8. A performance prediction device for a model, characterized in that, include: The acquisition module is used to acquire the prediction application parameters and constraint objectives of the model under test, which are input by the user. The determination module is used to determine the target inference mode based on the prediction application parameters; The prediction module is used to perform performance prediction on the model under test based on the prediction application parameters, constraint objectives, and target inference mode, and obtain performance prediction results, wherein the performance prediction results include predicted values ​​of performance indicators, hardware selection recommendation results, or model structure recommendation results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the performance prediction method for the model as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the performance prediction method for the model as described in any one of claims 1 to 7.

Images