Performance testing system and method for deep learning model, and prediction model generation method

By disassembling the deep learning model into a separate operator and using its performance index data to train the prediction model, the problem that deep learning model cannot be detected in time after compilation in the existing technology is solved, and fast and accurate performance detection is achieved. Data center scenarios of diversity and complexity.

WO2024230745A1PCT designated stage expired Publication Date: 2024-11-14ALIBABA CLOUD COMPUTING CO LTD

Patent Information

Application Number
PCT/CN2024/091764
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-09
Filing Date
2024-05-08
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

The existing technology cannot detect its performance in time after the deep learning model is compiled, resulting in the inability to quickly and accurately evaluate the model performance in practical applications, especially when the model compilation time is long.

Method used

By disassembling the deep learning model to be measured into a separate operator, the performance prediction model is trained using the performance index data of each operator, the performance indicators of a single operator are predicted, and the performance indicators of a single operator are combined into the comprehensive performance indicators of the model, so as to achieve rapid and Accurate performance detection.

Benefits of technology

Performance detection is performed without waiting for the deep learning model to be compiled. It fully considers the software/hardware diversity and complexity of scenarios such as data centers, improves the universality and accuracy of detection, and realizes a fast and accurate deep learning model. Performance detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024091764_14112024_PF_FP_ABST
    Figure CN2024091764_14112024_PF_FP_ABST
Patent Text Reader

Abstract

Provided in the embodiments of the present description are a performance testing system and method for a deep learning model, and a prediction model generation method. The performance testing method for a deep learning model comprises: determining a deep learning model to be subjected to testing; obtaining one or more operators of said deep learning model; for each of the one or more operators, performing performance index prediction by using a performance prediction model corresponding to the operator, so as to obtain a predicted performance index value corresponding to the operator, wherein the performance prediction model is obtained by means of performing training by using performance index data corresponding to the operator; and obtaining a predicted comprehensive performance index value of said deep learning model on the basis of the predicted performance index value corresponding to each of the one or more operators.
Need to check novelty before this filing date? Find Prior Art

Description

Deep learning model performance detection system, method and prediction model generation method

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 9, 2023, with application number 202310528889.X and application name “Deep Learning Model Performance Detection System, Method and Prediction Model Generation Method”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of this specification relate to the field of computer technology, and in particular to a method for detecting the performance of a deep learning model and a method for generating a prediction model. Background Art

[0003] With the development of artificial intelligence (AI) technology, deep learning models have been widely used in fields such as natural language processing, speech recognition, and autonomous driving. The actual deployment of deep learning models requires consideration of model performance. For example, performance can be measured through metrics such as latency and memory usage. Evaluating the performance of deep learning models not only helps select appropriate resource configurations for running them, but also aids in the design and optimization of the deep learning models themselves. Currently, performance metrics are typically collected during the execution of compiled deep learning models to monitor performance. However, this approach results in long compilation times, making it impossible to monitor the performance of deep learning models in a timely manner.

[0004] In view of this, there is an urgent need for a fast and accurate deep learning model performance testing solution.

[0005] Summary of the Invention

[0006] In view of this, embodiments of this specification provide a deep learning model performance testing system, a deep learning model performance testing method, and a prediction model generation method. One or more embodiments of this specification also relate to a deep learning model performance testing device, a prediction model generation device, a computing device, a computer-readable storage medium, and a computer program to address technical deficiencies in the prior art.

[0007] According to a first aspect of an embodiment of this specification, a deep learning model performance detection system is provided, comprising: a client and a server; the client is configured to send a performance prediction request to the server, the performance prediction request being used to instruct a prediction of a comprehensive performance indicator of the deep learning model to be tested; the server is configured to receive the performance prediction request, obtain the corresponding deep learning model to be tested according to the performance prediction request, obtain one or more operators of the deep learning model to be tested, and perform performance indicator prediction for the one or more operators using the performance prediction model corresponding to the operator to obtain a performance indicator prediction value corresponding to the operator, wherein the performance prediction model is trained by using performance indicator data of the corresponding operator, and based on the performance indicator prediction values ​​corresponding to the one or more operators, a comprehensive performance indicator prediction value of the deep learning model to be tested is obtained.

[0008] According to a second aspect of the embodiments of this specification, a deep learning model performance detection method is provided, including: determining a deep learning model to be tested; obtaining one or more operators of the deep learning model to be tested; for the one or more operators, using a performance prediction model corresponding to the operator to predict performance indicators, and obtaining a performance indicator prediction value corresponding to the operator, wherein the performance prediction model is trained by using performance indicator data of the corresponding operator; based on the performance indicator prediction values ​​corresponding to the one or more operators, obtaining a comprehensive performance indicator prediction value of the deep learning model to be tested.

[0009] According to a third aspect of the embodiments of this specification, a deep learning model performance detection device is provided, comprising: a model determination module, configured to determine a deep learning model to be tested; an operator acquisition module, configured to obtain one or more operators of the deep learning model to be tested; an operator performance prediction module, configured to perform performance indicator prediction on the one or more operators using a performance prediction model corresponding to the operator to obtain a performance indicator prediction value corresponding to the operator, wherein the performance prediction model is trained by using performance indicator data of the corresponding operator; and a model performance calculation module, configured to obtain a comprehensive performance indicator prediction value of the deep learning model to be tested based on the performance indicator prediction values ​​corresponding to the one or more operators.

[0010] According to the fourth aspect of the embodiments of this specification, a prediction model generation method is provided, including: collecting parameter samples and corresponding performance indicator data of an operator; using the parameter samples and corresponding performance indicator data of the operator to train a performance prediction model corresponding to the operator, and when the training stop condition is met, obtaining a performance prediction model for predicting the performance indicator corresponding to the operator.

[0011] According to the fifth aspect of the embodiments of this specification, a prediction model generation device is provided, including: a sampling module, configured to collect parameter samples and corresponding performance indicator data of an operator; a training module, configured to use the parameter samples and corresponding performance indicator data of the operator to train a performance prediction model corresponding to the operator, and when the training stop condition is met, obtain a performance prediction model for predicting the performance indicator corresponding to the operator.

[0012] According to the sixth aspect of the embodiments of this specification, a computing device is provided, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the deep learning model performance detection method described in any embodiment of this specification, or implement the steps of the prediction model generation method described in any embodiment of this specification.

[0013] According to the seventh aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the steps of the deep learning model performance detection method described in any embodiment of this specification are implemented, or the steps of the prediction model generation method described in any embodiment of this specification are implemented.

[0014] According to an eighth aspect of the embodiments of this specification, a computer program is provided, wherein, when the computer program is executed in a computer, the computer is caused to execute the steps of the deep learning model performance detection method described in any embodiment of this specification, or the steps of the prediction model generation method described in any embodiment of this specification.

[0015] One embodiment of the present specification implements a method for detecting the performance of a deep learning model. After determining the deep learning model to be tested, the method obtains one or more operators of the deep learning model to be tested, and uses the performance prediction model corresponding to the operator to predict the performance index for the one or more operators to obtain the performance index prediction value corresponding to the operator. The performance prediction model is trained using the performance index data of the corresponding operator. Therefore, based on the performance index prediction value corresponding to each operator in the one or more operators, the comprehensive performance index prediction value of the deep learning model to be tested can be obtained. It can be seen that the method obtains individual operators by disassembling the deep learning model to be tested, and uses the performance prediction model trained with the performance index data of the individual operators to predict the performance index of the individual operators, and then integrates the performance index prediction values ​​of the individual operators to obtain the comprehensive performance index prediction value of the deep learning model to be tested, thereby realizing a prediction scheme for the running performance of the deep learning model based on each operator. Due to the operator-by-operator prediction, there is no need to wait for the compilation of the deep learning model to be completed. In addition, a performance prediction model is trained based on the performance indicator data obtained by testing the operator in the running environment. It fully considers the software / hardware diversity of scenarios such as data centers and the complexity of software / hardware interactions, improves the versatility and accuracy of detection, and realizes a fast and accurate deep learning model performance detection solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] FIG1 is an architecture diagram of a deep learning model performance detection system provided by one embodiment of this specification;

[0017] FIG2 is an architecture diagram of another deep learning model performance detection system provided by one embodiment of this specification;

[0018] FIG3 is a flow chart of a method for detecting performance of a deep learning model provided by one embodiment of this specification;

[0019] FIG4 is a schematic diagram of the structure of a deep learning model performance detection device provided by one embodiment of this specification;

[0020] FIG5 is a flowchart of a prediction model generation method provided by one embodiment of this specification;

[0021] FIG6 is a flowchart of a process for detecting performance of a deep learning model and generating a prediction model according to an embodiment of the present disclosure;

[0022] FIG7 is a schematic diagram of the structure of a prediction model generating device provided by one embodiment of this specification;

[0023] FIG8 is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0024] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0025] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0026] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, without departing from the scope of one or more embodiments of this specification, first may also be referred to as second, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when...", "when...", or "in response to determining."

[0027] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0028] First, the terms involved in one or more embodiments of this specification are explained.

[0029] A deep learning model is an algorithmic expression that learns patterns and makes predictions by combing through massive amounts of data. In a deep learning model, data is represented as tensors, and a series of operators are applied to these tensors to achieve pattern learning and prediction. Operators are the fundamental building blocks of deep learning models. By using different operators and setting up different computational processes, deep learning models with different purposes and performance can be built.

[0030] An operator is a computational unit in a deep learning model, and can also be understood as a function, which is used to calculate the input tensor according to the calculation method set in the function.

[0031] Deep learning models are now widely used in real-time applications such as natural language processing, speech recognition, and autonomous driving. When deploying deep learning models, it's important to consider their performance. For example, performance can be measured through metrics such as latency and memory usage. Evaluating the performance of deep learning models not only helps select appropriate resource configurations for running them, but also aids in the design and optimization of the deep learning models themselves. Currently, performance metrics are typically collected after the deep learning model is compiled and run to assess performance. However, this approach results in long compilation times, making it impossible to measure the performance of the deep learning model in a timely manner. Predicting the performance metrics of deep learning models in real-world applications is a challenge. This is because the performance metrics of deep learning models are influenced by multiple factors, such as hardware, data load, and software environment, and the interactions between these factors are complex. Therefore, predicting the performance metrics of deep learning models requires comprehensive analysis that considers all of these factors.

[0032] In order to be able to quickly and accurately perform performance testing on deep learning models, the embodiments of this specification provide a deep learning model performance testing solution, which obtains individual operators by disassembling the deep learning model to be tested, and uses the performance indicator data of a single operator to train a performance prediction model to predict the performance indicator of a single operator, and then synthesizes the performance indicator prediction values ​​of the individual operators to obtain the comprehensive performance indicator prediction value of the deep learning model to be tested, thereby realizing a prediction solution for the running performance of the deep learning model based on each operator. Due to the operator-by-operator prediction, there is no need to wait for the compilation of the deep learning model to be completed. In addition, the performance prediction model is obtained by training the performance indicator data obtained by testing the operator in the running environment, which fully considers the software / hardware diversity of scenarios such as data centers and the complexity of software / hardware interactions, improves the versatility and accuracy of the detection, and realizes a fast and accurate deep learning model performance testing solution.

[0033] Specifically, this specification provides a deep learning model performance detection system, a deep learning model performance detection method, and a prediction model generation method. This specification also involves a deep learning model performance detection device, a prediction model generation device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0034] Refer to Figure 1, which shows an architecture diagram of a deep learning model performance detection system provided by an embodiment of this specification. The deep learning model performance detection system may include a client 100 and a server 200.

[0035] The client 100 is configured to send a performance prediction request to the server 200, where the performance prediction request is used to instruct a prediction of a comprehensive performance indicator of the deep learning model to be tested.

[0036] The server 200 is configured to receive the performance prediction request, obtain the corresponding deep learning model to be tested according to the performance prediction request, obtain one or more operators of the deep learning model to be tested, and use the performance prediction model corresponding to the operator to perform performance indicator prediction for the one or more operators to obtain the performance indicator prediction value corresponding to the operator, wherein the performance prediction model is trained by using the performance indicator data of the corresponding operator, and based on the performance indicator prediction value corresponding to the one or more operators, the comprehensive performance indicator prediction value of the deep learning model to be tested is obtained.

[0037] According to the system provided in the embodiments of this specification, the performance prediction request issued by the client 100 causes the server 200 to disassemble the deep learning model to be tested to obtain individual operators, and use the performance prediction model trained with the performance indicator data of the single operator tested in the operating environment to predict the performance indicator of the single operator, and then comprehensively evaluate the performance indicator prediction values ​​of the individual operators, thereby obtaining the comprehensive performance indicator prediction value of the deep learning model to be tested, realizing a prediction scheme for the operating performance of the deep learning model based on each operator, without waiting for the deep learning model to be compiled, and the operating environment can be set according to actual needs, fully considering the software / hardware diversity of scenarios such as data centers and the complexity of software / hardware interactions, thereby improving the speed, versatility and accuracy of detection.

[0038] Referring to Figure 2, Figure 2 shows an architecture diagram of another deep learning model performance detection system provided by an embodiment of this specification. The model processing system may include multiple clients 100 and a server 200. Multiple clients 100 can establish a communication connection through the server 200. In the deep learning model performance detection scenario, the server 200 is used to provide deep learning model performance detection services between multiple clients 100. Multiple clients 100 can act as senders or receivers respectively and communicate through the server 200.

[0039] Users can interact with the server 200 through the client 100 to receive data sent by other clients 100, or send data to other clients 100, etc. In the deep learning model performance testing scenario, the user can send a performance prediction request to the server 200 through the client 100. The server 200 performs deep learning model performance testing based on the performance prediction request and pushes the detected comprehensive performance indicator prediction value to other clients that have established communication.

[0040] The client 100 and the server 200 are connected via a network. The network provides a medium for the communication link between the client 100 and the server 200. The network can include various connection types, such as wired or wireless communication links or fiber optic cables. The data transmitted by the client 100 may need to be encoded, transcoded, compressed, or other processing before being released to the server 200.

[0041] The client 100 can be a browser, an APP (Application), or a web application such as an H5 (HyperText Markup Language 5, Hypertext Markup Language 5) application, a light application (also known as a mini-program, a lightweight application), or a cloud application. The client 100 can be based on the software development kit (SDK) of the corresponding service provided by the server 200, such as developed based on the real-time communication (RTC) SDK. The client 100 can be deployed in an electronic device and needs to rely on the device to run or certain APPs in the device to run. For example, the electronic device can have a display screen and support information browsing, such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. Various other types of applications can also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0042] The server 200 may include servers that provide various services, such as servers that provide communication services to multiple clients, servers that provide background training to support models used on clients, and servers that process data sent by clients. It should be noted that the server 200 can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The server can also be a server in a distributed system, or a server that is integrated with a blockchain. The server can also be a cloud server for basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0043] It is worth noting that the deep learning model performance detection method provided in the embodiments of this specification is generally executed by the server. However, in other embodiments of this specification, the client can also have similar functions to the server to execute the model processing method provided in the embodiments of this specification. In other embodiments, the model processing method provided in the embodiments of this specification can also be executed jointly by the client and the server.

[0044] Referring to FIG3 , FIG3 shows a flowchart of a deep learning model performance detection method provided by an embodiment of this specification, which specifically includes the following steps:

[0045] Step 302: Determine the deep learning model to be tested.

[0046] There are many ways to implement the determination of the deep learning model to be tested.

[0047] For example, in one or more embodiments of the present specification, the server applying the deep learning model performance detection method can receive a performance prediction request sent by the client. Since the performance prediction request is used to indicate the prediction of the comprehensive performance indicators of the deep learning model to be tested, the performance prediction request will carry relevant information of the deep learning model to be tested, such as the model name, model and other information, so that the server can determine the deep learning model to be tested based on this information.

[0048] For another example, in one or more embodiments of this specification, a server implementing the deep learning model performance testing method may be configured to periodically test deep learning models in a data center. In this case, the server may periodically scan some or all of the deep learning models in the data center, thereby identifying the deep learning models to be tested as the deep learning models to be tested. A data center can be understood as a center used to transmit, display, calculate, and store data over the Internet, specifically consisting of one or more servers.

[0049] The deep learning model to be tested may be any one or more deep learning models. For example, the deep learning model to be tested may be a deep learning model based on a computing card.

[0050] Step 304: Obtain one or more operators of the deep learning model to be tested.

[0051] The specific implementation method for obtaining the operators of a deep learning model is not limited. For example, a general model building tool can parse a deep learning model and display the operator stream of the deep learning model. Based on the operator stream displayed by the model building tool, one or more operators of the deep learning model can be obtained. For another example, some deep learning models have already published their own operators. In this case, one or more operators of the deep learning model can be obtained based on this published information.

[0052] Step 306: For the one or more operators, use the performance prediction model corresponding to the operator to perform performance indicator prediction to obtain the performance indicator prediction value corresponding to the operator, wherein the performance prediction model is obtained by training using the performance indicator data of the corresponding operator.

[0053] Wherein, the performance prediction model is obtained by training using the performance indicator data of the operator. According to implementation needs, the performance prediction model can be implemented by any machine learning model. For example, in one or more embodiments of this specification, the performance prediction model is a tree-based gradient boosting deep learning model. Specifically, the tree-based gradient boosting deep learning model adopts a histogram-based algorithm, which stores continuous feature (attribute) values ​​in discrete containers, thereby accelerating the running speed. The performance indicator data of the operator can be obtained through the historical running data of the prior deep learning model, or can be obtained through testing in one or more operating environments according to implementation needs, and this specification does not limit this. By training the performance prediction model using the performance indicator data corresponding to the operator, a performance prediction model for predicting the performance indicators of the operator can be obtained.

[0054] For example, the deep learning model to be tested includes three operators: operator 1, operator 2, and operator 3. The performance indicator data of operator 1 can be used in advance to run the model in one or more operating environments according to test requirements to obtain performance indicator data for operator 1. The performance indicator data of operator 1 can then be used to train the performance prediction model of operator 1, obtaining performance prediction model 1 that can predict the performance indicator of operator 1. Similarly, for operators 2 and 3, performance prediction model 2 that can predict the performance indicator of operator 2 and performance prediction model 3 that can predict the performance indicator of operator 3 are obtained. Furthermore, in the prediction stage, performance prediction model 1 of operator 1 is used to predict the performance indicator of operator 1, obtaining a predicted performance indicator value for operator 1; performance prediction model 2 of operator 2 is used to predict the performance indicator of operator 2, obtaining a predicted performance indicator value for operator 2; and performance prediction model 3 of operator 3 is used to predict the performance indicator of operator 3, obtaining a predicted performance indicator value for operator 3.

[0055] Step 308: Based on the performance indicator prediction values ​​corresponding to the one or more operators, obtain a comprehensive performance indicator prediction value of the deep learning model to be tested.

[0056] It can be understood that operators are the basic components for constructing deep learning models. Based on different operators, deep learning models with different usage purposes and performance can be built, and the performance exhibited by the deep learning model is determined by the performance of each operator it contains. Therefore, the comprehensive performance index prediction value of the deep learning model to be tested is a value calculated by a preset performance evaluation algorithm using the performance index prediction value of one or more operators of the deep learning model to be tested. Among them, the preset performance evaluation algorithm is determined based on the way the performance of the operators influence each other. For example, for certain performance indicators (such as running delay), there is no complex mutual influence between the performance of different operators, and the preset performance evaluation algorithm can be a simple accumulation. For another example, for certain performance indicators, there is a complex mutual influence between the performance of different operators, and the preset performance evaluation algorithm can be a weighted sum, and the specific value of the weight can be determined based on the degree of influence of the performance between different operators.

[0057] Since this method obtains one or more operators of the deep learning model to be tested after determining the deep learning model to be tested, and uses the performance prediction model corresponding to the operator to predict the performance index of each operator in the one or more operators to obtain the performance index prediction value corresponding to the operator, and the performance prediction model is obtained by training using the performance index data of the corresponding operator, therefore, based on the performance index prediction value corresponding to each operator in the one or more operators, the comprehensive performance index prediction value of the deep learning model to be tested can be obtained. It can be seen that this method obtains individual operators by disassembling the deep learning model to be tested, and uses the performance prediction model obtained by training the performance index data of a single operator to predict the performance index of a single operator, and then integrates the performance index prediction values ​​of the individual operators to obtain the comprehensive performance index prediction value of the deep learning model to be tested, thereby realizing a prediction scheme based on the operating performance of the deep learning model on an operator-by-operator basis. Due to the operator-by-operator prediction, there is no need to wait for the compilation of the deep learning model to be completed. Furthermore, a performance prediction model trained based on performance metrics obtained from operator testing in the operational environment fully accounts for the software / hardware diversity and complex interactions found in scenarios like data centers, improving the versatility and accuracy of detection. Consequently, this method provides a fast and accurate solution for deep learning model performance testing.

[0058] The embodiments of this specification perform a comprehensive calculation on the performance indicator prediction values ​​corresponding to each operator. The calculation method for obtaining the comprehensive performance indicator prediction value is not limited. Specifically, the calculation method can be set according to the type of performance indicator and the impact of operators on the comprehensive performance.

[0059] For example, in one or more embodiments of this specification, the performance indicator prediction value corresponding to the operator includes the operation delay of the operator. Accordingly, obtaining the comprehensive performance indicator prediction value of the deep learning model to be tested based on the performance indicator prediction values ​​corresponding to the one or more operators includes:

[0060] Accumulate the performance indicator prediction values ​​corresponding to the one or more operators to obtain the comprehensive performance indicator prediction value of the deep learning model to be tested.

[0061] It's understandable that a deep learning model is composed of a series of operators. These operators operate one by one, ultimately completing the entire model's execution. Therefore, summing the individual operator latencies yields the overall model latency. For example, a predictive deep learning model contains three convolution layers: conv1, conv2, and conv3. These three operators represent three operators. Conv2 requires the results of conv1, and conv3 requires the results of conv2. Therefore, summing the latencies of conv1, conv2, and conv3 yields the overall latency of the deep learning model under test.

[0062] The above embodiment predicts the runtime delay of a single operator and then accumulates the runtime delays of each individual operator to quickly obtain a comprehensive runtime delay prediction value for the deep learning model under test, thus implementing an operator-by-operator prediction scheme for deep learning model runtime delay. This operator-by-operator prediction eliminates the need to wait for the deep learning model to be compiled, thus achieving a fast and accurate deep learning model runtime delay prediction scheme.

[0063] The embodiments of this specification do not restrict the input parameters of the performance prediction model corresponding to the operator. It is understood that since the operating performance of an operator is at least determined by the operator's parameters, the input parameters of the prediction model corresponding to the operator in this specification may include at least the operator's parameters.

[0064] For example, in order to make the performance prediction model used by the method provided in the embodiment of this specification widely applicable to predictions under different operating environments. In one or more embodiments of this specification, for one operator, multiple performance prediction models corresponding to different operating environments can be trained. For example, for operator 1, a performance prediction model 1a corresponding to the GPU (Graphics Processing Unit, graphics processor) operating environment, a performance prediction model 1b corresponding to the CPU (Central Processing Unit, processor) operating environment, and a performance prediction model 1c corresponding to the NPU (Neural-network Process Units, embedded neural network processor) operating environment can be pre-trained. In this embodiment, in the prediction stage, the parameters of the operator can be used as input data, and the corresponding performance prediction model can be selected for prediction based on the operating environment of the deep learning model to be tested.

[0065] For another example, in order to make the performance prediction model used in the method provided in the embodiments of this specification widely applicable to predictions in different operating environments. In one or more embodiments of this specification, for an operator, a performance prediction model can be trained to obtain the performance prediction model, and during training, the operating environment information of the operator is used as one of the input data to participate in the training, so that the trained performance prediction model can recognize the operating environment information and predict the prediction result corresponding to the input operating environment information. Specifically, in this embodiment, the input data of the performance prediction model corresponding to the operator includes the parameters of the operator and the operating environment information of the deep learning model to be tested. Among them, the operating environment information, in the training stage, refers to the information used to describe the operating environment in which the operator is located, and in the prediction stage, refers to the information used to describe the operating environment in which the deep learning model to be tested is located. The operating environment information may include any one or more of the following information: device name, device type, device model, software name, software type and software model. In this embodiment, in the prediction stage, the parameters of the operator and the operating environment information of the deep learning model to be tested can be used as input data, and the corresponding performance prediction model can be used to perform predictions. According to this embodiment, the performance indicators of the deep learning model on any specified hardware can be predicted.

[0066] It should be noted that the deep learning model to be tested may include fusion operators. A fusion operator refers to an operator obtained by reprogramming an operator, or by fusing multiple operators using an operator fusion program. One or more operators of the deep learning model to be tested obtained in the embodiments of this specification may include a fusion operator. Since the performance of the fusion operator may be different in different operating environments, in one or more embodiments of this specification, the fusion operator in the operator of the deep learning model to be tested is further detected, and a more accurate performance prediction is made for the fusion operator in combination with the operating environment information. Specifically, in one or more embodiments of this specification, the one or more operators of the deep learning model to be tested are obtained, including:

[0067] Analyze the deep learning model to be tested using a model building tool to obtain a set of operators of the deep learning model to be tested;

[0068] Detecting a fused operator and / or a non-fused operator from the set of operators, wherein the fused operator is an operator obtained by fusing multiple operators in the deep learning model to be tested, and the non-fused operator is an operator not involved in the fusion in the deep learning model to be tested;

[0069] The input data of the performance prediction model corresponding to the non-fusion operator includes the parameters of the operator and the operating environment information of the deep learning model to be tested;

[0070] The input data of the performance prediction model corresponding to the fusion operator includes the parameters of the operator, the operating environment information of the deep learning model to be tested, and the fusion rules between the multiple operators involved in the fusion.

[0071] It can be understood that the operators in the set of operators obtained by using the model building tool to parse the deep learning model to be tested are operators of the minimum execution unit. In the deep learning model to be tested, multiple operators of the minimum execution unit actually perform fusion calculations, that is, multiple operators of the minimum execution unit are fused into fusion operators in the deep learning model to be tested. Therefore, for the set of operators parsed by the model building tool, fusion operators can be further detected to detect fusion operators and / or non-fusion operators from the set of operators. In this way, for non-fusion operators, the parameters of the operators and the operating environment information of the deep learning model to be tested can be used as input data of the performance prediction model to predict the performance index prediction value of the non-fusion operator; for fusion operators, the parameters of the operators, the operating environment information of the deep learning model to be tested and the fusion rules between the multiple operators involved in the fusion can be used as input data of the performance prediction model to predict the performance index prediction value of the fusion operator.

[0072] The fusion rule is the description information of the fusion relationship between multiple operators.

[0073] Among them, the detection means of the fusion operator is not limited. The fusion rule of the operator can be obtained based on the detection of the fusion operator. For example, in one or more embodiments of this specification, both black box and white box detection methods can be used. Among them, white box detection refers to the detection of a deep learning model that has clearly announced the operator fusion rule. Specifically, the operator fusion rule can be found from its published information to complete the detection. Black box detection refers to the detection of a deep learning model that has not announced the operator fusion rule. Specifically, the operator fusion rule can be detected by actual testing. For example, multiple separate operators can be used to test the performance indicator data of multiple separate operators, and the performance indicator data of multiple separate operators can be comprehensively calculated. For example, the running delay can be accumulated to obtain the comprehensive performance indicators of multiple separate operators without fusion. Then, the performance indicators of the model composed of the multiple operators in the operating environment where the deep learning model to be tested are located can be tested. If the difference between the two is greater than the preset threshold, it can be determined that the multiple operators are suitable for fusion in the operating environment, and the deep learning model to be tested has fused the multiple operators.

[0074] It is understandable that whether multiple operators are integrated is closely related to the operating environment. Multiple operators of the deep learning model to be tested may not be integrated when running in one operating environment, but may be integrated when running in another operating environment. In addition, the performance exhibited by fusion in different operating environments may also be different. Therefore, in one or more embodiments of this specification, for multiple operators, such as operator 1, operator 2, and operator 3, during the training phase, performance prediction models corresponding to each of the multiple operators will be trained, and performance prediction models corresponding to the fusion operators will be trained when multiple operators are integrated in one or more operating environments.

[0075] For example, assume that a model building tool is used to parse the deep learning model A to be tested, obtaining a set of operators including operator 1, operator 2, and operator 3. Assume that, based on testing, it is determined that, within the set of operators, operator 1 and operator 2 are operators that will be fused in the operating environment A where the deep learning model to be tested resides. Therefore, operator 1 and operator 2 are considered fused operator 12, and operator 3 is considered a non-fused operator 3. Furthermore, the operator parameters of operator 1 and operator 2, information about the operating environment A, and the fusion rules of fusion operator 12 are used as input data for fusion operator 12 to predict the performance indicator of fusion operator 12. For another example, suppose that according to the detection, it is determined that in the set of operators, operator 1, operator 2 and operator 3 will not be fused in the operating environment B where the deep learning model to be tested is located. Therefore, operator 1, operator 2 and operator 3 are all regarded as non-fused operators 12. For each operator, the parameters of the operator and the information of the operating environment B are used as the input data of the corresponding performance prediction model to predict the performance indicator prediction value of each operator.

[0076] In the above embodiment, since the parameters of the operator, the operating environment information of the deep learning model to be tested and the fusion rules between the multiple operators involved in the fusion are used as the input data of the performance prediction model corresponding to the fusion operator, the model can learn the pattern of the impact of the operating environment and the operator fusion rules on the operator performance during the training process, so that in the prediction stage, it can be widely applicable to predictions under different operating environments, fully considering the software / hardware diversity of scenarios such as data centers and the complexity of software / hardware interactions, improving the versatility and accuracy of detection, and realizing a fast and accurate deep learning model performance detection solution.

[0077] It should be noted that according to the deep learning model performance detection method provided in the embodiments of this specification, after the comprehensive performance index prediction value of the deep learning model to be tested is predicted, the use of the comprehensive performance index prediction value is not limited. Below, two possible implementation methods are used as examples for schematic illustration.

[0078] In one possible implementation, the predicted value of the comprehensive performance index predicted by the deep learning model can be used to select a resource configuration suitable for running the deep learning model. It is understandable that the deep learning model needs to consider the resources required for operation when deploying the application. For example, in a resource pooling scenario, there may be a large number of different hardware in the resource pool, and different hardware has different execution efficiency for different models. For different service scenarios, if the same resource standard is used for allocation, it will greatly lead to a waste of resource pool utilization. Therefore, the deep learning model performance detection method provided in the embodiment of this specification can evaluate the performance indicators under different software / hardware configurations, so as to select the configuration and improve the overall performance and efficiency of the resource pool.

[0079] In another possible implementation, the predicted value of the comprehensive performance index predicted by the deep learning model can be used for the design and optimization of the deep learning model itself. It is understandable that with the continuous advancement of hardware technology, more and more accelerators and dedicated hardware are being used to accelerate deep learning models. The performance and characteristics of these hardware will have an impact on the performance of the model and affect the actual effect of the optimization model. At the same time, the structure, parameters, input data, etc. of the deep learning model will affect the performance of the model. Therefore, when designing and optimizing the model, it is necessary to comprehensively consider the performance of the model, and make compromises and optimizations. For example, optimization can be achieved by searching for general technical implementations of optimized neural network architectures, and this specification does not limit this. Therefore, after obtaining the predicted value of the comprehensive performance index of the optimized model according to the deep learning model performance detection method provided in the embodiment of this specification, the model design and optimization can be guided based on the prediction results to improve the performance of the model.

[0080] Corresponding to the above-mentioned deep learning model performance detection method embodiment, this specification also provides a deep learning model performance detection device embodiment. Figure 4 shows a schematic diagram of the structure of a deep learning model performance detection device provided by one embodiment of this specification. As shown in Figure 4, the device includes:

[0081] A model determination module 402 is configured to determine a deep learning model to be tested;

[0082] An operator acquisition module 404 is configured to obtain one or more operators of the deep learning model to be tested;

[0083] The operator performance prediction module 406 is configured to perform performance indicator prediction for the one or more operators using a performance prediction model corresponding to the operator to obtain a performance indicator prediction value corresponding to the operator, wherein the performance prediction model is trained using performance indicator data of the corresponding operator;

[0084] The model performance calculation module 408 is configured to obtain a comprehensive performance indicator prediction value of the deep learning model to be tested based on the performance indicator prediction values ​​corresponding to the one or more operators.

[0085] In one or more embodiments of this specification, the performance indicator prediction value corresponding to the operator includes the operation delay of the operator. The model performance calculation module is configured to accumulate the performance indicator prediction values ​​corresponding to the one or more operators to obtain a comprehensive performance indicator prediction value of the deep learning model to be tested.

[0086] In one or more embodiments of this specification, the input data of the performance prediction model corresponding to the operator includes the parameters of the operator and the operating environment information of the deep learning model to be tested.

[0087] In one or more embodiments of this specification, the operator acquisition module includes:

[0088] A model parsing submodule is configured to parse the deep learning model to be tested using a model building tool to obtain a set of operators of the deep learning model to be tested;

[0089] The fusion determination submodule is configured to detect fused operators and / or non-fused operators from the set of operators, wherein the fused operator is an operator obtained by fusing multiple operators in the deep learning model to be tested, and the non-fused operator is an operator that does not participate in the fusion in the deep learning model to be tested.

[0090] The input data of the performance prediction model corresponding to the non-fusion operator includes the parameters of the operator and the operating environment information of the deep learning model to be tested; the input data of the performance prediction model corresponding to the fusion operator includes the parameters of the operator, the operating environment information of the deep learning model to be tested and the fusion rules between the multiple operators involved in the fusion.

[0091] The above is a schematic scheme of a deep learning model performance detection device of this embodiment. It should be noted that the technical scheme of the deep learning model performance detection device and the technical scheme of the deep learning model performance detection method described above are based on the same concept. For details not described in detail in the technical scheme of the deep learning model performance detection device, please refer to the description of the technical scheme of the deep learning model performance detection method described above.

[0092] Corresponding to the above-mentioned deep learning model performance detection method, this specification also provides a prediction model generation method. Referring to Figure 5, Figure 5 shows a flowchart of a prediction model generation method provided by one embodiment of this specification, which specifically includes the following steps:

[0093] Step 502: Collect operator parameter samples and corresponding performance indicator data.

[0094] The specific implementation method of the acquisition operator parameter samples and corresponding performance indicator data is not limited. They can be obtained through historical operation data of the prior deep learning model, or through testing in one or more operating environments according to implementation needs. This specification does not limit this.

[0095] Prior deep learning models refer to common deep learning models, such as some deep learning models running in scenarios such as data centers.

[0096] Operator parameters can be understood as input and / or output parameters, parameters used to define the operator's input, output, and / or computational process. Taking the convolution operator as an example, its parameters include the number of input and output channels, the size of the convolution kernel, the size of the input features, and other parameters. Based on a priori deep learning models, the operator parameters can be used as prior knowledge, and sampling can be performed near these parameters to obtain operator parameter samples. For example, some deep learning models have a convolution operator with a 64 / 64 input and output channel ratio. When sampling near this ratio, points such as 48 / 64 and 72 / 48 can be selected as parameter samples for the convolution operator.

[0097] Operator performance metrics refer to metrics that describe an operator's performance, such as latency and memory usage. For example, the convolution operator described above can be run in one or more runtime environments with 48 / 64, 72 / 48, and other proximity points as input and output channels to obtain the corresponding performance metrics.

[0098] Step 504: Using the parameter samples and corresponding performance indicator data of the operator, a performance prediction model corresponding to the operator is trained, and when a training stop condition is met, a performance prediction model for predicting the performance indicator corresponding to the operator is obtained.

[0099] The training stopping condition is not limited, for example, it can be that the number of iterations of model training reaches a preset threshold, or the loss value of the predicted result relative to the actual result converges to within a preset error threshold.

[0100] The method collects parameter samples and corresponding performance indicator data of operators, uses the parameter samples and corresponding performance indicator data of the operators to train the performance prediction model corresponding to the operators, and obtains a performance prediction model for predicting the performance indicator corresponding to the operators when the training stop condition is met. Therefore, in the prediction stage, the performance indicator prediction values ​​of each individual operator of the deep learning model to be tested can be predicted, and the performance indicator prediction values ​​of the individual operators are integrated to obtain the comprehensive performance indicator prediction value of the deep learning model to be tested. Therefore, based on the prediction model generation method, the prediction of the operating performance of the deep learning model based on each operator can be realized, thereby improving the speed and accuracy of the deep learning model performance detection.

[0101] The specific implementation method of the acquisition operator parameter samples and corresponding performance indicator data is not limited. In one or more embodiments of this specification, the acquisition operator parameter samples and corresponding performance indicator data include:

[0102] Sampling operator parameter samples from one or more prior deep learning models;

[0103] Based on the parameter samples of the operator, a performance test is performed on the operator to obtain corresponding performance indicator data.

[0104] The performance testing of an operator can also be understood as running the operator in a selected runtime environment. For example, using the parameters of a common deep learning model as prior knowledge, a certain number of points are sampled near the prior knowledge. These selected points are used as parameter samples for the operator. The operator is then run in the selected runtime environment, and performance metrics such as latency, memory usage, and any other performance metrics are monitored during runtime.

[0105] Through the above embodiment, parameter samples are sampled based on prior knowledge, and performance testing of operators is performed based on the sampled parameter samples, so that training data for actual testing can be obtained, thereby ensuring the accuracy and reliability of prediction.

[0106] In addition, in order to be able to correct erroneous prediction results more quickly and accurately, in one or more embodiments of this specification, the method also includes: for a sampling point where the prediction deviation of the performance prediction model reaches a re-sampling condition, re-entering the one or more prior deep learning models according to the sampling point, and sampling to obtain parameter samples of the operator.

[0107] For example, taking the above-mentioned convolution operator with an input / output channel of 64 / 64 as an example, sampling near it, neighboring points such as 48 / 64 and 72 / 48 have been selected as parameter samples for the convolution operator. Since the prediction deviation is large and the training stop condition is not met, resampling can be performed at the neighboring points of 48 / 64 where the prediction deviation is large. For example, neighboring points such as 54 / 62 can be reselected as parameter samples, and the performance of the convolution operator can be tested using the newly selected parameter samples to obtain corresponding performance indicator data. Therefore, the performance prediction model of the convolution operator can be continued to be trained using the reselected neighboring points such as 54 / 62 as input and the corresponding performance indicator data as output until the training stop condition is met.

[0108] To enable the methods provided in the embodiments of this specification to be widely applicable to a variety of operating environments, in one or more embodiments of this specification, the input data of the performance prediction model corresponding to the operator includes operator parameter samples and information about the operating environment in which the operator is tested for performance, and the output data includes performance indicator data. In this embodiment, during training, since the operator's operating environment information is used as input data for training, the pattern of the operating environment's impact on operator performance can be learned. Thus, during the prediction phase, the method can be widely applied to predictions in different operating environments. This fully considers the software / hardware diversity and the complexity of software / hardware interactions in scenarios such as data centers, thereby improving the versatility and accuracy of detection.

[0109] In addition, as described in the embodiments of the deep learning model performance detection method for fusion operators in this specification, in one or more embodiments of this specification, the fusion operator can also be trained to obtain performance prediction models corresponding to the fusion operator in one or more operating environments. Since the performance of the fusion operator is closely related to the operating environment, in one or more embodiments of this specification, the input data of the performance prediction model corresponding to the operator may include operator parameter samples, operating environment information for the operator performance test, and fusion rules corresponding to the operator when the operator is a fusion operator, and the output data includes performance indicator data. In this embodiment, during training, since the operating environment information and fusion rules of the fusion operator are used as input data for model training, the model can learn the pattern of the impact of the operating environment and operator fusion rules on operator performance during the training process. Therefore, in the prediction stage, it can be widely applied to predictions in different operating environments, fully considering the software / hardware diversity of fusion operators and data centers and the complexity of software / hardware interactions, thereby improving the versatility and accuracy of detection.

[0110] In addition, in order to reduce the time required to predict the performance indicators of the entire model, in one or more embodiments of this specification, the method further includes, before training the performance prediction model corresponding to the operator:

[0111] Determining the operator complexity corresponding to the operator;

[0112] The performance prediction model is constructed according to the operator complexity, wherein the lower the operator complexity is, the smaller the corresponding performance prediction model is.

[0113] The operator complexity refers to the computational complexity of the operator. Operator complexity can be obtained using general operator complexity evaluation methods, and this specification does not limit this. In this embodiment, different model parameters are set according to the complexity level for different operator complexities. Therefore, when the operator complexity is lower, a smaller performance prediction model is constructed for prediction, which can effectively reduce the time required to predict the performance indicators of the entire model.

[0114] The following, in conjunction with Figure 6, further illustrates the application of the deep learning model performance detection method and prediction model generation method provided in this specification in predicting model operation delay.

[0115] In the implementation method of predicting the running delay of the model, the prediction model generation method may include two parts: operator fusion detection and operator sampling modeling. Operator fusion detection and operator sampling modeling can be performed offline. The deep learning model performance detection method can predict the running delay of the deep learning model to be tested online. For example, the performance prediction model of the operator can be generated through offline sampling modeling. In one or more embodiments, the performance prediction model can learn the influence pattern of the selected hardware environment and the fusion rules of the operator on the running delay through training, so that after the performance prediction model is placed in the online environment, it can predict the running delay of the operator of the deep learning model to be tested online based on the input operator parameters, running environment information, and the corresponding fusion rules in the presence of the fusion operator.

[0116] For the operator fusion detection part, the fusion rules of the operators in the prior deep learning model can be detected based on the black box and white box detection methods described in the above embodiments.

[0117] For operator sampling modeling, we can first build an operator parameter sampler based on prior knowledge. This operator parameter sampler can use the operator parameters used by common deep learning models as prior knowledge and, based on this prior knowledge, perform more intensive sampling near these parameters. Therefore, during the training phase, the results of each sampling iteration can be used to test the accuracy of the performance prediction model in that round of sampling. For samples with large prediction deviations, a new round of sampling will be initiated based on these samples. Ultimately, a performance prediction model capable of predicting operator latency can be trained.

[0118] For the online prediction part, the fusion rules obtained in the aforementioned operator fusion detection part can be used to disassemble the deep learning model to be tested to obtain individual non-fused operators and / or fused operators. Then, the trained performance prediction model can be used to predict the running delay of each individual non-fused operator and / or fused operator. The running delay prediction values ​​of these operators are then accumulated, and the accumulated result is used as the running delay of the entire deep learning model to be tested.

[0119] Specifically, Figure 6 shows a processing flow chart of a deep learning model performance detection method and a prediction model generation method provided by an embodiment of this specification, which specifically includes the following steps.

[0120] Step 602: Offline refers to completing the modeling of the performance prediction model offline.

[0121] Step 604: Operator fusion detection refers to detecting the fusion operator in the selected operating environment and outputting the operator fusion rules.

[0122] Based on the operator fusion rules, it is possible to distinguish between fused operators and non-fused operators so that the corresponding performance prediction models can be trained separately. It is also possible to distinguish between non-fused operators and fused operators that need to be predicted in the deep learning model to be tested so that the running delay can be predicted separately.

[0123] Step 606: Operator data collection refers to using the parameters of the operators of common deep learning models as prior knowledge and sampling a certain number of points near the prior knowledge.

[0124] Step 608: Modeling and training the performance prediction model of the operator refers to using the sampled points to train the performance prediction model corresponding to the operator.

[0125] During training, the performance prediction model predicts the sampled operators and selects the points where the predictions were incorrect as new prior knowledge for the next round of sampling and prediction. This process is repeated several times to obtain the final performance prediction model.

[0126] Step 610: Online refers to receiving a performance prediction request online according to the method provided in the embodiments of this specification.

[0127] Step 612: Model input refers to determining the deep learning model to be tested based on the performance prediction request.

[0128] Step 614: Splitting into operators refers to using the fusion rules obtained in the above offline process to decompose the operators of the depth model to be measured.

[0129] Step 616: Operator prediction refers to using the performance prediction model obtained through the offline training to predict the running delay of the operator of the depth model to be tested that is input online.

[0130] Step 618: Model output refers to accumulating the operation delay prediction values ​​of each operator as the comprehensive operation delay prediction value of the entire model.

[0131] In the above embodiment, an operator-by-operator prediction scheme was adopted, achieving greater versatility and more accurate prediction results. Using a tree-based gradient boosting deep learning model as the operator prediction performance model can achieve better prediction accuracy and faster prediction speed. Furthermore, based on the above offline training process, the runtime latency of deep learning models in various operating environments can be predicted with minimal deviation, making it widely applicable to deep learning model runtime latency prediction for different models and devices.

[0132] Corresponding to the above-mentioned prediction model generation method embodiment, this specification also provides a prediction model generation device embodiment. FIG7 shows a structural diagram of a prediction model generation device provided in one embodiment of this specification.

[0133] As shown in FIG7 , the device includes:

[0134] The sampling module 702 is configured to collect operator parameter samples and corresponding performance indicator data;

[0135] The training module 704 is configured to train the performance prediction model corresponding to the operator using the parameter samples and corresponding performance indicator data of the operator, and obtain a performance prediction model for predicting the performance indicator corresponding to the operator when the training stop condition is met.

[0136] In one or more embodiments of this specification, the sampling module includes:

[0137] A parameter sampling submodule is configured to sample parameter samples of operators from one or more prior deep learning models;

[0138] The performance data sampling submodule is configured to perform a performance test on the operator based on the parameter sample of the operator to obtain corresponding performance indicator data.

[0139] In one or more embodiments of this specification, the prediction model generating device further includes:

[0140] The update sampling module is configured to re-enter the step of sampling and obtaining parameter samples of the operator from one or more prior deep learning models according to the sampling point when the prediction deviation of the performance prediction model reaches the re-sampling condition.

[0141] In one or more embodiments of this specification, the input data of the performance prediction model corresponding to the operator includes parameter samples of the operator and operating environment information of the operator for performance testing, and the output data includes performance indicator data;

[0142] or,

[0143] The input data of the performance prediction model corresponding to the operator includes the parameter sample of the operator, the operating environment information of the operator for performance testing and the corresponding fusion rules when the operator is a fusion operator, and the output data includes performance indicator data.

[0144] In one or more embodiments of this specification, the prediction model generating device further includes:

[0145] The model construction module is configured to determine the operator complexity corresponding to the operator, and construct the performance prediction model according to the operator complexity, wherein the lower the operator complexity is, the smaller the corresponding performance prediction model is.

[0146] The above is a schematic diagram of a prediction model generation device according to this embodiment. It should be noted that the technical solution of this prediction model generation device and the technical solution of the prediction model generation method described above are based on the same concept. For details not described in detail in the technical solution of the prediction model generation device, please refer to the description of the technical solution of the prediction model generation method described above.

[0147] Figure 8 shows a block diagram of a computing device 800 according to one embodiment of this specification. Components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0148] The computing device 800 also includes an access device 840 that enables the computing device 800 to communicate via one or more networks 860. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, or a near field communication (NFC) interface.

[0149] In one embodiment of the present specification, the aforementioned components of computing device 800 and other components not shown in FIG8 may also be connected to each other, for example, via a bus. It should be understood that the computing device structure block diagram shown in FIG8 is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0150] Computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or personal computer (PC). Computing device 800 may also be a mobile or stationary server.

[0151] Among them, the processor 820 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned deep learning model performance detection method or prediction model generation method.

[0152] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of this computing device is based on the same concept as the technical solution of the above-mentioned deep learning model performance detection method or prediction model generation method. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the above-mentioned deep learning model performance detection method or prediction model generation method.

[0153] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned deep learning model performance detection method or prediction model generation method.

[0154] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium is based on the same concept as the technical solution of the deep learning model performance detection method or the prediction model generation method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the deep learning model performance detection method or the prediction model generation method described above.

[0155] An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned deep learning model performance detection method or prediction model generation method.

[0156] The above is an illustrative embodiment of a computer program. It should be noted that the technical solution of this computer program is based on the same concept as the technical solution of the deep learning model performance testing method or the prediction model generation method described above. For details not described in detail in the technical solution of the computer program, please refer to the description of the technical solution of the deep learning model performance testing method or the prediction model generation method described above.

[0157] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0158] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0159] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0160] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0161] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A deep learning model performance detection system, comprising: Client and server; The client is configured to send a performance prediction request to the server, wherein the performance prediction request is used to instruct to predict a comprehensive performance indicator of the deep learning model to be tested; The server is configured to receive the performance prediction request, obtain the corresponding deep learning model to be tested according to the performance prediction request, obtain one or more operators of the deep learning model to be tested, and use the performance prediction model corresponding to the operator to perform performance indicator prediction for the one or more operators to obtain the performance indicator prediction value corresponding to the operator, wherein the performance prediction model is trained by using the performance indicator data of the corresponding operator, and based on the performance indicator prediction values ​​corresponding to the one or more operators, the comprehensive performance indicator prediction value of the deep learning model to be tested is obtained.

2. A deep learning model performance detection method, comprising: Determine the deep learning model to be tested; Obtaining one or more operators of the deep learning model to be tested; For the one or more operators, use the performance prediction model corresponding to the operator to perform performance indicator prediction to obtain the performance indicator prediction value corresponding to the operator, wherein the performance prediction model is trained by using the performance indicator data of the corresponding operator; Based on the performance indicator prediction values ​​corresponding to the one or more operators, a comprehensive performance indicator prediction value of the deep learning model to be tested is obtained.

3. According to the method of claim 2, the performance indicator prediction value corresponding to the operator includes the operation delay of the operator; The step of obtaining a comprehensive performance indicator prediction value of the deep learning model to be tested based on the performance indicator prediction values ​​corresponding to the one or more operators includes: The performance indicator prediction values ​​corresponding to the one or more operators are accumulated to obtain a comprehensive performance indicator prediction value of the deep learning model to be tested.

4. According to the method of claim 2, the input data of the performance prediction model corresponding to the operator includes the parameters of the operator and the operating environment information of the deep learning model to be tested.

5. According to the method of claim 4, obtaining one or more operators of the deep learning model to be tested comprises: Analyze the deep learning model to be tested by using a model building tool to obtain a set of operators of the deep learning model to be tested; Detecting a fused operator and / or a non-fused operator from the set of operators, wherein the fused operator is an operator obtained by fusing multiple operators in the deep learning model to be tested, and the non-fused operator is an operator that does not participate in the fusion in the deep learning model to be tested; The input data of the performance prediction model corresponding to the non-fusion operator includes the parameters of the operator and the operating environment information of the deep learning model to be tested; The input data of the performance prediction model corresponding to the fusion operator includes the parameters of the operator, the operating environment information of the deep learning model to be tested, and the fusion rules between the multiple operators involved in the fusion.

6. A prediction model generation method, comprising: Collect operator parameter samples and corresponding performance indicator data; The performance prediction model corresponding to the operator is trained using the parameter samples of the operator and the corresponding performance indicator data, and when the training stop condition is met, a performance prediction model for predicting the performance indicator corresponding to the operator is obtained.

7. According to the method of claim 6, the collecting operator parameter samples and corresponding performance indicator data comprises: Sampling parameter samples of operators from one or more prior deep learning models; Based on the parameter samples of the operator, a performance test is performed on the operator to obtain corresponding performance indicator data.

8. The method according to claim 7, further comprising: For the sampling point where the prediction deviation of the performance prediction model reaches the re-sampling condition, re-enter the step of sampling from one or more prior deep learning models to obtain parameter samples of the operator according to the sampling point.

9. According to the method of claim 6, the input data of the performance prediction model corresponding to the operator includes parameter samples of the operator and operating environment information of the operator for performance testing, and the output data includes performance indicator data; or, The input data of the performance prediction model corresponding to the operator includes parameter samples of the operator, operating environment information of the operator for performance testing, and corresponding fusion rules when the operator is a fusion operator, and the output data includes performance indicator data.

10. The method according to claim 6, before training the performance prediction model corresponding to the operator, further comprising: Determining an operator complexity corresponding to the operator; The performance prediction model is constructed according to the operator complexity, wherein the lower the operator complexity is, the smaller the corresponding performance prediction model is.

11. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the deep learning model performance detection method described in any one of claims 2 to 5 are implemented, or the steps of the prediction model generation method described in any one of claims 6 to 10 are implemented.

12. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the deep learning model performance detection method described in any one of claims 2 to 5, or implement the steps of the prediction model generation method described in any one of claims 6 to 10.

13. A computer program, wherein: When the computer program is executed in a computer, the computer is caused to execute the steps of the deep learning model performance detection method described in any one of claims 2 to 5, or execute the steps of the prediction model generation method described in any one of claims 6 to 10.

Citation Information

Patent Citations

  • Deep learning model training-oriented performance prediction method and related equipment

    CN114816954A

  • Method and device for generating algorithm model of vehicle-mounted terminal processor, equipment and medium

    CN116070691A

  • Deep learning model performance detection system and method and prediction model generation method

    CN116578465A

  • Predicting Machine-Learned Model Performance from the Parameter Values of the Model

    US20210256422A1

Cited By

  • Model parameter fusion method and device

    CN119720101A

  • Graphics processor energy efficiency prediction method and device, equipment and storage medium

    CN120510021A

  • Intelligent optimization method and system for metal additive manufacturing process parameters based on Bayesian optimization

    CN121146142A