A model time consumption prediction method and device, equipment and storage medium

Abstract

The application discloses a model time consumption prediction method and device, equipment and a storage medium, which comprises the following steps: obtaining a to-be-detected model and obtaining device information of a target peripheral corresponding to the to-be-detected model; generating an operator list based on the to-be-detected model, wherein the operator list comprises a plurality of to-be-tested operators; obtaining a preliminary time consumption prediction result of the to-be-detected model by predicting the time consumption result of each to-be-tested operator; obtaining a model coefficient based on the to-be-detected model and the device information and through a pre-trained random forest model; and taking the product of the model coefficient and the preliminary time consumption prediction result as the model time consumption prediction result of the to-be-detected model. The time consumption prediction accuracy under different peripherals can reach the expected effect.

Description

Technical Field

[0001] This invention relates to the field of electronic equipment technology, and in particular to a method, apparatus, device, and storage medium for predicting model time consumption. Background Technology

[0002] With the increasing deployment of deep learning on devices, latency has become a key metric for running deep neural network models on various mobile and edge devices. Previous assessments of hardware efficiency have largely used FLOPS as a metric or to predict efficiency, but these approaches are inaccurate because the deployment chain is long and involves issues such as the hardware and software optimization characteristics of the deployment framework, creating a complex system. While conducting latency tests directly on devices is a feasible method, the cost of such tests on real devices is too high. For example, searching for efficient deep neural network models from a vast model design space and deploying these models is a time-consuming and resource-intensive process, which is extremely challenging.

[0003] To address the challenges of latency prediction, Tsinghua University and Microsoft jointly developed nn-meter, a mobile hardware latency predictor for deep learning. nn-meter consists of two main parts: kernel fusion rules and a latency predictor. Kernel fusion rules primarily handle the mapping between the training framework's model graph and the deployment framework's kernel graph. Since the fusion rules between operators differ across devices and frameworks, and these rules significantly impact actual runtime, it's crucial to differentiate between device-specific fusion rules. The latency predictor, on the other hand, predicts the latency of each operator and then sums them up.

[0004] nn-meter achieved good prediction accuracy on CPU and GPU, but its performance on VPU was not ideal. The main reason is that the operator is multiplied by a fixed coefficient during the final accumulation of operator time. The fixed coefficient cannot be adapted to all peripherals, which leads to the prediction accuracy not reaching the expected effect on some peripherals. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device and storage medium for predicting model time consumption, in order to solve the problem of low accuracy in existing model time consumption prediction.

[0006] To achieve the above objectives, the present invention provides a method for predicting model time consumption, the method comprising:

[0007] Obtain the model to be detected and the device information of the target peripheral device corresponding to the model to be detected;

[0008] A list of operators is generated based on the model to be tested, and the list of operators includes multiple operators to be tested;

[0009] By predicting the time consumption of each of the operators to be tested, a preliminary time consumption prediction result of the model to be tested is obtained.

[0010] Based on the model to be detected and the device information, the model coefficients are obtained through a pre-trained random forest model;

[0011] The product of the model coefficients and the preliminary time prediction result is used as the model time prediction result of the model to be tested.

[0012] Preferably, the step of generating an operator list based on the model to be detected includes multiple operators to be tested, including:

[0013] The model to be detected is parsed to obtain a first list of operators with multiple operators;

[0014] Each operator is fused based on the device information to generate an operator list that includes multiple operators to be tested.

[0015] Preferably, the step of predicting the time consumption result of each of the tested operators to obtain the preliminary time consumption prediction result of the model to be tested includes:

[0016] The random forest model is used to predict the time consumption of each operator under the target peripheral.

[0017] The time consumption of each of the operators to be tested is multiplied by a first coefficient and then summed to obtain the preliminary time consumption prediction result of the model to be tested; wherein, the first coefficient is obtained by training the random forest model by inputting preset parameters, the preset parameters including the operator type of the operator to be tested, the attribute parameters of the operator to be tested, and the size information, the size information including the size of the input data of the model to be tested.

[0018] Preferably, obtaining the model coefficients based on the model to be detected and the device information using a pre-trained random forest model includes:

[0019] Calculate the computational density of the model to be detected and obtain the upper limit of the computational density of the target peripheral;

[0020] The computation density and the upper limit of the computation density are input into the random forest model to obtain the model coefficients.

[0021] Preferably, calculating the computational density of the model to be detected includes:

[0022] The computation density of the model under test is calculated by dividing the computational cost of the model under test by the memory access cost of the model under test. The computational cost of the model under test is the total number of computations required to complete the model under test, and the memory access cost of the model under test is the amount of data that needs to be stored / retrieved during the computation of the model under test.

[0023] Preferably, obtaining the upper limit of the computing density of the target peripheral includes:

[0024] The upper limit of the computing density of the target peripheral is obtained based on the device model in the device information.

[0025] To achieve the above objectives, the present invention also provides a model time prediction device, the device comprising:

[0026] The acquisition unit is used to acquire the model to be detected and the device information of the target peripheral device corresponding to the model to be detected.

[0027] A generation unit is used to generate an operator list based on the model to be detected, the operator list including multiple operators to be tested;

[0028] The first prediction unit is used to obtain the preliminary time prediction result of the model to be tested by predicting the time consumption result of each of the operators to be tested.

[0029] The coefficient calculation unit is used to obtain model coefficients based on the model to be detected and the device information, and through a pre-trained random forest model.

[0030] The second prediction unit is used to multiply the model coefficients and the preliminary time prediction result as the model time prediction result of the model to be detected.

[0031] To achieve the above objectives, the present invention also proposes an apparatus comprising a processor, a memory, and a computer program stored in the memory, the computer program being executed by the processor to implement the steps of a model time prediction method as described in the above embodiments.

[0032] To achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a computer program that is executed by a processor to implement the steps of a model time prediction method as described in the above embodiments.

[0033] Beneficial effects:

[0034] The above solution, by acquiring device information of peripherals and using a random forest model to obtain the model coefficients of the model to be tested, can reduce the time-consuming uncertainty in the process of manually selecting coefficients, thereby achieving the expected effect in predicting the time consumption under different peripherals.

[0035] The above solution parses the model to be detected into a list of individual operators and then fuses the operators based on the device information. The resulting fused operators are closer to the actual operating conditions of the peripheral devices, thereby improving the accuracy of model time prediction.

[0036] The above scheme predicts the time consumption of a single operator under peripheral devices by using a trained random forest model. It then multiplies each operator by a corresponding coefficient and sums the results to obtain a preliminary time consumption prediction for the model under test. By multiplying by the set coefficients, the prediction can be more closely approximated to the actual situation, thus improving the prediction accuracy.

[0037] The above scheme, by introducing an upper limit on the computational intensity of peripherals, can better fit the actual working conditions of the model under test in the peripherals and improve the accuracy of model time prediction. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating a model time prediction method according to an embodiment of the present invention.

[0040] Figure 2 This is a flowchart illustrating a model time prediction method provided in another embodiment of the present invention.

[0041] Figure 3 This is a schematic diagram of a model time prediction device provided in an embodiment of the present invention.

[0042] The realization of the invention's objective, its functional characteristics, and advantages will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0045] The present invention will be described in detail below with reference to the embodiments.

[0046] Reference Figure 1 and Figure 2 The diagram shown is a flowchart of a model time prediction method provided by the present invention.

[0047] In this embodiment, the method includes:

[0048] S11, Obtain the model to be detected and obtain the device information of the target peripheral device corresponding to the model to be detected.

[0049] S12, Generate an operator list based on the model to be tested, the operator list including multiple operators to be tested.

[0050] This step further includes:

[0051] S12-1, The model to be detected is analyzed to obtain a first operator list with multiple operators;

[0052] S12-2, each of the operators is fused according to the device information to generate an operator list including multiple operators to be tested.

[0053] In this embodiment, the model obtained in step S11 is parsed into a list of individual operators. Each operator is then fused according to the peripheral device to generate a new operator list. The operator fusion method involves traversing all layers of the model and determining whether subsequent layers of the current layer can be fused into that current layer (e.g., fusion of conv+relu layers will result in a new operator list without conv and relu layers, replaced by a new convrelu layer). This fusion method is not universal; for example, it may be fused on a CPU but not on OpenGL. The fusion strategy is adjusted based on whether fusion is hardcoded during actual device operation. The purpose of operator fusion is to closely approximate actual runtime conditions and improve accuracy.

[0054] S13, by predicting the time consumption result of each of the operators to be tested, the preliminary time consumption prediction result of the model to be tested is obtained.

[0055] This step further includes:

[0056] S13-1, The time consumption of each operator to be tested under the target peripheral is predicted by the random forest model;

[0057] S13-2, the time consumption of each of the operators to be tested is multiplied by a first coefficient and then accumulated to obtain the preliminary time consumption prediction result of the model to be tested; wherein, the first coefficient is obtained by training the random forest model by inputting preset parameters, the preset parameters including the operator type of the operator to be tested, the attribute parameters of the operator to be tested, and the size information, the size information including the size of the input data of the model to be tested.

[0058] In this embodiment, by traversing all the operators to be tested in the fused operator list and initializing the statistical variable sum of the preliminary time prediction result to sum = 0, a single operator to be tested is predicted using a trained random forest model to obtain the time consumption of a single operator under the target peripheral device. The time consumption of the corresponding operator to be tested is multiplied by the corresponding coefficient and accumulated into sum. After traversing all the operators to be tested and performing the accumulation calculation, the predicted value corresponding to the preliminary time prediction result of the model to be tested is obtained. Among them, the coefficient is obtained by training the random forest model. Since the time consumption prediction in this step is trained based on a single layer, and the initialization time needs to be deducted during actual operation, since the proportion of initialization time is different under different devices, it is necessary to multiply by this coefficient to obtain a value that is closer to the actual situation. The parameters input to the random forest model include operator type, operator attribute parameters, and size information. The size information includes the size of the input data of the model to be tested (e.g., image [1,3,512,512]).

[0059] S14. Based on the model to be detected and the device information, obtain the model coefficients through a pre-trained random forest model.

[0060] This step further includes:

[0061] S14-1, Calculate the computational density of the model to be detected and obtain the upper limit of the computational density of the target peripheral;

[0062] S14-2, Input the computation density and the upper limit of the computation density into the random forest model to obtain the model coefficients.

[0063] In step S14-1, calculating the computational density of the model to be detected includes:

[0064] The computation density of the model under test is calculated by dividing the computational cost of the model under test by the memory access cost of the model under test. The computational cost of the model under test is the total number of computations required to complete the model under test, and the memory access cost of the model under test is the amount of data that needs to be stored / retrieved during the computation of the model under test.

[0065] In step S14-1, obtaining the upper limit of the computing density of the target peripheral includes:

[0066] The upper limit of the computing density of the target peripheral is obtained based on the device model in the device information.

[0067] In this embodiment, the computational density of the model to be detected is calculated, specifically as follows:

[0068] The computation density I = computational cost (FLOPs) / memory access (Bytes). The computational cost is the total number of computations required to complete the model to be tested, and the memory access is the amount of data that needs to be stored / retrieved during the model's computation. Furthermore, the upper limit of the computational density for the target peripheral is obtained based on its device model and memory size. This upper limit is pre-collected, and the device model is obtained from a dictionary. The computational density of the model to be tested and the upper limit of the target peripheral's computational density are then input into the trained random forest model to obtain the corresponding overall model coefficients 'a'.

[0069] S15, the product of the model coefficients and the preliminary time prediction result is used as the model time prediction result of the model to be tested.

[0070] This solution primarily employs a random forest model to obtain the coefficients, reducing the time and uncertainty associated with manually selecting coefficients and improving the accuracy of model time prediction under different peripherals. Furthermore, by introducing computational density and an upper limit for computational density, the solution better fits the actual working conditions of the model on the corresponding peripherals, thereby further improving the accuracy of model time prediction.

[0071] Reference Figure 3 The diagram shown is a structural schematic of a model time prediction device provided in an embodiment of the present invention.

[0072] In this embodiment, the device 30 includes:

[0073] The acquisition unit 31 is used to acquire the model to be detected and the device information of the target peripheral device corresponding to the model to be detected.

[0074] The generation unit 32 is used to generate an operator list based on the model to be detected, the operator list including multiple operators to be tested;

[0075] The first prediction unit 33 is used to predict the time consumption result of each of the operators to be tested to obtain the preliminary time consumption prediction result of the model to be tested.

[0076] The coefficient calculation unit 34 is used to obtain model coefficients based on the model to be detected and the device information and through a pre-trained random forest model;

[0077] The second prediction unit 35 is used to take the product of the model coefficients and the preliminary time prediction result as the model time prediction result of the model to be detected.

[0078] The generation unit 32 further includes:

[0079] The parsing unit is used to parse the model to be detected and obtain a first operator list with multiple operators;

[0080] The fusion unit is used to fuse each of the operators according to the device information to generate an operator list including multiple operators to be tested.

[0081] The first prediction unit 33 further includes:

[0082] The operator prediction unit is used to predict the time consumption of each operator under the target peripheral through the random forest model.

[0083] The preliminary prediction unit is used to multiply the time consumption of each of the operators to be tested by a first coefficient and then accumulate them to obtain the preliminary time consumption prediction result of the model to be tested; wherein, the first coefficient is obtained by training the random forest model by inputting preset parameters, the preset parameters including the operator type of the operator to be tested, the attribute parameters of the operator to be tested, and the size information, the size information including the size of the input data of the model to be tested.

[0084] The coefficient calculation unit 34 further includes:

[0085] The first computing unit is used to calculate the computing density of the model to be detected and to obtain the upper limit of the computing density of the target peripheral.

[0086] The second calculation unit is used to input the calculation density and the upper limit of the calculation density into the random forest model to obtain the model coefficients.

[0087] Furthermore, calculating the computational density of the model to be detected includes:

[0088] The computation density of the model under test is calculated by dividing the computational cost of the model under test by the memory access cost of the model under test. The computational cost of the model under test is the total number of computations required to complete the model under test, and the memory access cost of the model under test is the amount of data that needs to be stored / retrieved during the computation of the model under test.

[0089] Furthermore, obtaining the upper limit of the computing density of the target peripheral includes:

[0090] The upper limit of the computing density of the target peripheral is obtained based on the device model in the device information.

[0091] Each unit module of the device 30 can execute the corresponding steps in the above method embodiment, so the details of each unit module will not be elaborated here. Please refer to the description of the corresponding steps above for details.

[0092] This invention also provides a device comprising the model time prediction apparatus as described above, wherein the model time prediction apparatus can employ... Figure 3 The structure of the embodiment, correspondingly, can be executed Figure 1 The technical solutions of the method embodiments shown are similar in implementation principle and technical effect. For details, please refer to the relevant records in the above embodiments, which will not be repeated here.

[0093] The device includes: a mobile phone, digital camera, or tablet computer, or other device with a camera function; or a device with an image processing function; or a device with an image display function. The device may include components such as a memory, processor, input unit, display unit, and power supply.

[0094] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for at least one function (such as image playback function), etc.; the data storage area can store data created according to the use of the device. In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory can also include a memory controller to provide access to the memory for the processor and input units.

[0095] The input unit can be used to receive input numerical, character, or image information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, in addition to a camera, the input unit of this embodiment may also include a touch-sensitive surface (e.g., a touch screen) and other input devices.

[0096] The display unit can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. The display unit may include a display panel, optionally configured as an LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or other similar display panel. Furthermore, a touch-sensitive surface may cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it transmits the information to the processor to determine the type of touch event. Subsequently, the processor provides corresponding visual output on the display panel based on the type of touch event.

[0097] This invention also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the memory described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement... Figure 1 The model time prediction method is shown. The computer-readable storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0098] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the device embodiments, equipment embodiments, and storage medium embodiments, since they are basically similar to the method embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions in the method embodiments.

[0099] Furthermore, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0100] The foregoing description illustrates and describes preferred embodiments of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the inventive concept by means of the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A model time-consuming prediction method, characterized in that, The method comprises: obtaining a to-be-detected model and obtaining device information of a target peripheral device corresponding to running the to-be-detected model; generating an operator list based on the to-be-detected model, the operator list comprising a plurality of to-be-tested operators; obtaining a preliminary time consumption prediction result of the to-be-detected model by predicting a time consumption result of each to-be-tested operator; obtaining a model coefficient based on the to-be-detected model and the device information and through a pre-trained random forest model, wherein the obtaining of the model coefficient based on the to-be-detected model and the device information and through the pre-trained random forest model comprises: calculating a calculation density of the to-be-detected model and obtaining an upper limit of the calculation density of the target peripheral device; inputting the calculation density and the upper limit of the calculation density into the random forest model to obtain the model coefficient; and multiplying the model coefficient and the preliminary time consumption prediction result to obtain a model time consumption prediction result of the to-be-detected model.

2. The model time-consumption prediction method of claim 1, wherein, The generating of the operator list based on the to-be-detected model, the operator list comprising a plurality of to-be-tested operators, comprises: analyzing the to-be-detected model to obtain a first operator list comprising a plurality of operators; fusing each operator according to the device information to generate an operator list comprising a plurality of to-be-tested operators.

3. The model time-consuming prediction method according to claim 1 or 2, characterized in that, The obtaining of the preliminary time consumption prediction result of the to-be-detected model by predicting a time consumption result of each to-be-tested operator comprises: predicting a time consumption of each to-be-tested operator under the target peripheral device through the random forest model; multiplying the time consumption of each to-be-tested operator by a first coefficient and then accumulating to obtain the preliminary time consumption prediction result of the to-be-detected model, wherein the first coefficient is obtained by inputting a preset parameter into the random forest model for training, the preset parameter comprising an operator type of the to-be-tested operator, an attribute parameter of the to-be-tested operator, and size information, the size information comprising a size of input data of the to-be-detected model.

4. The method of claim 1, wherein, The calculation of the calculation density of the to-be-detected model comprises: calculating the calculation density of the to-be-detected model according to a calculation amount of the to-be-detected model and a memory access amount of the to-be-detected model, wherein the calculation amount of the to-be-detected model is a total number of calculations required for running the to-be-detected model, and the memory access amount of the to-be-detected model is an amount of data required to be stored or accessed in a calculation process of the to-be-detected model.

5. The method of claim 1, wherein, The obtaining of the upper limit of the calculation density of the target peripheral device comprises: obtaining the upper limit of the calculation density of the target peripheral device according to a device model in the device information.

6. A model time-consumption prediction apparatus characterized by comprising: The device comprises: an obtaining unit, configured to obtain a to-be-detected model and obtain device information of a target peripheral device corresponding to running the to-be-detected model; a generating unit, configured to generate an operator list based on the to-be-detected model, the operator list comprising a plurality of to-be-tested operators; a first predicting unit, configured to obtain a preliminary time consumption prediction result of the to-be-detected model by predicting a time consumption result of each to-be-tested operator; and A coefficient calculation unit is configured to obtain model coefficients based on the to-be-detected model and the device information and through a pre-trained random forest model. A first calculation unit is configured to calculate a calculation density of the to-be-detected model and obtain an upper limit of the calculation density of the target peripheral device. A second calculation unit is configured to input the calculation density and the upper limit of the calculation density into the random forest model to obtain the model coefficients. A second prediction unit is configured to take a product of the model coefficients and the preliminary time consumption prediction result as a model time consumption prediction result of the to-be-detected model.

7. An apparatus, comprising: A computer program is stored in the memory and is executed by the processor to implement the steps of the model time consumption prediction method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, A computer program is stored in the memory and is executed by the processor to implement the steps of the model time consumption prediction method according to any one of claims 1 to 5.

Images