Model processing path determination method and device, equipment, medium and program product

Abstract

The invention provides a model processing path determination method and device, equipment, a medium and a program product, and can be applied to the technical field of artificial intelligence and the technical field of computers. The method comprises the steps that a first processing path file is generated based on a default processing path used in the process of processing input data through a target model, and the first processing path file comprises the default processing path and default operation duration; calling a plurality of candidate sub-paths from an acceleration library by using an offline model; according to the plurality of candidate sub-paths, adjusting a default processing path according to a path adjustment strategy to obtain a plurality of candidate processing paths; processing the input data in sequence according to each candidate processing path by using the target model to obtain candidate operation duration corresponding to each candidate processing path; and determining a target optimization processing path corresponding to the target model from the plurality of candidate processing paths based on the candidate operation duration and the default operation duration corresponding to each candidate processing path.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence technology and computer technology, and more specifically, to a method, device, equipment, medium and program product for determining a model processing path. Background Art

[0002] GEMM (General Matrix to Matrix Multiplication) is the basis of many components of neural networks, so executing GEMM as quickly and efficiently as possible is critical to improving the performance of ML (Machine Learning) workloads.

[0003] There may be multiple BLAS libraries (Basic Linear Algebra Subprograms) on the system, and each BLAS library may have multiple different algorithms to solve a given GEMM, and each algorithm has hundreds of possible unique parameter combinations. In other words, even for a given GEMM, there may be thousands of different ways to solve it and each method has different performance.

[0004] Therefore, how to select the fastest algorithm for GEMM to improve performance is a technical problem that needs to be solved in the prior art. Summary of the invention

[0005] In view of this, the present disclosure provides a method, apparatus, device, medium and program product for determining a model processing path.

[0006] One aspect of the present disclosure provides a method for determining a model processing path, comprising:

[0007] Generate a first processing path file based on a default processing path used in a process of processing input data by a target model, wherein the first processing path file includes a default processing path and a default running time, and the default running time represents the running time of the target model processing the input data according to the default processing path;

[0008] Using the offline model, multiple candidate subpaths are called from the acceleration library;

[0009] According to the multiple candidate sub-paths, the default processing path is adjusted according to the path adjustment strategy to obtain multiple candidate processing paths, wherein the default processing path includes multiple default processing sub-paths; the path adjustment strategy represents a strategy for replacing at least one of the default processing sub-paths in the default processing path with the candidate sub-path;

[0010] Processing the input data in turn according to each of the candidate processing paths using the target model to obtain a candidate running time corresponding to each of the candidate processing paths;

[0011] Based on the candidate running time corresponding to each of the candidate processing paths and the default running time, a target optimized processing path corresponding to the target model is determined from the plurality of candidate processing paths.

[0012] According to an embodiment of the present disclosure, the above-mentioned default processing path is adjusted according to the above-mentioned multiple candidate sub-paths according to the path adjustment strategy to obtain multiple candidate processing paths, including:

[0013] In the case where the path adjustment strategy represents replacing n of the default processing sub-paths in the default processing path, the multiple candidate sub-paths are grouped into multiple candidate sub-path groups, wherein each of the candidate sub-path groups includes n candidate sub-paths, the default processing path includes N of the default processing sub-paths, n is an integer greater than or equal to 1 and less than or equal to N, and N is a positive integer; and

[0014] For each of the above-mentioned candidate sub-path groups, use the above-mentioned candidate sub-path groups to replace any n default processing sub-paths in the above-mentioned default processing path, obtain multiple candidate processing paths corresponding to each of the above-mentioned candidate sub-path groups, and increase n incrementally until n takes the value of N to obtain the above-mentioned multiple candidate processing paths.

[0015] According to an embodiment of the present disclosure, based on the candidate runtime corresponding to each of the candidate processing paths and the default runtime, determining the target optimization processing path corresponding to the target model from the multiple candidate processing paths includes repeatedly performing the following operations until the loop rounds meet the preset conditions:

[0016] If the above cycle does not meet the above preset conditions,

[0017] Compare the candidate running time corresponding to the mth candidate processing path with the target candidate running time to obtain a comparison result, wherein when the value of m is 1, the target candidate running time is the default running time;

[0018] According to the above comparison results, determine the new target candidate running time and increment m;

[0019] The new target candidate running time obtained when the above cycle rounds satisfy the above preset conditions is determined as the target optimized running time, and the candidate processing path corresponding to the above target optimized running time is determined as the above target optimized processing path.

[0020] According to an embodiment of the present disclosure, determining a new target candidate running time according to the comparison result includes:

[0021] When the comparison result indicates that the candidate running time corresponding to the m-th candidate processing path is less than the target candidate running time, the candidate running time corresponding to the m-th candidate processing path is determined as the new target candidate running time.

[0022] According to an embodiment of the present disclosure, the method for determining the model processing path further includes:

[0023] When the comparison result indicates that the candidate running time corresponding to the m-th candidate processing path is greater than or equal to the target candidate running time, the target candidate running time is determined as the new target candidate running time.

[0024] According to an embodiment of the present disclosure, the method for determining the model processing path further includes:

[0025] For the target model, the target optimization processing path corresponding to the target model is recorded to obtain a second processing path file;

[0026] For other types of models, the target optimized processing paths corresponding to the default processing paths of the other types of models are cumulatively recorded in the second processing path file.

[0027] According to an embodiment of the present disclosure, the method for determining the model processing path further includes:

[0028] In the case of using the target model for online application, reading the target optimization processing path corresponding to the target model from the second processing path file; and

[0029] In the above target model, the data to be processed is processed according to the target optimization processing path corresponding to the above target model to achieve performance acceleration of the above target model.

[0030] Another aspect of the present disclosure provides a device for determining a model processing path, including a generation module, a calling module, an adjustment module, an acquisition module and a determination module.

[0031] Wherein, the generation module is used to generate a first processing path file based on the default processing path used in the process of processing the input data by the target model, wherein the first processing path file includes the default processing path and the default running time, and the default running time represents the running time of the target model processing the input data according to the default processing path;

[0032] A calling module is used to call multiple candidate sub-paths from the acceleration library using an offline model;

[0033] an adjustment module, configured to adjust the default processing path according to the plurality of candidate sub-paths and a path adjustment strategy to obtain a plurality of candidate processing paths, wherein the default processing path includes a plurality of default processing sub-paths; the path adjustment strategy represents a strategy for replacing at least one of the default processing sub-paths in the default processing path with the candidate sub-path;

[0034] An acquisition module, configured to use the target model to process the input data in turn according to each of the candidate processing paths to obtain a candidate running time corresponding to each of the candidate processing paths;

[0035] A determination module is used to determine a target optimization processing path corresponding to the target model from the plurality of candidate processing paths based on the candidate running time corresponding to each of the candidate processing paths and the default running time.

[0036] Another aspect of the present disclosure provides an electronic device, comprising:

[0037] one or more processors;

[0038] a memory for storing one or more programs,

[0039] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0040] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0041] Another aspect of the present disclosure provides a computer program product, which includes computer executable instructions, and when the instructions are executed, are used to implement the method as described above.

[0042] According to the method, apparatus, device, medium and program product for determining the model processing path provided by the present disclosure, since the subsequent processing is to be performed offline, the default processing path can be recorded to generate a first processing path file, so as to facilitate the subsequent direct reading of the data recorded in the first processing path file to determine the target optimized processing path; in the offline state, multiple candidate sub-paths are called from the acceleration library, and based on the path adjustment strategy, the default processing path is adjusted using the multiple candidate sub-paths to obtain multiple candidate processing paths, and the input data is processed according to the candidate processing paths through the target model to obtain the corresponding candidate running time, so that based on the running time corresponding to each candidate processing path, the target optimized processing path corresponding to the shortest running time can be determined from the multiple candidate processing paths, so that when the target model processes the data according to the target optimized processing path, the speed of the model processing the input data can be increased, thereby achieving improved model performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0044] Figure 1 An exemplary system architecture to which the method for determining a model processing path disclosed in the present invention can be applied is schematically shown;

[0045] Figure 2 A flowchart schematically illustrates a method for determining a model processing path according to an embodiment of the present disclosure;

[0046] Figure 3 A schematic diagram schematically shows the form of a GEMM according to an embodiment of the present disclosure;

[0047] Figure 4 A schematic diagram of a system for implementing a method for determining a model processing path according to an embodiment of the present disclosure is schematically shown;

[0048] Figure 5 A block diagram schematically shows a device for determining a model processing path according to an embodiment of the present disclosure; and

[0049] Figure 6 A block diagram of an electronic device suitable for implementing a method for determining a model processing path according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0050] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0051] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0052] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0053] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0054] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0055] In the embodiments of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information. In the process of implementing the present disclosure, it is found that GEMM is the basic building block of many components of neural networks, including fully connected layers, convolutions, attention mechanisms, etc. GEMM is also one of the several components of the Basic Linear Algebra Subroutines (BLAS) library (such as hipBLAS).

[0056] When running a matrix multiplication C=A@B in PyTorch (an open source deep learning framework for machine learning and deep learning), PyTorch does this by passing matrices A and B to the BLAS library; the BLAS library uses built-in heuristics (usually a lookup table) to select an algorithm to solve the given GEMM; the BLAS library computes the result based on the selected algorithm and passes it to C; the result is also passed back to PyTorch.

[0057] However, when performing matrix multiplication in PyTorch, it is not known which GEMM algorithm is the fastest, and these heuristics built into the BLAS library do not always choose the fastest algorithm, but may choose an algorithm with higher priority, which may depend on many factors, including the specific environment, architecture, etc.

[0058] Therefore, how to select the fastest algorithm for GEMM to improve performance is a technical problem that needs to be solved in the prior art.

[0059] To this end, an embodiment of the present disclosure provides a method for determining a model processing path, including: generating a first processing path file based on a default processing path used in a process of processing input data by a target model, wherein the first processing path file includes a default processing path and a default runtime, and the default runtime represents the runtime of the target model processing input data according to the default processing path; using an offline model, calling multiple candidate sub-paths from an acceleration library; adjusting the default processing path according to a path adjustment strategy based on the multiple candidate sub-paths to obtain multiple candidate processing paths, wherein the default processing path includes multiple default processing sub-paths; the path adjustment strategy represents a strategy for replacing at least one default processing sub-path with a candidate sub-path for multiple default processing sub-paths in the default processing path; using the target model to process input data according to each candidate processing path in turn, to obtain a candidate runtime corresponding to each candidate processing path; and determining a target optimization processing path corresponding to the target model from multiple candidate processing paths based on the candidate runtime and the default runtime corresponding to each candidate processing path.

[0060] Figure 1 The exemplary system architecture 100 of the method for determining the model processing path according to the embodiment of the present disclosure is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0061] like Figure 1As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0062] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).

[0063] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0064] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0065] For example, a first processing path file can be generated by the server 105 based on the default processing path used in the process of processing input data according to the target model, wherein the first processing path file includes a default processing path and a default running time; using the offline model, multiple candidate sub-paths are called from the acceleration library to adjust the default processing path according to the path adjustment strategy based on the multiple candidate sub-paths to obtain multiple candidate processing paths, and the target model is used to process the input data according to each candidate processing path in turn to obtain the candidate running time corresponding to each candidate processing path, so that the target optimized processing path corresponding to the target model can be determined from the multiple candidate processing paths based on the candidate running time and the default running time corresponding to each candidate processing path.

[0066] It should be noted that the method for determining the model processing path provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the device for determining the model processing path provided in the embodiment of the present disclosure can generally be set in the server 105. The method for determining the model processing path provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the device for determining the model processing path provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the device for determining the model processing path provided in the embodiment of the present disclosure can also be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103, or can also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103. Correspondingly, the device for determining the model processing path provided in the embodiment of the present disclosure may also be arranged in the first terminal device 101, the second terminal device 102 or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103.

[0067] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0068] The following will be based on Figure 1 The scene described by Figure 2~Figure 4 The method for determining the model processing path of the embodiment of the present disclosure is described in detail.

[0069] Figure 2 The flowchart of the method for determining the model processing path according to the embodiment of the present disclosure is schematically shown.

[0070] like Figure 2 As shown, the method 200 includes operations S210 to S250.

[0071] In operation S210, a first processing path file is generated based on a default processing path used in a process of processing input data by a target model.

[0072] The first processing path file may include a default processing path and a default running time, and the default running time represents the running time for the target model to process input data according to the default processing path.

[0073] According to an embodiment of the present disclosure, the target model includes but is not limited to LLM (Large Language Model). The input data includes but is not limited to text data, for example, the input data may be a sentence or a question.

[0074] According to an embodiment of the present disclosure, input data is input into the target model, and various operators such as addition, multiplication, and normalization are used in the process of the target model processing the input data, and the sizes of the operators are also different. That is, the default processing path can represent the various operators used in sequence in the process of the target model processing the input data.

[0075] According to an embodiment of the present disclosure, the default processing path used in the process of the target model processing input data can be recorded to generate a first processing path file. The default processing path may include multiple default processing sub-paths, each of which may correspond to an operator used in the target model processing input data, that is, in the first processing path file, a line of text may be recorded for each default processing sub-path, and the line of text may include the name of the default processing sub-path and the corresponding size.

[0076] Figure 3 A schematic diagram schematically shows the form of a GEMM according to an embodiment of the present disclosure.

[0077] like Figure 3 As shown, A and B are input matrices (which can be transposed), and C is a pre-existing matrix used as an output buffer. Matrices A, B, and C can be defined as (M, K), (K, N), and (M, N), respectively. Therefore, a single GEMM can be defined by the following input parameters: transA, transB, M, N, K, and these parameters are collectively referred to as the problem size, that is, a GEMM can be defined by the problem size.

[0078] Among them, transA can indicate whether matrix A should be transposed. If transA is true or "T", then it is transposed, otherwise it is not transposed; transB indicates whether matrix B should be transposed. If transB is true or "T", then it is transposed, otherwise it is not transposed; M indicates the number of rows of the result matrix C, N indicates the number of columns of the result matrix C, K indicates the number of columns (transA is false) or rows (transA is true) of matrix A; the number of rows (transB is false) or columns (transB is true) of matrix B.

[0079] For example, for a default processing sub-path, it can be recorded in the first processing path file in the form of GemmTunableOp_float_TN,tn_200_100_20, where GemmTunableOp_float_TN can represent the name of the operator, and 200, 100, 20 can represent the size of the operator, that is, 200, 100, 20 can correspond to M, N, and K respectively.

[0080] According to an embodiment of the present disclosure, the first processing path file may also record the runtime of the target model processing input data according to the default processing path. For each default processing sub-path, the first processing path file may also record the runtime of the target model processing data according to the default processing sub-path. For example, the first processing path file may be named llm-untuned.csv.

[0081] In operation S220 , a plurality of candidate sub-paths are called from an acceleration library using an offline model.

[0082] According to an embodiment of the present disclosure, the acceleration library may include rocBLAS, hipBLASLt, etc. The acceleration library includes multiple operators to solve a given GEMM, and each operator may have hundreds of possible parameter combinations, that is, a candidate sub-path corresponds to an operator and a parameter combination corresponding to the operator, and the parameter combination may be the size of the operator.

[0083] According to the embodiments of the present disclosure, since there are multiple candidate subpaths in the acceleration library, when the optimal operator is determined online in real time through the acceleration library, that is, when the operator with the fastest calculation speed is determined online in real time, it is very time-consuming and inefficient because the optimal operator is determined while performing calculations. At the same time, it often causes insufficient video memory and frequent errors. Therefore, the present disclosure uses an offline model to determine the optimal operator, that is, the optimal operator is determined offline.

[0084] In operation S230 , the default processing path is adjusted according to the path adjustment policy based on the multiple candidate sub-paths to obtain multiple candidate processing paths.

[0085] The default processing path includes multiple default processing sub-paths; the path adjustment strategy represents a strategy for replacing at least one default processing sub-path with a candidate sub-path for multiple default processing sub-paths in the default processing path.

[0086] According to an embodiment of the present disclosure, it is possible to determine what kind of adjustment to make to the default processing path based on a path adjustment strategy, so that the default processing path can be adjusted accordingly based on multiple candidate sub-paths to obtain multiple candidate processing paths.

[0087] According to an embodiment of the present disclosure, it is also possible to first determine, based on a path adjustment strategy, how many default processing sub-paths in the default processing path need to be replaced, and then call a corresponding number of candidate sub-paths from the acceleration library multiple times to adjust the default processing path.

[0088] For example, according to the path adjustment strategy, it can be determined that two default processing sub-paths in the default processing path need to be replaced. Then, two candidate sub-paths can be called from the acceleration library each time to replace the two default processing sub-paths in the default processing path to obtain the corresponding candidate processing paths.

[0089] In operation S240, the target model is used to process the input data in turn according to each candidate processing path to obtain a candidate running time corresponding to each candidate processing path.

[0090] According to an embodiment of the present disclosure, when multiple candidate processing paths are obtained, the target model can be used to process the input data in sequence according to the corresponding candidate processing paths to obtain corresponding candidate running times, so as to facilitate the subsequent determination of the shortest running time based on the multiple candidate running times. This operation is performed offline.

[0091] In operation S250, a target optimized processing path corresponding to the target model is determined from among the plurality of candidate processing paths based on the candidate runtime corresponding to each candidate processing path and the default runtime.

[0092] According to an embodiment of the present disclosure, since it is necessary to realize that the target model processes input data fastest according to the processing path, a target optimization processing path can be determined from multiple candidate processing paths based on the candidate runtime and the default runtime corresponding to each candidate processing path. The target optimization processing path can represent that the target model processes input data fastest according to the target optimization processing path, and this operation is also performed offline.

[0093] According to an embodiment of the present disclosure, since subsequent processing is to be performed offline, the default processing path can be recorded to generate a first processing path file, so as to facilitate the subsequent direct reading of the data recorded in the first processing path file to determine the target optimization processing path; in an offline situation, multiple candidate sub-paths are called from the acceleration library, and based on the path adjustment strategy, the default processing path is adjusted using multiple candidate sub-paths to obtain multiple candidate processing paths, and the input data is processed according to the candidate processing paths through the target model to obtain the corresponding candidate running times, so that based on the running time corresponding to each candidate processing path, the target optimization processing path corresponding to the shortest running time can be determined from the multiple candidate processing paths, so that when the target model processes the data according to the target optimization processing path, the speed of the model processing the input data can be increased, thereby achieving improved model performance.

[0094] Figure 4 A system diagram for implementing a method for determining a model processing path according to an embodiment of the present disclosure is schematically shown.

[0095] like Figure 4 As shown, input data (input) can be input into the target model 410. When the target model 410 processes the input data to obtain output data (output), a first processing path file can be generated based on the default processing path used by the target model 410 in the process, that is, the above-mentioned operation S210.

[0096] According to an embodiment of the present disclosure, the above operations S220 to S250 can be performed on PyTorch TunableOp. The constructed PyTorch TunableOp may include an offline module and an online line model. In the constructed PyTorchTunableOp, using the offline model, multiple candidate subpaths can be called from the acceleration library 420 in an offline state, that is, the above operation S220 is performed; the first processing path file can be given to the offline model, so that in an offline state, based on the data recorded in the first processing path file and multiple candidate subpaths, GEMM performance analysis is performed to determine the target optimization processing path that can achieve the best performance of the target model, that is, the above operations S220 to S250 are performed.

[0097] According to an embodiment of the present disclosure, PyTorch TunableOp does not use the default GEMM, that is, the default processing path, but searches for the best GEMM suitable for a specific environment, that is, the target optimized processing path, so that the model can be used according to the target optimized processing path on the GPU (Graphics Processing Unit) to achieve performance improvement.

[0098] According to an embodiment of the present disclosure, using an online model, the target optimization processing path can be used when the target model 410 is applied online.

[0099] According to an embodiment of the present disclosure, based on multiple candidate sub-paths, the default processing path is adjusted according to the path adjustment strategy to obtain multiple candidate processing paths, including: when the path adjustment strategy represents the replacement of n default processing sub-paths in the default processing path, the multiple candidate sub-paths are grouped into multiple candidate sub-path groups; for each candidate sub-path group, any n default processing sub-paths in the default processing path are replaced by the candidate sub-path group to obtain multiple candidate processing paths corresponding to each candidate sub-path group, and n is incremented until n takes the value of N to obtain multiple candidate processing paths.

[0100] Each candidate sub-path group includes n candidate sub-paths, the default processing path includes N default processing sub-paths, n is an integer greater than or equal to 1 and less than or equal to N, and N is a positive integer.

[0101] According to an embodiment of the present disclosure, based on a path adjustment strategy, it can be determined which default processing sub-paths in the default processing path need to be replaced. The path adjustment strategy may include a strategy of replacing 1 to N default processing sub-paths in the default processing path with candidate sub-paths in sequence.

[0102] For example, when the value of N is 3, the path adjustment strategy may include replacing one default processing sub-path in the default processing path, replacing two default processing sub-paths in the default processing path, and replacing three default processing sub-paths in the default processing path.

[0103] According to an embodiment of the present disclosure, when the path adjustment strategy represents the replacement of n default processing sub-paths in the default processing path, multiple candidate sub-paths called from the acceleration library can be grouped into multiple candidate sub-path groups, wherein the n candidate sub-paths in every two candidate sub-path groups may be completely different or may not be completely the same.

[0104] For example, when n is 2, the multiple candidate sub-paths called from the acceleration library may include A, B, C, D, E, and F, and the multiple candidate sub-path groups may include (A, B), (A, C), (C, D), and (E, F).

[0105] According to an embodiment of the present disclosure, for each candidate sub-path group, any n default processing sub-paths in the default processing path can be replaced by the candidate sub-path group, thereby obtaining multiple candidate processing paths corresponding to the candidate sub-path group.

[0106] For each candidate sub-path group, any n default processing sub-paths in the default processing path may be replaced multiple times.

[0107] For example, when n is 2 and N is 5, a candidate sub-path group includes two candidate sub-paths A and B, and the default processing path includes five default processing sub-paths C, D, E, F, and G in sequence. The order of the five default processing sub-paths is fixed. Then, the first and second default processing sub-paths in the default processing path can be replaced, the second and fifth default processing sub-paths in the default processing path can be replaced, and so on.

[0108] For each candidate sub-path group, the candidate sub-paths in the candidate sub-path group may replace any n default processing sub-paths in the default processing path in different orders.

[0109] For example, when replacing the first and second default processing sub-paths in the default processing path, the candidate sub-path A can replace the first default processing sub-path C in the default processing path, and the candidate sub-path B can replace the second default processing sub-path D in the default processing path; the candidate sub-path A can also replace the second default processing sub-path D in the default processing path, and the candidate sub-path B can replace the first default processing sub-path C in the default processing path.

[0110] Based on the above strategies, multiple candidate processing paths can be obtained.

[0111] According to an embodiment of the present disclosure, based on a path adjustment strategy, the default processing sub-path in the default processing path can be replaced with a candidate sub-path in different positions and orders to obtain multiple candidate processing paths. Thus, based on a large number of obtained candidate processing paths, omission of candidate processing paths can be avoided, thereby facilitating determination of the processing path that achieves the best model performance acceleration effect.

[0112] According to an embodiment of the present disclosure, based on the candidate running time and the default running time corresponding to each candidate processing path, a target optimization processing path corresponding to the target model is determined from multiple candidate processing paths, including repeatedly performing the following operations until the loop rounds meet the preset conditions: when the loop rounds do not meet the preset conditions, the candidate running time corresponding to the mth candidate processing path is compared with the target candidate running time to obtain a comparison result; based on the comparison result, a new target candidate running time is determined, and m is incremented; the new target candidate running time obtained when the loop rounds meet the preset conditions is determined as the target optimization running time, and the candidate processing path corresponding to the target optimization running time is determined as the target optimization processing path.

[0113] When the value of m is 1, the target candidate running time is the default running time.

[0114] According to an embodiment of the present disclosure, the preset condition may be determined by the number of candidate processing paths. For example, when the number of candidate processing paths is 10, the preset condition may be that 10 cycles are required. The multiple candidate processing paths may be randomly sorted first, and the order remains unchanged in subsequent processes.

[0115] According to an embodiment of the present disclosure, when the value of m is 1, that is, for the first candidate processing path after sorting, the candidate running time corresponding to the first candidate processing path can be compared with the default running time; when the value of m is greater than 1, that is, for the mth candidate processing path after sorting, the candidate running time corresponding to the mth candidate processing path can be compared with the target candidate running time.

[0116] According to an embodiment of the present disclosure, based on the comparison result, the shorter runtime can be determined as a new target candidate runtime for comparison in the next cycle.

[0117] According to an embodiment of the present disclosure, by comparing the candidate runtimes corresponding to each of a plurality of candidate processing paths with the target candidate runtime one by one, the shortest runtime can be determined, and the candidate processing path corresponding to the shortest runtime is determined as the target optimized processing path.

[0118] According to an embodiment of the present disclosure, by comparing the candidate runtimes of each of a plurality of candidate processing paths with the target candidate runtime one by one, after each comparison, the shorter runtime is determined as the new target candidate runtime and used in the comparison of the next round of cycles, so that the shortest runtime can be determined layer by layer, and ultimately the target optimization processing path with the best acceleration effect can be determined, thereby avoiding omission of candidate runtimes.

[0119] According to an embodiment of the present disclosure, a new target candidate runtime is determined based on the comparison result, including: when the comparison result indicates that the candidate runtime corresponding to the mth candidate processing path is less than the target candidate runtime, the candidate runtime corresponding to the mth candidate processing path is determined as the new target candidate runtime.

[0120] According to an embodiment of the present disclosure, based on the comparison result, when the candidate running time corresponding to the mth candidate processing path is less than the target candidate running time, the candidate running time corresponding to the mth candidate processing path can be determined as the new target candidate running time for use in the next round of the cycle.

[0121] For example, when the candidate running time corresponding to the mth candidate processing path is less than the target candidate running time, that is, a better processing path is found, the mth candidate processing path and the corresponding candidate running time can be recorded, such as: found better instance id=71. 0.00758317ms. Gemm_Rocblas_53909, which records the id=71 of the candidate processing path, the candidate running time 0.00758317ms, and the name of the candidate processing path Gemm_Rocblas_53909 in sequence.

[0122] According to an embodiment of the present disclosure, since the shorter candidate runtime corresponding to the mth candidate processing path is determined as the new target candidate runtime and used in the comparison process of the next round of cycles, a runtime shorter than the candidate runtime is selected to achieve the determination of the target optimization runtime with the shortest runtime through layer-by-layer comparison, thereby determining the corresponding target optimization processing path with the best acceleration effect.

[0123] According to an embodiment of the present disclosure, the above-mentioned method for determining the model processing path also includes: when the comparison result indicates that the candidate runtime corresponding to the mth candidate processing path is greater than or equal to the target candidate runtime, determining the target candidate runtime as the new target candidate runtime.

[0124] According to an embodiment of the present disclosure, based on the comparison result, when the candidate runtime corresponding to the mth candidate processing path is greater than or equal to the target candidate runtime, the target candidate runtime in the cycle round can be determined as the new target candidate runtime, that is, the target candidate runtime in the next cycle round, that is, the target candidate runtime in the next cycle round remains unchanged.

[0125] For example, when the candidate runtime corresponding to the mth candidate processing path is greater than or equal to the target candidate runtime, that is, no better processing path than the processing path corresponding to the target candidate runtime is found, it can be recorded as skip slow instance id=70, GemmTunableOp_float_TN(tn_200_100_20) Gemm_Rocblas_53908, that is, the processing path needs to be skipped.

[0126] According to an embodiment of the present disclosure, based on the comparison result, when the candidate runtime of the candidate processing path is greater than or equal to the target candidate runtime of the loop round, the target candidate runtime in the next loop round does not need to be changed and can be kept unchanged, so that the shorter runtime can be determined in subsequent loop rounds based on the smaller runtime. According to an embodiment of the present disclosure, the above-mentioned method for determining the model processing path also includes: for the target model, recording the target optimization processing path corresponding to the target model to obtain a second processing path file; for other types of models, accumulating and recording the target optimization processing paths corresponding to the default processing path of other types of models in the second processing path file.

[0127] According to an embodiment of the present disclosure, for the target optimization processing path corresponding to the target model determined by the above operations S210 to S250, the target optimization processing path can be recorded to obtain a second processing path file, so that the subsequent target model can directly read the second processing path file to determine the corresponding target optimization processing path.

[0128] For example, after determining the target optimization processing path, for each processing sub-path in the target optimization processing path, the following may be recorded in the second processing path file:

[0129] GemmTunableOp_float_TN,tn_200_100_20,Gemm_Rocblas_53909,0.00758317, which respectively represent the size of the operator corresponding to the processing sub-path 200_100_20, the name of the operator corresponding to the processing sub-path Gemm_Rocblas_53909, and the running time of the operator corresponding to the processing sub-path 0.00758317.

[0130] For example, the second processing path file may be named llm-tuned.csv.

[0131] According to an embodiment of the present disclosure, the second processing path file may include a default processing sub-path and a processing sub-path corresponding to the default processing sub-path in the target optimization processing path.

[0132] According to an embodiment of the present disclosure, for other types of models, the corresponding target optimization processing path can also be determined through the above operations S210 to S250, and the determined corresponding target optimization processing path can be accumulated and recorded in the second processing path file.

[0133] According to an embodiment of the present disclosure, there is a mapping relationship between the model type and the target optimization processing path in the second processing path file.

[0134] According to the embodiments of the present disclosure, since the target optimization processing paths of other types of models can be cumulatively recorded in the second processing path file, it is equivalent to adding automatic adaptation, allowing the operators automatically optimized in the second processing path file, that is, the target optimization processing paths can be continuously accumulated, so that more models and scenarios can be applied, so that the tuning can be reused and expanded, which increases the flexibility and executability of operator optimization.

[0135] According to an embodiment of the present disclosure, the method for determining the above-mentioned model processing path also includes: when using the target model for online application, reading the target optimization processing path corresponding to the target model from the second processing path file; in the target model, processing the data to be processed according to the target optimization processing path corresponding to the target model to achieve performance acceleration of the target model.

[0136] According to an embodiment of the present disclosure, based on the second processing path file, since there is a mapping relationship between the model type and the target optimization processing path in the second processing path file, when the target model is used for online application, the corresponding target optimization processing path can be read from the second processing path file based on the model type of the target model, so that the data to be processed can be processed in the target model according to the target optimization processing path.

[0137] According to an embodiment of the present disclosure, due to the second processing path file, when the target model is applied online again, the default GEMM operator is no longer used, that is, the default processing path is no longer used. Instead, the second processing path file is read first to read the corresponding optimized operator, that is, the corresponding target optimization processing path is read, thereby achieving the effect of accelerating the performance of the target model.

[0138] Based on the above, in this disclosure, PyTorch TunableOp can achieve moderate performance improvement on GPU without changing the existing code and only adding some additional optimization steps; for PyTorch TunableOp, offline tuning and automatic adaptation are supported, and the expansion of repeated use increases the flexibility and executability of operator optimization.

[0139] Figure 5 A block diagram schematically shows a device for determining a model processing path according to an embodiment of the present disclosure.

[0140] like Figure 5 As shown, the device 500 for determining the model processing path of this embodiment includes a generation module 510 , a calling module 520 , an adjustment module 530 , an acquisition module 540 and a determination module 550 .

[0141] The generation module 510 is used to generate a first processing path file based on the default processing path used in the process of processing the input data by the target model, wherein the first processing path file includes the default processing path and the default running time, and the default running time represents the running time of the target model processing the input data according to the default processing path. In one embodiment, the generation module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0142] The calling module 520 is used to call multiple candidate sub-paths from the acceleration library using the offline model. In one embodiment, the calling module 520 can be used to perform the operation S220 described above, which will not be described in detail here.

[0143] The adjustment module 530 is used to adjust the default processing path according to the path adjustment strategy based on the multiple candidate sub-paths to obtain multiple candidate processing paths, wherein the default processing path includes multiple default processing sub-paths; the path adjustment strategy represents a strategy for replacing at least one default processing sub-path with a candidate sub-path for multiple default processing sub-paths in the default processing path. In one embodiment, the adjustment module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0144] The acquisition module 540 is used to process the input data according to each candidate processing path in turn using the target model to obtain the candidate running time corresponding to each candidate processing path. In one embodiment, the acquisition module 540 can be used to perform the operation S240 described above, which will not be repeated here.

[0145] The determination module 550 is used to determine the target optimization processing path corresponding to the target model from multiple candidate processing paths based on the candidate runtime and the default runtime corresponding to each candidate processing path. In one embodiment, the determination module 550 can be used to perform the operation S250 described above, which will not be repeated here.

[0146] According to an embodiment of the present disclosure, the adjustment module 530 includes a grouping unit and a replacement unit.

[0147] A grouping unit is used to group multiple candidate sub-paths into multiple candidate sub-path groups when the path adjustment strategy represents the replacement of n default processing sub-paths in the default processing path, wherein each candidate sub-path group includes n candidate sub-paths, the default processing path includes N default processing sub-paths, n is an integer greater than or equal to 1 and less than or equal to N, and N is a positive integer.

[0148] A replacement unit is used to replace any n default processing sub-paths in the default processing path with the candidate sub-path group for each candidate sub-path group, to obtain multiple candidate processing paths corresponding to each candidate sub-path group, and to increase n until n takes the value of N, to obtain multiple candidate processing paths.

[0149] According to an embodiment of the present disclosure, the determination module 550 includes a comparison unit, a first determination unit, and a second determination unit.

[0150] A comparison unit is used to compare the candidate running time corresponding to the mth candidate processing path with the target candidate running time when the loop round does not meet the preset conditions, and obtain a comparison result, wherein when the value of m is 1, the target candidate running time is the default running time.

[0151] The first determining unit is used to determine a new target candidate running time according to the comparison result, and increment m.

[0152] The second determination unit is used to determine the new target candidate running time obtained when the loop round meets the preset conditions as the target optimization running time, and determine the candidate processing path corresponding to the target optimization running time as the target optimization processing path.

[0153] According to an embodiment of the present disclosure, the comparison unit includes a first determination subunit.

[0154] The first determining subunit is configured to determine the candidate running time corresponding to the mth candidate processing path as a new target candidate running time when the comparison result indicates that the candidate running time corresponding to the mth candidate processing path is less than the target candidate running time.

[0155] According to an embodiment of the present disclosure, the comparison unit further includes a second determination subunit.

[0156] The second determining subunit is configured to determine the target candidate running time as a new target candidate running time when the comparison result indicates that the candidate running time corresponding to the mth candidate processing path is greater than or equal to the target candidate running time.

[0157] According to an embodiment of the present disclosure, the device 500 for determining a model processing path further includes a first recording module and a second recording module.

[0158] The first recording module is used to record the target optimization processing path corresponding to the target model for the target model to obtain a second processing path file.

[0159] The second recording module is used to accumulate and record the target optimized processing paths corresponding to the default processing paths of other types of models in the second processing path file.

[0160] According to an embodiment of the present disclosure, the device 500 for determining a model processing path further includes a reading module and a processing module.

[0161] The reading module is used to read the target optimization processing path corresponding to the target model from the second processing path file when the target model is used for online application.

[0162] The processing module is used to process the data to be processed in the target model according to the target optimization processing path corresponding to the target model to achieve performance acceleration of the target model.

[0163] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0164] For example, any multiple of the generation module 510, the call module 520, the adjustment module 530, the acquisition module 540, and the determination module 550 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the generation module 510, the call module 520, the adjustment module 530, the acquisition module 540, and the determination module 550 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the generating module 510 , the calling module 520 , the adjusting module 530 , the obtaining module 540 and the determining module 550 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

[0165] It should be noted that the part of the device for determining the model processing path in the embodiment of the present disclosure corresponds to the part of the method for determining the model processing path in the embodiment of the present disclosure. The description of the part of the device for determining the model processing path specifically refers to the part of the method for determining the model processing path, which will not be repeated here.

[0166] Figure 6 A block diagram of an electronic device suitable for implementing the method for determining the model processing path described above according to an embodiment of the present disclosure is schematically shown. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0167] like Figure 6 As shown, the electronic device 600 according to an embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0168] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0169] According to an embodiment of the present disclosure, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0170] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0171] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0172] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0173] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 602 and / or the RAM 603 described above and / or one or more memories other than the ROM 602 and the RAM 603 .

[0174] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method for determining the model processing path provided by the embodiment of the present disclosure.

[0175] When the computer program is executed by the processor 601, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0176] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0177] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0178] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0179] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for determining a model processing path, comprising: Generate a first processing path file based on a default processing path used in a process of processing input data by a target model, wherein the first processing path file includes a default processing path and a default running time, and the default running time represents the running time for the target model to process the input data according to the default processing path; Using the offline model, multiple candidate subpaths are called from the acceleration library; According to the multiple candidate sub-paths, the default processing path is adjusted according to the path adjustment strategy to obtain multiple candidate processing paths, wherein the default processing path includes multiple default processing sub-paths; the path adjustment strategy represents a strategy for replacing at least one of the default processing sub-paths with the candidate sub-path for the multiple default processing sub-paths in the default processing path; Processing the input data in turn according to each of the candidate processing paths using the target model to obtain a candidate running time corresponding to each of the candidate processing paths; Based on the candidate running time corresponding to each of the candidate processing paths and the default running time, a target optimized processing path corresponding to the target model is determined from the plurality of candidate processing paths.

2. The method according to claim 1, wherein: The step of adjusting the default processing path according to the multiple candidate sub-paths and the path adjustment strategy to obtain multiple candidate processing paths includes: In a case where the path adjustment strategy represents replacing n default processing sub-paths in the default processing path, the multiple candidate sub-paths are grouped into multiple candidate sub-path groups, wherein each candidate sub-path group includes n candidate sub-paths, the default processing path includes N default processing sub-paths, n is an integer greater than or equal to 1 and less than or equal to N, and N is a positive integer; and For each of the candidate sub-path groups, any n default processing sub-paths in the default processing path are replaced by the candidate sub-path groups to obtain multiple candidate processing paths corresponding to each of the candidate sub-path groups, and n is incremented until n takes the value of N to obtain the multiple candidate processing paths.

3. The method according to claim 1, wherein: The determining, based on the candidate running time corresponding to each of the candidate processing paths and the default running time, a target optimization processing path corresponding to the target model from the multiple candidate processing paths comprises repeatedly performing the following operations until a cycle round satisfies a preset condition: If the cycle does not meet the preset condition, Compare the candidate running time corresponding to the mth candidate processing path with the target candidate running time to obtain a comparison result, wherein when the value of m is 1, the target candidate running time is the default running time; According to the comparison result, determine a new target candidate running time, and increment m; The new target candidate running time obtained when the loop round meets the preset condition is determined as the target optimization running time, and the candidate processing path corresponding to the target optimization running time is determined as the target optimization processing path.

4. The method according to claim 3, wherein: Determining a new target candidate running time according to the comparison result includes: When the comparison result indicates that the candidate running time corresponding to the mth candidate processing path is less than the target candidate running time, the candidate running time corresponding to the mth candidate processing path is determined as the new target candidate running time.

5. The method according to claim 4, further comprising: When the comparison result indicates that the candidate runtime corresponding to the mth candidate processing path is greater than or equal to the target candidate runtime, the target candidate runtime is determined as the new target candidate runtime.

6. The method according to claim 1, further comprising: For the target model, recording the target optimization processing path corresponding to the target model to obtain a second processing path file; For other types of models, the target optimized processing paths corresponding to the default processing paths of the other types of models are cumulatively recorded in the second processing path file.

7. The method according to claim 6, further comprising: In the case of using the target model for online application, reading a target optimization processing path corresponding to the target model from the second processing path file; as well as In the target model, the data to be processed is processed according to a target optimization processing path corresponding to the target model to achieve performance acceleration of the target model.

8. A device for determining a model processing path, comprising: A generating module, configured to generate a first processing path file based on a default processing path used in a process of processing input data by a target model, wherein the first processing path file includes a default processing path and a default running time, and the default running time represents a running time for the target model to process the input data according to the default processing path; A calling module is used to call multiple candidate sub-paths from the acceleration library using an offline model; an adjustment module, configured to adjust the default processing path according to the plurality of candidate sub-paths and a path adjustment strategy to obtain a plurality of candidate processing paths, wherein the default processing path includes a plurality of default processing sub-paths; the path adjustment strategy represents a strategy for replacing at least one of the default processing sub-paths in the default processing path with the candidate sub-path; An acquisition module, configured to use the target model to process the input data in turn according to each of the candidate processing paths, to obtain a candidate running time corresponding to each of the candidate processing paths; A determination module is used to determine a target optimization processing path corresponding to the target model from the multiple candidate processing paths based on the candidate running time corresponding to each of the candidate processing paths and the default running time.

9. An electronic device, comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 7.

11. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images