Scheduling search method and device, cloud service providing method, electronic device and computer readable storage medium

Abstract

The present application discloses a scheduling search method and device, a cloud service provision method, an electronic device, and a computer-readable storage medium. The method includes: obtaining a first execution time of a general computing acceleration library executing a target operator; performing a search for the target operator in a first search space to obtain a first schedule; obtaining a second execution time of the target operator executed by the first schedule; and determining the final schedule of the target operator according to the first execution time and the second execution time. The embodiment of the present application can comprehensively consider the execution efficiency of the target operator to determine the scheduling scheme of the target operator, thereby ensuring the optimization of the operator scheduling while saving computing power.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a scheduling search method and device, a cloud service providing method, an electronic device, and a computer-readable storage medium. Background Art

[0002] In machine learning, in order to improve the computing efficiency of operators, in the existing technology, general hardware manufacturers will provide a general computing acceleration library, that is, a computing solution suitable for their hardware specifications, which can give full play to their hardware advantages to optimize the operator's computing solution and optimize the operator manually.

[0003] However, due to the rapid development of AI models and hardware, the general acceleration library provided by hardware manufacturers cannot effectively cope with the efficiency improvement of various operators that are widely used in practice. In the prior art, operators can also be manually optimized by experienced developers, but this requires a lot of manpower for manual optimization. Therefore, only efficient optimization for general operators (common types or common sizes) can be achieved, and it is difficult to achieve large-scale implementation. Therefore, for a large number of non-general operators, there is still a lot of room for efficiency improvement. Summary of the invention

[0004] The embodiments of the present application provide a scheduling search method and device, a cloud service providing method, an electronic device, and a computer-readable storage medium to solve the defect of wasting computing power in performing scheduling searches in the prior art.

[0005] To achieve the above object, the present application provides a scheduling search method, including:

[0006] Obtaining the first execution time of the general computing acceleration library executing the target operator;

[0007] In a first search space, performing a search for the target operator to obtain a first schedule;

[0008] Obtaining a second execution time of executing the target operator by the first scheduling;

[0009] A final schedule of the target operator is determined according to the first execution time and the second execution time.

[0010] The present application also provides a cloud service providing method, including:

[0011] Determine a corresponding cloud computing task according to a received cloud service request from a user;

[0012] Parsing the cloud computing task to obtain at least one target operator for executing the cloud computing task;

[0013] Obtaining the first execution time of the general computing acceleration library executing the target operator;

[0014] In a first search space, performing a search for the target operator to obtain a first schedule;

[0015] Obtaining a second execution time of executing the target operator by the first scheduling;

[0016] Determining a final schedule of the target operator according to the first execution time and the second execution time;

[0017] The cloud computing task is executed using the target operator based on the final schedule.

[0018] The present application also provides a scheduling search device, including:

[0019] A first acquisition module, used to acquire a first execution time of a target operator executed by a general computing acceleration library;

[0020] A first search module, configured to perform a search for the target operator in a first search space to obtain a first schedule;

[0021] A second acquisition module, used to acquire a second execution time of executing the target operator by the first scheduling;

[0022] A determination module is used to determine a final schedule of the target operator according to the first execution time and the second execution time.

[0023] The present application also provides an electronic device, including:

[0024] Memory, used to store programs;

[0025] A processor is used to run the program stored in the memory, and when the program is running, it executes the scheduling search method provided in the embodiment of the present application, or executes the cloud service providing method provided in the embodiment of the present application.

[0026] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program executable by a processor is stored, wherein when the program is executed by the processor, the scheduling search method provided in the embodiment of the present application is implemented, or the cloud service provision method provided in the embodiment of the present application is implemented.

[0027] The scheduling search method and device, cloud service provision method, electronic device, and computer-readable storage medium provided by the embodiments of the present application can compare the performance, such as execution time, achieved by the scheduling obtained by the two schemes by taking the execution time of the general computing acceleration library as a benchmark and performing a tentative search in a coarse-grained small-range search space. Therefore, the ratio of the execution time achieved by the two schemes can be used to determine whether the target operator is worth continuing to use the scheme of the search space to find a better schedule, and if it is worth it, that is, the ratio is greater than the threshold, then an iterative method is used to continue to find a better schedule, and if it is not worth it, that is, the ratio is less than the threshold, then the search process of the search space is terminated, and the scheme of the general acceleration library is directly used as the schedule. In this way, the execution efficiency of the target operator can be comprehensively considered to determine the scheduling scheme of the target operator, thereby ensuring the optimization of the operator scheduling while saving computing power.

[0028] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0030] Figure 1a A schematic diagram of an application scenario of the scheduling search method provided in an embodiment of the present application;

[0031] Figure 1b A schematic diagram of a scenario of a scheduling search system applied to AI computing provided in an embodiment of the present application;

[0032] Figure 2 A flowchart of an embodiment of the scheduling search method provided by the present application;

[0033] Figure 3 A flowchart of another embodiment of the scheduling search method provided by the present application;

[0034] Figure 4 A schematic diagram of the structure of an embodiment of a scheduling search device provided by the present application;

[0035] Figure 5 A schematic diagram of the structure of an electronic device embodiment provided in this application. DETAILED DESCRIPTION

[0036] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0037] Embodiment 1

[0038] The solution provided in the embodiments of the present application can be applied to any computing system with the code generation capability of operators, such as a computing server installed with an automatic code generation module, etc. Figure 1a A schematic diagram of an application scenario of the scheduling search method provided in an embodiment of the present application, Figure 1a The scenario shown is only one example of a scenario to which the technical solution of the present application can be applied.

[0039] In machine learning, various computational operations used to perform computational tasks are called operators. An operator can have multiple computational implementation methods when implementing computations. Each computational implementation method can be called a schedule. Although different schedules have the same execution results, they take different execution times. Therefore, the shorter the execution time of the adopted schedule, the higher the computational efficiency of the operator.

[0040] When hardware is used to perform a computing task of machine learning, each operator of the computing task is actually executed separately. Specifically, when the hardware performs a computing task of machine learning, it first determines the corresponding code based on the operators in the computing task, which is also called the coding process of the operator. Therefore, for an operator, there can be multiple implementation plans, that is, multiple codes, when performing specific calculations on the hardware. Each plan can be called a schedule. Although different schedules have the same execution results, there are differences in the time spent on execution. Therefore, the shorter the execution time of the adopted schedule, the higher the computing efficiency of the operator.

[0041] In order to improve the computing efficiency of operators, in the prior art, hardware manufacturers generally provide general computing acceleration libraries, that is, computing solutions suitable for their hardware specifications, or provide solutions for automatically generating codes according to operators, so as to optimize the computing efficiency of operators by giving full play to their hardware advantages. However, since it optimizes the computing efficiency of operators based on their own specific hardware structure and is usually constructed by experts from hardware manufacturers based on experience to optimize the operator optimization method for better scheduling, such general computing acceleration libraries are usually only effective for common operators, that is, they can improve their computing efficiency. However, due to the rapid development of AI models and the development of hardware, a large number of various operators have emerged, and such general acceleration libraries provided by hardware manufacturers require time and effort to develop, so they cannot well cope with the efficiency improvement of various operators used in practice. In addition, in the prior art, operators can also be manually optimized by experienced developers, but this requires a lot of manpower and is difficult to implement on a large scale. Therefore, for a large number of non-general operators, there is still a lot of room for efficiency improvement. In the prior art, there has also been a process of using artificial intelligence technology to automatically or semi-automatically generate operator codes. This process is not actually a process of replacing traditional experts to carry out special development, but a process of searching for a better scheduling for operators. However, the search space used in this process is an important factor that directly affects the performance upper limit of the optimization process. For example, expanding the search space means more opportunities to find a better scheduling solution, but it will pay a greater price, such as consuming a large amount of computing resources and correspondingly requiring more search time, while narrowing the search space can reduce the price paid, that is, reducing the consumed computing resources and correspondingly reducing the search time, but this may result in the scheduling solution finally found not meeting the requirements, and may even be worse than the solution of the general acceleration library. Therefore, in the current face of various systems that need to process tens of thousands of operators, it is necessary to balance the search for a better scheduling of operators and the overall throughput of the system.

[0042] Therefore, in an embodiment of the present application, when a new operator is found to be executed, a general acceleration library can be used to execute the new operator, i.e., the target operator, and its execution time can be measured, so as to use the execution time as a benchmark. According to the applicant's discovery, in the field of the present invention, usually only a small part of the operators are using the scheduling scheme determined by the search, i.e., the coding scheme can achieve an efficiency exceeding that of using a general computing acceleration library, and a small part of the operators use the scheduling scheme determined by the search to achieve performance or efficiency close to that achieved by using a general acceleration library, and the remaining majority of the operators cannot achieve performance or efficiency beyond that achieved by the general computing acceleration library even if the scheduling scheme determined by searching using a larger search space. Therefore, in the present application, the execution time of the operator using the general acceleration library as the scheduling execution operator is used as a benchmark to search for the above-mentioned small part of the operators, and the remaining majority of the operators end the search in advance to save computing power. Therefore, in an embodiment of the present application, a general acceleration library is first used to execute the target operator and thereby determine the benchmark execution time. After determining the benchmark execution time, in an embodiment of the present application, a scheduling search is first performed in a smaller scheduling search space, i.e., a coarse-grained search space, and the target operator is executed with the better scheduling determined in the scheduling search space and the execution time of the operator executed with the scheduling is measured, and then it can be compared with the benchmark execution time. For example, the ratio of the execution time of the operator executed with the scheduling in the search space to the benchmark execution time can be used as a basis for judgment. For example, when the ratio is greater than a preset threshold, for example, when the ratio is greater than a preset threshold and less than 1, it can generally be indicated that the performance (i.e., execution time) of the better scheduling implementation found in the search space is not much different from the performance achieved by executing the operator using a general acceleration library, or when the ratio is greater than 1, it generally indicates that the performance (i.e., execution time) of the better scheduling implementation found in the search space has surpassed the performance achieved by executing the operator using a general acceleration library, and therefore, such a result can indicate that the scheme for searching using the search space has the opportunity to surpass the scheme for executing the operator using the general computing acceleration library. In other words, this means that for the target operator, a search scheme should be used to determine its scheduling. For example, at least the current coarse-grained search space can be used, or even the search space can be expanded or a finer-grained search space can be used to perform the scheduling search. Therefore, through the scheduling search method of the present application, it is possible to compare the performance, such as execution time, of the scheduling obtained by the two schemes by taking the execution time of the general computing acceleration library as a benchmark and performing a exploratory search in a coarse-grained small-range search space.Therefore, the ratio of the execution time achieved by the two solutions can be used to determine whether the target operator is worth continuing to use the search space solution to find a better schedule, and if it is worth it, that is, the ratio is greater than the threshold, then an iterative method is used to continue to find a better schedule, and if it is not worth it, that is, the ratio is less than the threshold, then the search process of the search space is terminated, and the general acceleration library solution is directly used as the schedule. In this way, it is possible to comprehensively consider the advantages and disadvantages of the general computing acceleration library and the search space search solution in terms of the execution efficiency of the target operator to select which solution to use as the scheduling solution for the target operator, thereby saving unnecessary waste of computing power and ensuring the optimization of the operator scheduling.

[0043] Figure 1b A schematic diagram of a scenario of a scheduling search system for AI computing provided in an embodiment of the present application. Figure 1b As shown, in an embodiment of the present application, when it is necessary to use an AI model to perform a computing task, such as a computing task of face recognition, the face recognition task can be split into multiple operators (operator 1, operator 2, ..., operator n). For example, operator 1 can be a partition operator for calculating a feature area, and operator 2 can be a feature value operator for calculating the feature value of a feature part of a face, and so on. In an embodiment of the present application, these operators are sent to a scheduling search system to obtain the final schedule corresponding to each operator. Taking operator 1 for calculating the feature area as an example, the scheduling search system first calculates the benchmark time of operator 1 through a general computing acceleration library provided by the hardware used for the current AI model calculation. Then, based on an additional search space developed by others or obtained on the Internet, a search can be performed for operator 1 to obtain a first schedule and the corresponding execution time. That is, in an embodiment of the present application, when the operator to be executed is currently determined, Figure 1b In the scenario shown in , operator 1 has been determined to be a partition operator for calculating feature areas. Therefore, different implementation codes, i.e., scheduling, can be found through the hardware used and other search spaces obtained based on the Internet, and the execution time can be calculated respectively. When the ratio of the benchmark time using the general computing acceleration library to the execution time of the scheduling searched in the search space is less than the preset threshold, the scheduling in the general computing acceleration library is used as the final scheduling of operator 1, that is, the code of the partition operator provided in the general acceleration library is used as the implementation code for calculating feature partitions in the calculation task of the face recognition. When the ratio of the benchmark time to the execution time is greater than or equal to the preset threshold, it means that the search space using a non-general acceleration library can achieve greater efficiency improvement. Therefore, the preset threshold and the search range of the search space can be adjusted to repeatedly perform the iterative operation of the search scheduling until the ratio of the benchmark time to the execution time is less than the preset threshold, thereby determining the final implementation code, i.e., scheduling.

[0044] In an embodiment of the present application, a general computing acceleration library is used to calculate the execution time of each operator in the computing task of the AI ​​model as a reference time. After determining the reference time, a scheduling search is first performed in a smaller search space, that is, a coarse-grained search space, and the operator is executed with a better schedule determined in the search space and the execution time of the operator executed with the schedule is measured, which can then be compared with the reference time to screen out a smaller number of operators that meet the preset conditions. Then, by increasing the preset threshold and expanding the search range of the search space, an iterative search is performed on a smaller number of operators, while most of the remaining operators end the search in advance. Therefore, the execution efficiency of the operator can be comprehensively considered to determine the scheduling scheme of the operator, thereby ensuring the optimization of the operator scheduling while saving computing power, and thus greatly improving the computing efficiency of the AI ​​model.

[0045] In addition, in an embodiment of the present application, when a user uses a cloud service to which the scheduling search method of an embodiment of the present application is applied, the cloud computing task to be performed by the cloud server to which the scheduling search method of the present application is applied can be determined based on the user's request for the use of the cloud service, such as search or database services. After determining the cloud computing task for the user's cloud service use request, the scheduling search method of an embodiment of the present application can be used to parse the corresponding computing task, for example, it can be parsed into multiple operators, so as to determine a suitable computing acceleration library for the user based on the result of performing the search scheduling using the operators. Therefore, when a user uses a cloud service to which the scheduling search method of an embodiment of the present application is applied, a more efficient computing response can be provided for the user's service request.

[0046] The above embodiments are illustrations of the technical principles and exemplary application frameworks of the embodiments of the present application. The specific technical solutions of the embodiments of the present application are further described in detail below through multiple embodiments.

[0047] Embodiment 2

[0048] Figure 2 This is a flowchart of an embodiment of the scheduling search method provided by the present application. The execution subject of the method can be any computing system with the code generation capability of the operator, such as a computing server with an automatic code generation module installed, etc., or a device or chip integrated on these devices. Figure 2 As shown, the scheduling search method includes the following steps:

[0049] S201, obtaining a first execution time of a general computing acceleration library executing a target operator.

[0050] In an embodiment of the present application, when an intelligent computing task is received, it is parsed into multiple operators, each of which corresponds to an operation task. In step S201, a general computing acceleration library can be used for each operator to perform the corresponding operation and measure its execution time as a benchmark execution time. In the technical field of this invention, usually only a small part of the operators are using the scheduling scheme determined by search, that is, the coding scheme can achieve an efficiency exceeding the efficiency of using a general computing acceleration library, and a small part of the operators use the scheduling scheme determined by search to achieve performance or efficiency close to the efficiency achieved by using a general acceleration library, and most of the remaining operators cannot achieve performance or efficiency beyond the general computing acceleration library even if the scheduling scheme determined by searching using a larger search space. Therefore, in the present application, the execution time of the general acceleration library as the scheduling execution operator is used as a benchmark to select the above-mentioned small part of the operators suitable for search processing for search, and most of the remaining operators end the search in advance to save computing power.

[0051] S202: In a first search space, perform a search for a target operator to obtain a first schedule.

[0052] After the first execution time serving as the benchmark execution time is determined in step S201, in an embodiment of the present application, a scheduling search can be first performed in a smaller scheduling search space, i.e., a coarse-grained search space, in step S202 to determine a better scheduling in the coarse-grained search space as the first scheduling. In other words, the first scheduling is a scheduling with a shorter execution time for executing the target operator in the first search space. Preferably, the scheduling with the shortest execution time can be selected as the first scheduling.

[0053] S203: Obtain a second execution time of the first scheduled execution target operator.

[0054] After determining a better schedule in the first search space in step S202, the target operator can be executed with the better schedule determined in the scheduling search space in step S203, thereby measuring the execution time of the operator executed with the schedule, so as to facilitate subsequent comparison with the first execution time determined in step S201 as the benchmark execution time.

[0055] S204: Determine a final schedule of the target operator according to the first execution time and the second execution time.

[0056] In step S204, the execution time of the better scheduled execution target operator in the first search space measured and determined in step S203, that is, the second execution time, can be compared with the first execution time determined in step S201. For example, the ratio of the second execution time of the scheduled execution operator in the first search space to the first execution time of the operator executed using the general computing acceleration library can be used as a basis for judgment. For example, when the ratio is less than a preset threshold, it means that the execution performance of the scheduled execution operator determined in the first search space is lower than the performance achieved using the general computing acceleration library. Therefore, in step S304, the scheduling in the general computing acceleration library can be directly used as the scheduling of the target operator, and the search process for the target operator can be terminated accordingly.

[0057] Therefore, the scheduling search scheme provided by the embodiment of the present application can be performed by using the execution time of the general computing acceleration library as a benchmark and performing a tentative search in a coarse-grained small-range search space, thereby comparing the performance that can be achieved by the scheduling obtained by the two schemes, such as execution time. Therefore, the ratio of the execution time achieved by the two schemes can be used to determine whether the target operator is worth continuing to use the scheme of the search space to find a better schedule, and if it is worth it, that is, the ratio is greater than the threshold, then an iterative method is used to continue to find a better schedule, and if it is not worth it, that is, the ratio is less than the threshold, then the search process of the search space is terminated, and the scheme of the general acceleration library is directly used as the schedule. In this way, the execution efficiency of the target operator can be comprehensively considered to determine the scheduling scheme of the target operator, thereby ensuring the optimization of the operator scheduling while saving computing power.

[0058] Embodiment 3

[0059] Figure 3 This is a flowchart of another embodiment of the scheduling search method provided by the present application. The execution subject of the method can be any computing system with the code generation capability of the operator, such as a computing server equipped with an automatic code generation module, etc., or a device or chip integrated on these devices. Figure 3 As shown, the scheduling search method includes the following steps:

[0060] S301, obtaining a first execution time of a general computing acceleration library executing a target operator.

[0061] In an embodiment of the present application, when an intelligent computing task is received, it is parsed into multiple operators, each of which corresponds to an operation task. In step S201, a general computing acceleration library can be used for each operator to perform the corresponding operation and measure its execution time as a benchmark execution time. In the technical field of this invention, usually only a small part of the operators are using the scheduling scheme determined by search, that is, the coding scheme can achieve an efficiency exceeding the efficiency of using a general computing acceleration library, and a small part of the operators use the scheduling scheme determined by search to achieve performance or efficiency close to the efficiency achieved by using a general acceleration library, and the remaining majority of the operators cannot achieve performance or efficiency beyond the general computing acceleration library even if the scheduling scheme determined by searching using a larger search space. Therefore, in the present application, the execution time of the general acceleration library as the scheduling execution operator is used as a benchmark to select the above-mentioned small part of the operators suitable for search processing for search, and the remaining majority of the operators end the search in advance to save computing power.

[0062] S302: In a first search space, perform a search for a target operator to obtain a first schedule.

[0063] After the first execution time serving as the benchmark execution time is determined in step S301, in an embodiment of the present application, a scheduling search can be first performed in a smaller scheduling search space, i.e., a coarse-grained search space, in step S302 to determine a better scheduling in the coarse-grained search space as the first scheduling. In other words, the first scheduling is the scheduling with the shortest execution time for executing the target operator in the first search space.

[0064] S303: Obtain a second execution time of the first scheduled execution target operator.

[0065] After determining a better schedule in the first search space in step S302, the target operator can be executed with the better schedule determined in the scheduling search space in step S203, thereby measuring the execution time of the operator executed with the schedule, so as to facilitate subsequent comparison with the first execution time determined in step S201 as the benchmark execution time.

[0066] S304: When the ratio of the first execution time to the second execution time is less than a preset threshold, the scheduling in the general computing acceleration library is used as the final scheduling of the target operator.

[0067] In step S304, the execution time of the better scheduled execution target operator in the first search space measured and determined in step S303, that is, the second execution time, can be compared with the first execution time determined in step S301. For example, the ratio of the second execution time of the scheduled execution operator in the first search space to the first execution time of the operator executed using the general computing acceleration library can be used as a basis for judgment. For example, when the ratio is less than a preset threshold, it means that the execution performance of the scheduled execution operator determined in the first search space is lower than the performance achieved using the general computing acceleration library. Therefore, in step S304, the scheduling in the general computing acceleration library can be directly used as the scheduling of the target operator, and the search process for the target operator can be terminated accordingly.

[0068] In addition, for example, when the ratio is greater than or equal to a preset threshold, for example, when the ratio is greater than the preset threshold and less than 1, it can generally be indicated that the performance (i.e., execution time) of the better scheduling implementation found in the search space is not much different from the performance achieved by executing the operator using a general acceleration library, or when the ratio is greater than 1, it generally indicates that the performance (i.e., execution time) of the better scheduling implementation found in the search space has surpassed the performance achieved by executing the operator using a general acceleration library, and therefore, such a result can indicate that the scheme using the search space for searching has the opportunity to surpass the scheme for executing the operator using a general computing acceleration library. In other words, this means that for the target operator, a search scheme should be used to determine its scheduling. In this case, an iterative operation can be performed on the target operator to search for a better schedule. For example, the following operations can be performed:

[0069] S305: Increase the preset threshold and expand the search range of the first search space.

[0070] As described above, for example, when the execution time of the target operator executed by the first scheduling determined by performing a preliminary search in step S302 is more than 70% of the execution time of the target operator executed using the general computing acceleration library, that is, the ratio in step S304 is greater than 0.7, this indicates that the search scheme may be able to obtain a scheduling whose performance exceeds that of the general computing acceleration library. Therefore, the search range of the scheduling search space when performing the preliminary search in step S302 can be expanded in step S305 and the preset threshold can be adjusted accordingly from 0.7 to 1, for example, to try to obtain a scheduling that can achieve better performance.

[0071] Therefore, according to the embodiment of the present application, step S306 can be executed with the search range expanded in step S305, that is, in the expanded first search space, the target operator is searched to re-acquire the first schedule, in which case the first schedule is the schedule with the shortest execution time for executing the target operator in the expanded first search space. And accordingly, step S307 is executed to obtain the second execution time of the newly obtained first schedule to execute the target operator, and step S308 is executed according to the measured second execution time, that is, when the ratio of the first execution time to the second execution time is less than the preset threshold set in step S305, the first schedule obtained in step S306 executed in the iterative operation is determined as the final schedule of the target operator, and the iterative operation is terminated.

[0072] In addition, when the ratio of the first execution time to the second execution time is greater than or equal to a preset threshold, the iterative operation is repeatedly performed.

[0073] Therefore, through the above-mentioned iterative operation, the search space can be gradually expanded based on the coarse-grained search space used in the initial search in step S302, or the scheduling search can be iteratively performed using a gradually finer-grained search space until a better schedule is found.

[0074] In addition, in the embodiment of the present application, in order to prevent the above-mentioned iterative search operation from being difficult to converge, it may further include:

[0075] S309: When the ratio of the first execution time to the second execution time is greater than or equal to a preset threshold, determine whether the search range of the first search space reaches a preset search range.

[0076] S310, when the search range of the first search space reaches the preset search range, the iterative operation ends, and the scheduling in the general computing acceleration library is used as the final scheduling of the target operator.

[0077] In step S309, when it is determined that the execution efficiency of the better schedule in the search space determined in step S306 is still lower than that of the general computing acceleration library, it can be further determined whether the search space is large enough, or whether sufficient searches have been performed. Thus, when it is determined that the preset search range has been reached, if the execution efficiency of the searched schedule is still lower than that of the general computing acceleration library, the search can be terminated in time to save computing power.

[0078] In an embodiment of the present application, the size of the preset threshold value can also be adjusted according to the instruction input by the user, and the appropriate threshold value can be determined as soon as possible by interacting with the user, so as to facilitate the rapid convergence of the search iteration process. For example, the information of the determined search space and the corresponding execution time can be output to the user before the iteration starts or during the iteration process, so that the user can adjust the preset threshold value according to its needs or usage, for example, by issuing instructions for the output, so as to accelerate convergence or further expand the search. Thus, the system can adjust the threshold value according to such adjustment instructions for the current search space and the corresponding execution time input by the user, or continue to perform the iteration operation according to the threshold value input by the user.

[0079] In addition, in an embodiment of the present application, the first search space may also be determined based on the AI ​​model currently used, that is, based on the application scenario that the user is currently processing, so that it is possible to use different search spaces as the first search space to perform iterative operations for different AI models or different application scenarios. For example, the system may obtain the classification identifier of the current AI model, and select the corresponding search space as the first search space according to the classification identifier, or the system may output the current search space information to the user, so that the user may issue an instruction to specify the search space that he wishes to use.

[0080] In addition, in the present application, in addition to the above-mentioned iterative operation, the cost model can also be used to find suitable scheduling for operators that are expected to achieve scheduling beyond the general computing acceleration library through search. In the technical field of the present application, the cost model is a spatial search model generated by training based on historical training data, so it can also achieve the early convergence of the above-mentioned iterative operation in effect. Therefore, in the present application, the cost model can also be combined to further improve the efficiency of finding a better schedule.

[0081] For example, in an embodiment of the present application, the cost model can be constructed based on the cost function with the scheduling and corresponding computational efficiency of each operator as parameters, where the cost function can represent the corresponding relationship between scheduling and computational efficiency, i.e., coefficient. Thus, such a model can be obtained by training the model using the historical data of the corresponding relationship between the scheduling of the operator determined each time and the computational efficiency actually executed. Therefore, in an embodiment of the present application, it can be used as an alternative to the scheme of the present application to input the determined scheduling into the cost model for each operator to find the corresponding relationship between the computational efficiency and scheduling when the cost function takes the minimum value, so that the corresponding relationship can be used to limit the step size when searching for the next scheduling, thereby also being able to achieve rapid convergence with the exploration in the above-mentioned multiple exploration spaces. In addition, the cost model can also be used as a supplementary scheme for the scheme of exploration in multiple exploration spaces in the embodiment of the present application, that is, before each exploration is performed, the cost model is first used to predict the computational efficiency of each current scheduling, and the scheduling scheme that can converge the fastest is selected to perform the current scheduling exploration, thereby being able to speed up the convergence of the scheduling search scheme of the present application.

[0082] Therefore, the scheduling search scheme provided by the embodiment of the present application can be performed by using the execution time of the general computing acceleration library as a benchmark and performing a tentative search in a coarse-grained small-range search space, thereby comparing the performance that can be achieved by the scheduling obtained by the two schemes, such as execution time. Therefore, the ratio of the execution time achieved by the two schemes can be used to determine whether the target operator is worth continuing to use the scheme of the search space to find a better schedule, and if it is worth it, that is, the ratio is greater than the threshold, then an iterative method is used to continue to find a better schedule, and if it is not worth it, that is, the ratio is less than the threshold, then the search process of the search space is terminated, and the scheme of the general acceleration library is directly used as the schedule. In this way, the execution efficiency of the target operator can be comprehensively considered to determine the scheduling scheme of the target operator, thereby ensuring the optimization of the operator scheduling while saving computing power.

[0083] Embodiment 4

[0084] Figure 4 The schematic diagram of the structure of the scheduling search device embodiment provided in the present application can be used to perform the following steps: Figure 2 and Figure 3 The method steps shown. Figure 4 As shown, the scheduling search device may include: a first acquisition module 41, a first search module 42, a second acquisition module 43 and a determination module 44.

[0085] The first acquisition module 41 may be used to acquire a first execution time of a general computing acceleration library executing a target operator.

[0086] In an embodiment of the present application, when an intelligent computing task is received, it is parsed into multiple operators, each of which corresponds to an operation task. A general computing acceleration library can be used for each operator to perform the corresponding operation and its execution time can be measured by the first acquisition module 41 as a benchmark execution time. In the technical field of this invention, usually only a small part of the operators are using the scheduling scheme determined by the search, that is, the coding scheme can achieve an efficiency exceeding the efficiency of using the general computing acceleration library, and a small part of the operators use the scheduling scheme determined by the search to achieve performance or efficiency close to the efficiency achieved by using the general acceleration library, and most of the remaining operators cannot achieve performance or efficiency beyond the general computing acceleration library even if the scheduling scheme is determined by searching using a larger search space. Therefore, in the present application, the execution time of the general acceleration library as the scheduling execution operator is used as a benchmark to select the above-mentioned small part of the operators suitable for search processing for search, and most of the remaining operators end the search in advance to save computing power.

[0087] The first search module 42 may be configured to perform a search for a target operator in a first search space to obtain a first schedule.

[0088] When the first acquisition module 41 determines the first execution time as the benchmark execution time, in an embodiment of the present application, the first search module 42 can first perform a scheduling search in a smaller scheduling search space, that is, a coarse-grained search space, to determine a better scheduling in the coarse-grained search space as the first scheduling. In other words, the first scheduling is a scheduling with a shorter execution time for executing the target operator in the first search space. Preferably, the scheduling with the shortest execution time can be selected as the first scheduling.

[0089] The second acquisition module 43 is used to acquire the second execution time of the first scheduling execution of the target operator.

[0090] When the first search module 42 determines a better schedule in the first search space, the second acquisition module 43 can be used to execute the target operator with the better schedule determined in the scheduling search space, thereby measuring the execution time of the operator executed with the schedule, so as to facilitate subsequent comparison with the first execution time determined by the first acquisition module 41 as the benchmark execution time.

[0091] The determination module 44 may be configured to determine a final schedule of the target operator according to the first execution time and the second execution time.

[0092] The determination module 44 can compare the execution time of the better scheduling execution target operator in the first search space measured and determined by the second acquisition module 43, that is, the second execution time with the first execution time determined by the first acquisition module 41. For example, the ratio of the second execution time of the scheduling execution operator in the first search space to the first execution time of the operator executed using the general computing acceleration library can be used as a basis for judgment. For example, when the ratio is less than a preset threshold, it means that the execution performance of the scheduling execution operator determined in the first search space is lower than the performance achieved using the general computing acceleration library. Therefore, it can be determined that the module 44 can directly use the scheduling in the general computing acceleration library as the scheduling of the target operator, and accordingly end the search processing for the target operator.

[0093] In addition, for example, when the ratio is greater than a preset threshold, for example, when the ratio is greater than a preset threshold and less than 1, it can generally be indicated that the performance (i.e., execution time) of the better scheduling implementation found in the search space is not much different from the performance achieved by executing the operator using a general acceleration library, or when the ratio is greater than 1, it generally indicates that the performance (i.e., execution time) of the better scheduling implementation found in the search space has surpassed the performance achieved by executing the operator using a general acceleration library, and therefore, such a result can indicate that the scheme using the search space for searching has the opportunity to surpass the scheme for executing the operator using a general computing acceleration library. In other words, this means that for the target operator, a search scheme should be used to determine its scheduling. In this case, the scheduling search device of the present application may further include an iterative control module 45, which may perform iterative operations on the target operator to search for a better schedule. For example, the following operations may be performed:

[0094] Increase the preset threshold and expand the search range of the first search space.

[0095] As described above, for example, when the first search module 42 performs a preliminary search to determine that the execution time of the first scheduling execution target operator is more than 70% of the execution time of the target operator executed using the general computing acceleration library, that is, the ratio determined by the determination module 44 is greater than 0.7, this indicates that the use of the search scheme may be able to obtain a scheduling whose performance exceeds that of the general computing acceleration library. Therefore, the search range of the scheduling search space when the first search module 42 performs the preliminary search can be expanded and the preset threshold can be adjusted accordingly from 0.7, for example, to 1.1, in an attempt to obtain a scheduling that can achieve better performance.

[0096] Therefore, according to the embodiment of the present application, the first search module 42 can repeatedly perform the search operation with the expanded search range, that is, in the expanded first search space, search for the target operator to re-acquire the first schedule, in which case the first schedule is the schedule with the shortest execution time for executing the target operator in the expanded first search space. And accordingly, the iterative control module 45 can control the second acquisition module 43 to obtain the second execution time of the newly obtained first schedule to execute the target operator, and control the determination module 44 according to the measured second execution time, that is, when the ratio of the first execution time to the second execution time is less than the newly set preset threshold, the first schedule obtained by controlling the first search module 42 in the iterative operation is determined as the final schedule of the target operator, and the iterative operation is terminated.

[0097] In addition, the iteration control module 45 may repeatedly perform the iteration operation when the ratio of the first execution time to the second execution time is greater than or equal to a preset threshold.

[0098] Therefore, through the above-mentioned iterative operation, the search space can be gradually expanded based on the coarse-grained search space used by the first search module 42 in the initial search, or the scheduling search can be iteratively performed using a gradually finer-grained search space until a better schedule is found.

[0099] In addition, in the embodiment of the present application, in order to prevent the above-mentioned iterative search operation from being difficult to converge, the iterative operation of the iterative control module 45 may further include:

[0100] When the ratio of the first execution time to the second execution time is greater than or equal to a preset threshold, it is determined whether the search range of the first search space reaches a preset search range.

[0101] When the search range of the first search space reaches the preset search range, the iterative operation ends, and the scheduling in the general computing acceleration library is used as the final scheduling of the target operator.

[0102] Therefore, when the determination module 44 determines that the execution efficiency of the better schedule in the search space is still not as good as that of the general computing acceleration library, the iteration control module 45 can further determine whether the search space is large enough, or whether enough searches have been performed, so that when it is determined that the preset search range has been reached, if the execution efficiency of the searched schedule is still not as good as that of the general computing acceleration library, the search can be terminated in time to save computing power.

[0103] In addition, in the present application, in addition to the above-mentioned iterative operation, the cost model can also be used to find suitable scheduling for operators that are expected to achieve scheduling beyond the general computing acceleration library through search. In the technical field of the present application, the cost model is a spatial search model generated by training based on historical training data, so it can also achieve the early convergence of the above-mentioned iterative operation in effect. Therefore, in the present application, the cost model can also be combined to further improve the efficiency of finding a better schedule.

[0104] Therefore, the scheduling search scheme provided by the embodiment of the present application can be performed by using the execution time of the general computing acceleration library as a benchmark and performing a tentative search in a coarse-grained small-range search space to compare the performance that can be achieved by the scheduling obtained by the two schemes, such as execution time. Therefore, the ratio of the execution time achieved by the two schemes can be used to determine whether the target operator is worth continuing to use the search space scheme to find a better schedule, and if it is worth it, that is, the ratio is greater than the threshold, then an iterative method is used to continue to find a better schedule, and if it is not worth it, that is, the ratio is less than the threshold, then the search process of the search space is terminated, and the scheme of the general acceleration library is directly used as the schedule. In this way, it is possible to comprehensively consider the advantages and disadvantages of the search schemes of the general computing acceleration library and the search space in terms of the execution efficiency of the target operator to select which scheme to use as the scheduling scheme of the target operator, thereby saving unnecessary waste of computing power and ensuring the optimization of the scheduling of the operator.

[0105] Embodiment 5

[0106] The internal functions and structure of the scheduling search device are described above. The device can be implemented as an electronic device. Figure 5 This is a schematic diagram of the structure of an electronic device embodiment provided by this application. Figure 5 As shown, the electronic device includes a memory 51 and a processor 52 .

[0107] The memory 51 is used to store programs. In addition to the above programs, the memory 51 can also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0108] The memory 51 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0109] The processor 52 is not limited to a central processing unit (CPU), but may also be a processing chip such as a graphics processing unit (GPU), a field programmable gate array (FPGA), an embedded neural network processor (NPU) or an artificial intelligence (AI) chip. The processor 52 is coupled to the memory 51 and executes a program stored in the memory 51. When the program is executed, the scheduling search method of the above-mentioned embodiments 2 and 3 is executed.

[0110] Further, if Figure 5 As shown, the electronic device may also include: a communication component 53, a power component 54, an audio component 55, a display 56 and other components. Figure 5 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 5 Components shown.

[0111] The communication component 53 is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 3G, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 53 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 53 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0112] The power supply component 54 provides power to various components of the electronic device. The power supply component 54 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0113] The audio component 55 is configured to output and / or input audio signals. For example, the audio component 55 includes a microphone (MIC), and when the electronic device is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 51 or sent via the communication component 53. In some embodiments, the audio component 55 also includes a speaker for outputting audio signals.

[0114] The display 56 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0115] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A scheduling search method, comprising: Obtaining the first execution time of the general computing acceleration library executing the target operator; In a first search space, performing a search for the target operator to obtain a first schedule; The first schedule is a schedule with the shortest execution time for executing the target operator in the first search space; Obtaining a second execution time of executing the target operator by the first scheduling; Determining a final schedule of the target operator according to the first execution time and the second execution time; The determining the final schedule of the target operator according to the first execution time and the second execution time includes: when the ratio of the first execution time to the second execution time is less than a preset threshold, taking the schedule in the general computing acceleration library as the final schedule of the target operator; The final scheduling of the target operator is determined based on the first execution time and the second execution time, and further includes: when the ratio of the first execution time to the second execution time is greater than or equal to the preset threshold, the preset threshold and the search range of the first search space are adjusted to repeatedly perform the iterative operation until the ratio of the first execution time to the second execution time is less than the preset threshold; wherein the adjustment of the preset threshold and the search range of the first search space includes increasing the preset threshold and expanding the search range of the first search space.

2. The scheduling search method according to claim 1, wherein: The iterative operation includes: Increasing the preset threshold and expanding the search range of the first search space; In the first search space, searching for the target operator to reacquire a first schedule; Obtaining a second execution time of executing the target operator by the first scheduling; When the ratio of the first execution time to the second execution time is less than a preset threshold, determining the schedule as the final schedule of the target operator and ending the iterative operation; When the ratio of the first execution time to the second execution time is greater than or equal to a preset threshold, the iterative operation is repeatedly performed.

3. The scheduling search method according to claim 2, wherein: The iterative operation also includes: outputting the first search space and the second execution time corresponding to the first search space to a user; According to the user's instruction for the output, the preset threshold is adjusted, and the iterative operation is continued.

4. The scheduling search method according to claim 1, wherein: The iterative operation also includes: When the ratio of the first execution time to the second execution time is greater than or equal to a preset threshold, determining whether the search range of the first search space reaches a preset search range; When the search range of the first search space reaches the preset search range, the iterative operation is terminated, and the scheduling in the general computing acceleration library is used as the final scheduling of the target operator.

5. The scheduling search method according to any one of claims 1 to 4, wherein: The scheduling search method also includes: When the ratio of the first execution time to the second execution time is greater than or equal to the preset threshold, a cost model is used to determine a final schedule for executing the target operator in a second search space.

6. A cloud service providing method, comprising: Determine a corresponding cloud computing task according to a received cloud service request from a user; Parsing the cloud computing task to obtain at least one target operator for executing the cloud computing task; Obtaining the first execution time of the general computing acceleration library executing the target operator; In a first search space, performing a search for the target operator to obtain a first schedule; The first schedule is a schedule with the shortest execution time for executing the target operator in the first search space; Obtaining a second execution time of executing the target operator by the first scheduling; Determining a final schedule of the target operator according to the first execution time and the second execution time; The determining the final schedule of the target operator according to the first execution time and the second execution time includes: when the ratio of the first execution time to the second execution time is less than a preset threshold, taking the schedule in the general computing acceleration library as the final schedule of the target operator; The determining the final scheduling of the target operator according to the first execution time and the second execution time further includes: when the ratio of the first execution time to the second execution time is greater than or equal to the preset threshold, adjusting the preset threshold and the search range of the first search space to repeatedly perform the iterative operation until the ratio of the first execution time to the second execution time is less than the preset threshold; wherein the adjusting the preset threshold and the search range of the first search space includes increasing the preset threshold and expanding the search range of the first search space; The cloud computing task is executed using the target operator based on the final schedule.

7. A scheduling search device, comprising: A first acquisition module, used to acquire a first execution time of a target operator executed by a general computing acceleration library; A first search module, configured to perform a search for the target operator in a first search space to obtain a first schedule; A second acquisition module, used to acquire a second execution time of executing the target operator by the first scheduling; A determination module, configured to determine a final schedule of the target operator according to the first execution time and the second execution time; The determination module is specifically configured to, when a ratio of the first execution time to the second execution time is less than a preset threshold, use the schedule in the general computing acceleration library as the final schedule of the target operator; An iterative control module, configured to, when the ratio of the first execution time to the second execution time is greater than or equal to the preset threshold, perform the following iterative operations: increase the preset threshold and expand the search range of the first search space; control the first search module to search for the target operator in the first search space to reacquire a first schedule, wherein the first schedule is the schedule with the shortest execution time for executing the target operator in the first search space; control the second acquisition module to acquire the second execution time of the target operator executed by the first schedule; control the determination module to determine the schedule as the final schedule of the target operator when the ratio of the first execution time to the second execution time is less than the preset threshold, and terminate the iterative operation; When the ratio of the first execution time to the second execution time is greater than or equal to a preset threshold, the iterative operation is repeatedly performed.

8. The scheduling search device according to claim 7, wherein: The iterative operation also includes: When the ratio of the first execution time to the second execution time is greater than or equal to a preset threshold, determining whether the search range of the first search space reaches a preset search range; When the search range of the first search space reaches the preset search range, the iterative operation is terminated, and the scheduling in the general computing acceleration library is used as the final scheduling of the target operator.

9. The scheduling search device according to any one of claims 7 to 8, wherein: The determination module is further used for: When the ratio of the first execution time to the second execution time is greater than or equal to the preset threshold, a cost model is used to determine a final schedule for executing the target operator in a second search space.

10. An electronic device, comprising: Memory, used to store programs; A processor, used to run the program stored in the memory, wherein when the program is run, the scheduling search method as described in any one of claims 1 to 5 is executed, or the cloud service providing method as described in claim 6 is executed.

11. A computer-readable storage medium having stored thereon a computer program executable by a processor, wherein: When the program is executed by a processor, it implements the scheduling search method as described in any one of claims 1 to 5, or implements the cloud service providing method as described in claim 6.

Images