Server, client, system, method, device and medium for optimizing strategy

Through the policy server generation and allocation of optimization strategies suitable for computing devices, the problem of time-consuming and inefficient optimization process of artificial intelligence algorithm model in the existing technology is solved, and efficient and flexible optimization strategy application is realized, reducing the optimization burden and energy consumption of the client .

CN115225497BActive Publication Date: 2024-11-26XINQIAO (BEIJING) SEMICONDUCTOR CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210827417.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-13
Publication Date
2024-11-26
Estimated Expiration
2042-07-13

AI Technical Summary

Technical Problem

The existing technology lacks flexibility and adaptability when optimizing artificial intelligence algorithm models. Static and dynamic optimization strategies are mainly executed on a stand-alone machine, resulting in time-consuming and inefficient optimization processes, and optimization strategies are not necessarily optimal for specific hardware performance.

Method used

Provide a policy server to solve the current optimal model optimization strategy suitable for the device by generating and allocating the model optimization strategy to be verified, based on the attributes and performance data of the computing device, and send it to the client, which is received by the client. Run and feedback performance data after the appropriate optimization strategy is reached to determine the optimal strategy.

Benefits of technology

It realizes offline solution of optimization strategies on the policy server, reduces the optimization burden on the client, improves optimization efficiency and flexibility, reduces energy consumption, and avoids the need for upgrading model optimization strategies for each client.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115225497B_ABST
    Figure CN115225497B_ABST
Patent Text Reader

Abstract

A server, client, system, method, electronic device and non-transitory storage medium for optimizing strategies are provided. The strategy server includes: a generating device configured to make one or more changes to one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified, and assign them to at least one computing device for operation; a solving device configured to solve the current optimal model optimization strategy suitable for the at least one computing device in the one or more model optimization strategies to be verified based on the properties of at least one computing device running the one or more model optimization strategies to be verified and the performance data obtained by the at least one computing device running the one or more optimization models to be verified.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a server, a client, a system, a method, an electronic device, and a non-temporary storage medium for solving and applying optimization strategies of artificial intelligence models. Background Art

[0002] Artificial intelligence algorithm models are usually composed of multiple operators. Usually, artificial intelligence algorithm models are run using, for example, a graphics processing unit (GPU). In order to speed up the GPU running artificial intelligence algorithm models, it is necessary to optimize the artificial intelligence algorithm models. Currently, this optimization includes static strategy optimization and dynamic optimization to improve the operating efficiency and performance of artificial intelligence algorithm models. Static strategy optimization is the optimization of artificial intelligence algorithm models before they are loaded onto the GPU processor for operation. Dynamic optimization is performed during the operation of the artificial intelligence algorithm model after it is loaded.

[0003] There is still a need to optimize the artificial intelligence algorithm model in order to improve the running efficiency and performance of the artificial intelligence algorithm model on specific hardware, such as a specific GPU. Summary of the invention

[0004] According to one aspect of the present application, a policy server for solving optimization strategies of artificial intelligence models is provided, comprising: a generating device, configured to make one or more changes to one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified, and assign them to at least one computing device for operation; a solving device, configured to solve the current optimal model optimization strategy suitable for the at least one computing device among the one or more model optimization strategies to be verified based on the properties of at least one computing device running the one or more model optimization strategies to be verified and the performance data obtained by the at least one computing device running the one or more optimization models to be verified.

[0005] In one embodiment, the policy server further comprises: a receiving device configured to receive a request for querying the optimal model optimization policy and attributes of the client from the client.

[0006] In one embodiment, when the policy server has solved the current optimal model optimization strategy suitable for the client, the solving device of the policy server is configured to send the current optimal model optimization strategy suitable for the client to the at least one computing device.

[0007] In one embodiment, when the policy server has not yet solved the current optimal model optimization strategy suitable for the client: the generating device is configured to allocate one or more to-be-verified strategies of the one or more to-be-verified model optimization strategies to the client based on the request so that the client runs one or more to-be-verified optimization models optimized by the one or more to-be-verified strategies, and the client is configured to send performance data obtained by running the one or more to-be-verified optimization models to the policy server.

[0008] In one embodiment, the solving device is configured to determine whether there is a current optimal model optimization strategy suitable for the client among the one or more strategies to be verified based on the client's attributes sent by the client and the obtained performance data.

[0009] In one embodiment, the solving device is configured to partially solve the current optimal model optimization strategy upon receiving partial performance data, and cache the current optimal model optimization strategy of the computing device, the current time associated therewith, and the partial performance data in a memory.

[0010] In one embodiment, the generating device is configured to assign different optimization strategies for the model to be verified to different clients so that the solving device determines whether the properties of the client are suitable for running the operator changes in the assigned optimization strategy for the model to be verified.

[0011] In one embodiment, if it is determined that the attributes of the client are suitable for running the operator change in the assigned model optimization strategy to be verified, the generating device is configured to assign the model optimization strategy to be verified with the operator change to the client with the attributes.

[0012] In one embodiment, the performance data includes at least one of running time, clock count, memory usage, data transmission volume, number of errors, and temperature, and the attributes of the computing device include: the model of the computing device's chip, the type of chip, the amount of chip memory of the computing device, the supported bandwidth, the convolution operation capacity provided by the chip, and at least one of the composition of the computing kernel of the computing device, wherein the policy server is configured to determine the model optimization strategy to be verified that optimizes the performance data of at least one computing device as the current optimal model optimization strategy suitable for at least one computing device with the attributes.

[0013] In one embodiment, the generating device is configured to perform one or more changes of operator type conversion, operator replacement, operator splitting, and operator merging on one or more operators in the original artificial intelligence model to generate the one or more model optimization strategies to be verified.

[0014] In one embodiment, the generating device is configured to eliminate unsuitable model optimization strategies to be verified from the generated one or more model optimization strategies to be verified based on the properties of the at least one computing device.

[0015] In one embodiment, the solution device is configured to: perform the solution step regularly; or perform the solution step in response to receiving new performance data; or a combination of the two.

[0016] According to another aspect of the present application, a method for solving an optimization strategy for an artificial intelligence model is provided, comprising: making one or more changes to one or more operators in an original artificial intelligence model to generate one or more model optimization strategies to be verified, and assigning them to at least one computing device for execution; solving a current optimal model optimization strategy suitable for the at least one computing device among the one or more model optimization strategies to be verified based on the properties of at least one computing device running the one or more model optimization strategies to be verified and the performance data obtained by the at least one computing device running the one or more optimization models to be verified.

[0017] According to another aspect of the present application, there is provided a client for applying an optimization strategy for an artificial intelligence model, comprising: a sending device, configured to send a request for querying an optimal model optimization strategy and the attributes of the client to a policy server; an application device, configured to receive from the policy server a model optimization strategy suitable for the client, which is sent to the client by the policy server based on the attributes of the client, and apply the model optimization strategy.

[0018] In one embodiment, the client further includes a receiving device configured to receive one or more policies to be verified allocated from the policy server, and the application device of the client runs one or more optimization models to be verified optimized by the one or more policies to be verified, the sending device is configured to send performance data obtained by running the one or more optimization models to be verified to the policy server, the policy server is configured to determine whether there is a current optimal model optimization strategy suitable for the client in the one or more policies to be verified based on the attributes of the client sent by the client and the obtained performance data, or the client is configured not to send the performance data obtained by running the one or more policies to be verified to the policy server if it is set to be confidential.

[0019] In one embodiment, the client also includes a determination device, which is configured to determine whether the most recent model optimization strategy previously allocated to the client is cached, when the policy server has not yet solved the current optimal model optimization strategy suitable for the client, and does not have one or more model optimization strategies to be verified or does not assign a model optimization strategy to be verified to the client: if it is determined to be cached, the application device is configured to apply the cached most recent model optimization strategy, and if it is determined to be not cached, the sending device is configured to send the previously cached performance data to the policy server.

[0020] According to another aspect of the present application, a method for applying an optimization strategy for an artificial intelligence model is provided, comprising: sending a request for querying the optimal model optimization strategy and the attributes of the client to a policy server; receiving from the policy server a model optimization strategy suitable for the client sent to the client by the policy server based on the attributes of the client, and applying the model optimization strategy.

[0021] According to another aspect of the present application, a system for solving and applying optimization strategies for artificial intelligence models is provided, including: a strategy server, configured to: make one or more changes to one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified, and assign them to at least one computing device for execution; solve the current optimal model optimization strategy suitable for the at least one computing device in the one or more model optimization strategies to be verified based on the properties of at least one computing device running the one or more model optimization strategies to be verified and the performance data obtained by the at least one computing device running the one or more optimization models to be verified; a client, configured to send a request for querying the optimal model optimization strategy and the properties of the client to the policy server, wherein the policy server is configured to send the model optimization strategy suitable for the client to the client according to the properties of the client.

[0022] In one embodiment, when the policy server has solved the current optimal model optimization strategy suitable for the client, the policy server is configured to send the current optimal model optimization strategy suitable for the client to the client.

[0023] In one embodiment, when the policy server has not yet solved the current optimal model optimization strategy suitable for the client: the policy server is configured to assign one or more of the one or more model optimization strategies to be verified to the client based on the request so that the client runs one or more optimization models to be verified optimized by the one or more strategies to be verified, the client is configured to send performance data obtained by running the one or more optimization models to be verified to the policy server, and the policy server is configured to determine whether there is a current optimal model optimization strategy suitable for the client among the one or more strategies to be verified based on the attributes of the client sent by the client and the obtained performance data.

[0024] In one embodiment, the client is configured to determine whether the most recent model optimization strategy previously allocated to the client is cached, and if it is determined to be cached, apply the cached most recent model optimization strategy, and if it is determined to be not cached, send the previously cached performance data to the policy server.

[0025] In one embodiment, the client is configured not to send the performance data obtained by running the one or more policies to be verified to the policy server when confidentiality is set.

[0026] In one embodiment, the policy server is configured to partially solve the current optimal model optimization strategy upon receiving partial performance data, and cache the current optimal model optimization strategy of the computing device, the current time associated therewith, and the partial performance data in a memory.

[0027] In one embodiment, the policy server is configured to assign different to-be-verified model optimization policies to different clients in order to determine whether the properties of the client are suitable for running the operator changes in the assigned to-be-verified model optimization policies.

[0028] In one embodiment, if it is determined that the attributes of the client are suitable for running the operator change in the assigned model optimization strategy to be verified, the strategy server is configured to assign the model optimization strategy to be verified with the operator change to the client with the attributes.

[0029] In one embodiment, the performance data includes at least one of running time, clock count, memory usage, data transmission volume, number of errors, and temperature, and the attributes of the computing device include: the model of the chip of the computing device, the type of chip, the amount of chip memory of the computing device, the supported bandwidth, the convolution operation capacity provided by the chip, and at least one of the composition of the computing kernel of the computing device, wherein the policy server is configured to determine the model optimization strategy to be verified that optimizes the performance data of the at least one computing device as the current optimal model optimization strategy suitable for the at least one computing device having the attributes.

[0030] In one embodiment, the policy server is configured to perform one or more changes including operator type conversion, operator replacement, operator splitting, and operator merging on one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified.

[0031] In one embodiment, the policy server is configured to: based on the properties of the at least one computing device, eliminate unsuitable model optimization strategies to be verified from the generated one or more model optimization strategies to be verified.

[0032] In one embodiment, the policy server is configured to: perform the solution step regularly; or perform the solution step in response to receiving new performance data; or a combination of the two.

[0033] According to another aspect of the present application, a method for solving and applying optimization strategies of artificial intelligence models is provided, comprising: a policy server: making one or more changes to one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified, and assigning them to at least one computing device for running; solving the current optimal model optimization strategy suitable for the at least one computing device in the one or more model optimization strategies to be verified based on the properties of at least one computing device running the one or more model optimization strategies to be verified and the performance data obtained by the at least one computing device running the one or more optimization models to be verified; and a client: sending a request for querying the optimal model optimization strategy and the properties of the client to the policy server, wherein the policy server is configured to send the model optimization strategy suitable for the client to the client according to the properties of the client.

[0034] According to one aspect of the present application, an electronic device is provided, including: a memory for storing instructions; a processor for reading the instructions in the memory and executing the methods described in various embodiments of the present application.

[0035] According to one aspect of the present application, a non-transitory storage medium is provided, on which instructions are stored, wherein when the instructions are read by a processor, the processor executes the method described in various embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 A schematic block diagram of a system for solving and applying an optimization strategy for an artificial intelligence model according to an embodiment of the present application is shown.

[0038] Figure 2A The functions that the policy server and the client are responsible for respectively according to the implementation manner of the present application are shown.

[0039] Figure 2B A schematic flow chart showing the functional steps of a system for solving and applying an optimization strategy for an artificial intelligence model according to an embodiment of the present application.

[0040] Figure 3 A schematic flow chart showing the steps of requesting and obtaining an optimization strategy at a client according to an embodiment of the present application is shown.

[0041] Figure 4 A schematic diagram of an application scenario of a system for solving and applying an optimization strategy for an artificial intelligence model according to an embodiment of the present application is shown.

[0042] Figure 5 A schematic flow chart showing the steps of an offline optimization strategy solving service and an optimization strategy request service interacting with a client, mainly performed at a strategy server according to an embodiment of the present application.

[0043] Figure 6 A schematic flow chart of a method for solving and applying an optimization strategy for an artificial intelligence model according to an embodiment of the present application is shown.

[0044] Figure 7 A block diagram of a policy server for solving an optimization strategy for an artificial intelligence model according to an embodiment of the present application is shown.

[0045] Figure 8 A schematic flow chart of a method for solving an optimization strategy for an artificial intelligence model according to an embodiment of the present application is shown.

[0046] Fig. 9 A block diagram of a client for applying an optimization strategy of an artificial intelligence model according to an embodiment of the present application is shown.

[0047] Fig.10 A schematic flow chart of a method for applying an optimization strategy of an artificial intelligence model according to an embodiment of the present application is shown.

[0048] Fig.11 A block diagram of an exemplary electronic device suitable for implementing the embodiments of the present application is shown.

[0049] Fig.12 A schematic diagram of a non-transitory computer-readable storage medium according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0050] Reference will now be made in detail to specific embodiments of the present application, examples of which are illustrated in the accompanying drawings. Although the present application will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit the present application to the described embodiments. On the contrary, it is intended to cover changes, modifications and equivalents included in the spirit and scope of the present application as defined by the appended claims. It should be noted that the method steps described herein can all be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of the two.

[0051] Artificial intelligence algorithm models are usually neural network models, such as image reasoning models, speech reasoning models, etc. Input an image with an animal, and pass it through the neural network model, and directly output a label to indicate what the object in the image is, such as a dog or a cat. Neural network models usually include convolutional neural network models.

[0052] A neural network model can be composed of operators. Operators refer to various operations performed by each layer in a neural network model. For example, the convolution operation performed by the convolution layer of a neural network model on the input data of the neural network model is a convolution operator. A neural network model can include many types of operators, such as convolution operators, fully connected operators, pooling operators, transposition operators, Sobel operators, reshape operators, and so on.

[0053] In the actual application of the neural network model, computing hardware can be used to implement (calculate) each operator in the neural network model. For example, for the above-mentioned convolution operator, a graphics processing unit (GPU), a central processing unit (CPU), a machine learning unit (machine learning unit) or a field programmable gate array (FPGA) and other hardware can be used to implement the operator in the neural network model.

[0054] The computational complexity of each operator may be large or small, but in general, the computational complexity of the artificial intelligence algorithm model is huge and time-consuming. The calculation of the artificial intelligence algorithm model needs to be optimized in order to save time, computational complexity and computational cost.

[0055] Moreover, different computing hardware and different hardware attributes have different processing capabilities, supported bandwidth and other performance, as well as storage space. The performance of the same neural network model may also be different when running on different computing hardware.

[0056] The steps of static strategy optimization and dynamic optimization of the artificial intelligence algorithm model mentioned above are all executed on a single machine, which means that the optimization actions must be executed according to the task execution schedule of the single machine (such as computing hardware). Therefore, the entire optimization process may be time-consuming and inefficient. Moreover, this kind of optimization is a fixed optimization program and strategy based on the client's experience, and this optimization strategy is not necessarily optimal for the hardware performance of the computing hardware running the artificial intelligence algorithm model. Therefore, the optimization of the existing technology also lacks flexibility and adaptability. When upgrading the optimization strategy, special upgrade software is required to upgrade the optimization strategy of each computing hardware separately.

[0057] Figure 1 A schematic block diagram of a system 100 for solving and applying an optimization strategy for an artificial intelligence model according to an embodiment of the present application is shown.

[0058] like Figure 1As shown, a system 100 for solving and applying optimization strategies for artificial intelligence models includes: a strategy server 101, configured to: make one or more changes to one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified, and assign them to at least one computing device for execution; solve the current optimal model optimization strategy suitable for at least one computing device in one or more model optimization strategies to be verified based on the properties of at least one computing device running one or more model optimization strategies to be verified and the performance data obtained by the at least one computing device running the one or more optimization models to be verified; a client 102, configured to send a request for querying the optimal model optimization strategy and the properties of the client to the strategy server, wherein the strategy server is configured to send a model optimization strategy suitable for the client to the client based on the properties of the client.

[0059] In this way, the optimization strategy is liberated to the policy server 101 for offline execution, and the optimization strategy solution in the model compilation process and the optimized model compilation and operation at the client can be separated and decoupled to a certain extent, simplifying the implementation of the two parts, centralizing the calculation at the policy server, and using large computing power to solve the optimization strategy offline, reducing the constraints on the performance requirements of the client when the client runs alone, and reducing the model optimization burden at the client. The runtime calculation of the traditional model dynamic optimization can be converted into a query initiated by the client and a static optimization at the server, and the query and static optimization are energy-friendly and efficient, so this technology can achieve low energy consumption and high-efficiency strategy optimization. Moreover, the policy server 101 can receive and improve performance data and solve the optimal strategy from the client 102 offline step by step, and have more time and probability to solve the optimal strategy without affecting the workload of the client 102. In addition, the policy server 101 can combine the specific attributes of the computing device and the performance data obtained by the computing device running the model to solve the current optimal model optimization strategy suitable for the computing device, so as to send the optimal model optimization strategy that best suits the attributes of the client to the client that issues the request later. Moreover, since the policy server solves and upgrades the current optimal model optimization strategy by itself, there is no need to upgrade the model optimization strategy for each client, which reduces a lot of workload. The policy server can also coordinate the relationship between the attributes of different clients and different model optimization strategies, understand which attributes are suitable for running which optimization strategy (operator change), and thus coordinate the allocation of optimization strategies to be verified to different clients and the generation of which optimization strategies to be verified.

[0060] Figure 2A The functions that the policy server and the client are responsible for respectively according to the implementation manner of the present application are shown.

[0061] The policy server is responsible for updating data including policies and performance data, solving the optimal policy offline, and storing the optimal solution. The client does not solve the policy, but obtains the optimization policy from the policy server and uploads the performance data to the policy server. Specifically, the client obtains the optimization result of the policy server and applies (calculates) the optimized model, caches the obtained optimization policy locally, and then obtains the performance data after applying (calculates) the optimized model and uploads it to the policy server.

[0062] It can be seen that by liberating the solution of the optimization strategy to the policy server 101 for offline execution, the solution of the optimization strategy in the model compilation process and the compilation and operation of the optimized model at the client can be separated and decoupled to a certain extent, simplifying the implementation of the two parts, centralizing the calculation at the policy server, and using large computing power to solve the optimization strategy offline, thereby reducing the constraints on the performance requirements of the solution when the client runs alone, and reducing the model optimization burden at the client.

[0063] Figure 2B A schematic flow chart showing the functional steps of a system 100 for solving and applying an optimization strategy for an artificial intelligence model according to an embodiment of the present application is shown.

[0064] like Figure 2B As shown, the policy server on the server side makes one or more changes to one or more operators in the original artificial intelligence model in 201 to generate one or more model optimization strategies to be verified, and distributes them to at least one computing device for operation in 202. The client can now obtain the assigned optimization strategy in 205, and apply (operate) the artificial intelligence model optimized by the optimization strategy. The client can collect performance data of the optimized artificial intelligence model in 206. The client then transmits the collected performance data of the optimized artificial intelligence model back to the policy server in 207. After receiving the performance data, the policy server can update the current performance data related to the hardware information (attributes) and the model information (indicating which model has what kind of operator changes) to the latest received performance data based on the hardware information (attributes) and model information of the client. In 203, the policy server can solve the current optimal model optimization strategy suitable for the client among one or more model optimization strategies to be verified based on the properties and performance data of the client running one or more model optimization strategies to be verified, and can update the optimal model optimization strategy to the current optimal model optimization strategy, and can update the set of strategies to be verified (for example, indicating that one of the strategies to be verified has completed verification, deleting the verified strategy from the set, etc.).

[0065] Then, if the client sends a request for querying the optimal model optimization strategy and the client's attributes to the policy server, the policy server is configured to send the currently updated optimal model optimization strategy suitable for the client to the client according to the client's attributes.

[0066] In one embodiment, in order to generate one or more model optimization strategies to be verified, the policy server can be configured to: perform one or more changes in operator type conversion, operator replacement, operator splitting, and operator merging on one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified. All operator changes can be exhaustively enumerated here, that is, indiscriminate branch attempts are made, and all splitting and merging possibilities of each operator are tried to generate all possible model optimization strategies to be verified. Of course, when it is known that some operator changes (such as the merger or fusion of two operators) can obtain optimized results, the model optimization strategy to be verified with such changes can be generated and verified first.

[0067] In one embodiment, the policy server is configured to: based on the properties of at least one computing device, remove unsuitable model optimization strategies to be verified from one or more generated model optimization strategies to be verified. For example, the entire set of model optimization strategies to be verified can be eliminated in combination with hardware characteristics (such as the presence or absence of Tcore and the size of Tcore, etc.), all unsuitable model optimization strategies to be verified can be deleted, and the remaining selectable model optimization strategies to be verified can be randomly selected for verification. In this way, the number of model optimization strategies to be verified can be minimized.

[0068] Figure 3 A schematic flow chart showing the steps of requesting and obtaining an optimization strategy at a client according to an embodiment of the present application is shown.

[0069] At 301, the client may send a request to query and obtain the optimal model optimization strategy and the attributes of the client to the policy server. In the case where the policy server has solved the current optimal model optimization strategy suitable for the client, the policy server is configured to send the current optimal model optimization strategy suitable for the client to the client. At 302, the client may confirm that there is a current optimal model optimization strategy suitable for the client. At 305, the client applies the solved current optimal model optimization strategy.

[0070] When the policy server has not yet solved the current optimal model optimization strategy suitable for the client: the policy server is configured to assign one or more to-be-verified strategies among one or more to-be-verified model optimization strategies to the client based on the request, so that the client confirms that there are to-be-verified strategies at 303, decides to use the to-be-verified strategies at 304, and runs (or applies) one or more to-be-verified optimization models optimized by one or more to-be-verified strategies at 305.

[0071] In one embodiment, the client can configure the confidentiality of the client. If it is determined at 308 that the client is not set to confidentiality and has the right to return performance data, the client is configured to send the performance data obtained by running one or more optimization models to be verified to the policy server at 309. If the client is configured not to send the performance data obtained by running one or more policies to be verified to the policy server when confidentiality is set, it is determined at 308 not to return performance data, and the client will not return performance data, thereby increasing the confidentiality of the client.

[0072] Then, the policy server is configured to determine whether there is a current optimal model optimization policy suitable for the client among one or more policies to be verified based on the client's attributes sent by the client and the obtained performance data ( Figure 3 not shown).

[0073] In one embodiment, the client is configured to, at 306, determine whether the most recent model optimization strategy previously allocated to the client is cached, and if it is determined to be cached, apply the cached most recent model optimization strategy at 305. If it is determined that there is no cached most recent model optimization strategy, obtain previously cached performance data at 307, such as performance data obtained by the client running other strategies before. Afterwards, the client can determine whether to transmit back the previously cached performance data at 308. After the policy server obtains the previously cached performance data sent by the client, it can solve the optimization strategy based on the previously cached performance data to obtain the solved optimization strategy suitable for the client as soon as possible so as to send it to the client.

[0074] In this way, the model optimization strategy can be reused at the client, saving the computational loss caused by repeated calculation of the model on the client.

[0075] If the client has neither received the currently optimal model optimization strategy being solved, nor the strategy to be verified, nor the most recent model optimization strategy cached, the client can continue to wait for the strategy server to solve and allocate, or it can end.

[0076] In this way, the client does not solve the specific optimization algorithm, but only analyzes and applies the optimization strategy, which also reduces software upgrades at the client.

[0077] In one embodiment, the policy server is configured to partially solve the current optimal model optimization strategy when receiving partial performance data, and cache the current optimal model optimization strategy of the computing device, the current time (e.g., timestamp) associated therewith, and partial performance data in the memory. Thus, the policy server can collect (partial) performance data when various models are run by various types of computing devices and various computing tasks, so as to dynamically optimize the computing models running on various computing devices based on the collected performance data, and can be used for sharing and reuse.

[0078] The policy server can establish a black box cost model based on the collected performance data to calculate how many time cycles the operators in the optimization model will take, and solve the optimal optimization strategy based on the performance data. In this way, performance estimation data can also be provided to provide parameters for task scheduling in hybrid computing environments (such as large-scale clusters). The policy server can also gradually solve better model optimization strategies as it gradually receives more performance data, and send the optimal strategy for the current partial solution when the client requests it, and then continue to solve it offline based on more collected performance data.

[0079] Here, the performance data may include at least one of the following: the running time, clock count, memory usage, data transfer volume, number of errors (bugs), computing device temperature, etc., obtained by running one or more optimization models to be verified. In short, the performance data may include various parameters used to evaluate whether a model is the most suitable for running on the computing device. In most cases, the execution time can usually be used for evaluation. For example, if the computing device has the shortest execution time for running a certain optimization model to be verified, then the optimization model to be verified can be considered to be the best on the computing device, or the most suitable for the computing device.

[0080] The properties of a computing device may include, for example, the model of the computing device's chip, the type of chip (e.g., GPU chip), the amount of memory on the computing device's chip, supported bandwidth, the convolution operation capacity provided by the chip (5ms, 10ms, etc.), the composition of the computing kernel of the computing device, etc. In other words, the properties of a computing device are usually related to the computing performance of the computing device, especially the computing power of various operators in the artificial intelligence model. For example, different characteristics of different GPUs may lead to different model optimization strategies, while the optimization strategies of the same GPU may be the same or similar.

[0081] The policy server is configured to determine the to-be-verified model optimization policy that optimizes the performance data of at least one computing device as the current optimal model optimization policy suitable for at least one computing device with the attribute. In this way, it is possible to understand which optimization operations of which operators in which to-be-verified model optimization policy are suitable for the attributes of the chip of a computing device of a certain client.

[0082] For example, when a computing device has a specific chip model, a specific chip type, the amount of chip memory of a specific computing device, etc., a certain model optimization strategy to be verified has the shortest execution time, the fewest clocks, the least memory usage, the least data transmission, the least number of errors when running on the computing device (of course, only one of the parameters can be used as the evaluation criterion), etc., then it can be judged that the model optimization strategy to be verified is the optimal model optimization strategy most suitable for running on the computing device.

[0083] In one embodiment, the performance data includes at least one of running time, clock count, memory usage, and data transmission volume, wherein the policy server is configured to determine the to-be-verified model optimization strategy that optimizes the performance data of at least one computing device as the current optimal model optimization strategy suitable for at least one computing device.

[0084] In one embodiment, the policy server is configured to assign different optimization strategies for models to be verified to different clients in order to determine whether the attributes of the client are suitable for running the operator changes in the assigned optimization strategies for models to be verified. In this way, the performance of different chips of computing devices of different clients after running different changed operators in different optimization strategies for models to be verified can be understood, so that the performance relationship between the attributes of different chips of computing devices and different changed operators can be analyzed and derived. For example, the first optimization strategy for the model to be verified is to merge the first operator and the second operator. The second optimization strategy for the model to be verified is to merge the second operator and the third operator. The running time of the optimized model to be verified after merging the first operator and the second operator on the first GPU chip with the first attribute of the first computing device is shorter than the running time of the optimized model to be verified after merging the second operator and the third operator on the second GPU chip with the second attribute of the second computing device, then it can be determined that this optimization strategy of merging the first operator and the second operator is suitable for running on the first GPU chip with the first attribute. Thus, in one embodiment, if it is determined that the attributes of the client are suitable for running the operator change in the assigned model optimization strategy to be verified, the policy server is configured to assign the model optimization strategy to be verified with the operator change to the client with the attribute. For example, when the policy server receives a query optimization strategy request from a first computing device with a first GPU chip having a first attribute, it can assign the optimization strategy to be verified with the first operator and the second operator merged from one or more model optimization strategies to be verified, so that unsuitable optimization strategies to be verified can be eliminated to more efficiently determine the current optimal model optimization strategy suitable for the client.

[0085] In one embodiment, the policy server can automatically solve the current optimal model optimization strategy from the model optimization strategies to be verified without a query request from the client. The policy server is configured to: perform the solution step regularly; or perform the solution step in response to receiving new performance data; or a combination of the above. For example, the policy server can perform the above solution every hour, or perform the solution again in response to receiving performance data sent by a computing device. In short, the policy server can perform the solution task of the current optimal model optimization strategy offline, asynchronously, and automatically with the client, so that the solution of the model optimization strategy can be separated from the performance data collection and the actual application of the client's optimization strategy, so as to obtain efficient optimization effects.

[0086] Figure 4 A schematic diagram of an application scenario of a system for solving and applying an optimization strategy for an artificial intelligence model according to an embodiment of the present application is shown.

[0087] like Figure 4As shown, the application scenario includes an offline policy optimization service cluster, including multiple policy servers 401. Each policy server 401 completes the following services: scheduling service, optimization policy request service, offline optimization policy solution service, and data access interface service.

[0088] The scheduling service is responsible for the execution of various services within the overall scheduling strategy server.

[0089] The optimization strategy request service is responsible for receiving optimization strategy query requests from the client, querying the strategy database, and deciding which optimization strategy to send to the client. The request may also be a request from the client to obtain a performance test task. At this time, the optimization strategy service needs to collect performance data from the client after the client runs the optimization strategy to decide which optimization strategy is most suitable. The optimization strategy request service can also receive performance data from the client and persist the performance data, such as storing the performance data in a persistence device.

[0090] The offline optimization strategy solving service is responsible for using optimization strategy exploration algorithms, such as heuristics, deep learning, dynamic programming, etc., to perform offline calculations based on performance data from the client to obtain optimization strategies suitable for the hardware and framework, and store the solved optimization strategies in the optimization strategy database. The optimization strategy database can also store timestamps associated with the optimization strategy, such as the timestamp when the optimization strategy is solved. The offline optimization strategy solving service can also obtain strategies to be verified and persist these strategies to be verified. The offline optimization strategy solving service can also obtain competition failure strategies (solved as unsuitable strategies, which can form an optimization strategy blacklist) and store them in the optimization strategy database, which is used to remove these competition failure strategies from the strategies to be verified in the future to optimize calculation pruning.

[0091] The data access interface is mainly responsible for sending and receiving data with the persistence device and the service gateway to which the client is connected.

[0092] The performance data (including historical performance data) and optimization strategies (including the obtained model optimization strategies, the optimization strategy blacklist, and the optimization strategies to be verified) are persistently stored in the persistence device 402. Note that persistence can refer to saving data to persistent storage, such as a local disk or local distributed storage, cloud storage, etc., for subsequent reading.

[0093] The service gateway 404 is connected to the client 406. The client 406 may include various types of nodes, such as reasoning service nodes, training nodes, performance testing nodes, other performance data providing nodes, and the like.

[0094] The inference service node includes a local optimization time optimizer, a local optimization model cache, a performance data collection module, and an inference service engine. The training node may include a local optimization time optimizer, a local optimization model cache, a performance data collection module, and an inference service engine, just like the inference service node. The performance test node may only include a performance data collection module and a test task module. Other performance data providing nodes provide other performance data.

[0095] The node here is a logical concept of service, and a physical server can serve one or more clients as several different types of nodes at the same time.

[0096] The local runtime optimizer is responsible for optimizing the model according to the optimization strategy obtained from the policy server. The local optimization model cache is responsible for caching the optimization strategy.

[0097] The inference service engine loads the optimized model to be verified and performs static optimization, dynamic optimization, memory usage planning, inference task scheduling, etc.

[0098] The performance data collection module is responsible for collecting performance data of the optimized model to be verified, including but not limited to running time, clock count, memory usage, data transmission volume, number of errors, temperature, etc.

[0099] Figure 5 A schematic flow chart showing the steps of an offline optimization strategy solving service and an optimization strategy request service interacting with a client, mainly performed at a strategy server according to an embodiment of the present application.

[0100] The triggering condition for the policy server to perform the solution may be: performing the solution step regularly; or performing the solution step in response to receiving new performance data; or a combination of the above two. Figure 5 An example is shown in which the receipt of new performance data is used as a trigger for the solution step.

[0101] like Figure 5 As shown, in order to perform offline optimization strategy solving service, at 501, the strategy server determines whether new performance data has been received and updated. If not, it continues to wait for a timer at 502, and continues to determine whether new performance data has been received and updated after the timer expires. In other words, the receiving of new performance data is still used as a triggering solution step.

[0102] If at 501, the policy server determines that new performance data has been received and updated, the solution step is triggered. Specifically, at 503, the policy server obtains the updated performance data and the metadata of the corresponding model (such as the hardware information (attributes) of the computing device, model information, optimized operators (such as which operators have been changed), optimization methods, etc.). At 504, the policy server obtains the corresponding existing optimization strategy, data pruning strategy, etc., and calls the optimization strategy solution algorithm to solve the strategy and update the optimal optimization strategy based on various information.

[0103] like Figure 5 As shown, in order to perform the optimization policy request service interacting with the client, in 505, the policy server receives the optimization policy query request from the client, and in 506, the policy server can retrieve the optimization policy in the optimization policy database, and if there is a current optimal optimization policy, the current optimal optimization policy is determined in 506, and if there is no optimal optimization policy, a policy to be verified is generated, and different policies to be verified are allocated to different clients in 507. In 508, the policy server returns the determined current optimal optimization policy or the policy to be verified to the requesting client.

[0104] Note that the server and client mentioned in this article can both exist as applications. They can run on different physical machines or coexist on the same physical machine. There is no limitation here.

[0105] Figure 6 A schematic flow chart of a method 600 for solving and applying an optimization strategy for an artificial intelligence model according to an embodiment of the present application is shown.

[0106] like Figure 6 As shown, a method for solving and applying optimization strategies for artificial intelligence models includes: by a strategy server: step 601, one or more changes are made to one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified, and the strategies are assigned to at least one computing device for running; step 602, based on the properties of at least one computing device running one or more model optimization strategies to be verified and the performance data obtained by the at least one computing device running the one or more optimization models to be verified, solving the current optimal model optimization strategy suitable for at least one computing device in the one or more model optimization strategies to be verified; step 603, by a client: sending a request for querying the optimal model optimization strategy and the properties of the client to the strategy server, wherein the strategy server is configured to send the model optimization strategy suitable for the client to the client according to the properties of the client.

[0107] In one embodiment, method 600 includes: when the policy server has solved the current optimal model optimization strategy suitable for the client, the policy server sends the current optimal model optimization strategy suitable for the client to the client.

[0108] In one embodiment, method 600 includes: when the policy server has not yet solved the current optimal model optimization strategy suitable for the client: the policy server, based on the request, assigns one or more to-be-verified strategies from one or more to-be-verified model optimization strategies to the client so that the client runs one or more to-be-verified optimization models optimized by the one or more to-be-verified strategies, the client sends performance data obtained by running the one or more to-be-verified optimization models to the policy server, and the policy server determines whether there is a current optimal model optimization strategy suitable for the client among the one or more to-be-verified strategies based on the client attributes sent by the client and the obtained performance data.

[0109] In one embodiment, method 600 includes: when the policy server has not yet solved the current optimal model optimization strategy suitable for the client, and there is no one or more model optimization strategies to be verified or no model optimization strategy to be verified is assigned to the client: determining by the client whether the most recent model optimization strategy previously assigned to the client is cached, and if it is determined to be cached, applying the cached most recent model optimization strategy, and if it is determined not to be cached, sending the previously cached performance data to the policy server.

[0110] In one embodiment, method 600 includes: when the client is set to be confidential, the client does not send performance data obtained by running one or more policies to be verified to the policy server.

[0111] In one embodiment, method 600 includes: the policy server partially solves the current optimal model optimization strategy upon receiving partial performance data, and caches the current optimal model optimization strategy of the computing device, the current time associated therewith, and the partial performance data in a memory.

[0112] In one embodiment, the method 600 includes: allocating different to-be-verified model optimization strategies to different clients by a strategy server so as to determine whether the properties of the client are suitable for running the operator changes in the allocated to-be-verified model optimization strategy.

[0113] In one embodiment, method 600 includes: the policy server is configured to assign a model optimization strategy to be verified with an operator change to a client having the attributes if it is determined that the attributes of the client are suitable for running the operator change in the assigned model optimization strategy to be verified.

[0114] In one embodiment, method 600 includes: the policy server performs one or more changes including operator type conversion, operator replacement, operator splitting, and operator merging on one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified.

[0115] In one embodiment, method 600 includes: the policy server, based on the property of at least one computing device, removes unsuitable model optimization strategies to be verified from the generated one or more model optimization strategies to be verified.

[0116] In one embodiment, the method 600 includes: the policy server: regularly performing the solving step; or performing the solving step in response to receiving new performance data; or a combination of the above two.

[0117] Figure 7 A block diagram of a policy server 700 for solving an optimization strategy for an artificial intelligence model according to an embodiment of the present application is shown.

[0118] The policy server 700 includes: a generating device 701, configured to make one or more changes to one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified, and assign them to at least one computing device for running; a solving device 702, configured to solve the current optimal model optimization strategy suitable for at least one computing device in the one or more model optimization strategies to be verified based on the properties of at least one computing device running the one or more model optimization strategies to be verified and the performance data obtained by the at least one computing device running the one or more optimization models to be verified.

[0119] In this way, the optimization strategy is liberated to the policy server for offline execution, and the computing device is only designed to run the optimized model and feedback performance data, simplifying the implementation of the two parts.

[0120] The policy server 700 may further include a receiving device configured to receive a request from a client to query an optimal model optimization policy and attributes of the client.

[0121] In one embodiment, when the policy server 700 has solved the current optimal model optimization strategy suitable for the client, the solving device of the policy server 700 is configured to send the current optimal model optimization strategy suitable for the client to at least one computing device.

[0122] In one embodiment, when the policy server 700 has not yet solved the current optimal model optimization strategy suitable for the client: the generation device 701 of the policy server 700 is configured to allocate one or more to-be-verified strategies of one or more to-be-verified model optimization strategies to the client based on the request so that the client runs one or more to-be-verified optimization models optimized by the one or more to-be-verified strategies, and the client is configured to send performance data obtained by running one or more to-be-verified optimization models to the policy server.

[0123] In one embodiment, the solving device 702 of the policy server 700 is configured to determine whether there is a current optimal model optimization policy suitable for the client in one or more policies to be verified based on the client's attributes sent by the client and the obtained performance data.

[0124] In one embodiment, the solving device 702 of the policy server 700 is configured to partially solve the current optimal model optimization strategy upon receiving partial performance data, and cache the current optimal model optimization strategy of the computing device, the current time associated therewith, and partial performance data in a memory.

[0125] The generating device 701 of the policy server 700 is configured to assign different to-be-verified model optimization strategies to different clients so that the solving device 702 determines whether the properties of the client are suitable for running the operator changes in the assigned to-be-verified model optimization strategy.

[0126] If it is determined that the client's attributes are suitable for running the operator change in the assigned model optimization strategy to be verified, the generation device 701 of the strategy server 700 is configured to allocate the model optimization strategy to be verified with the operator change to the client with the attributes.

[0127] In one embodiment, the performance data includes at least one of running time, clock count, memory usage, data transmission volume, number of errors, and temperature, and the attributes of the computing device include: the model of the computing device's chip, the type of chip, the amount of chip memory of the computing device, the supported bandwidth, the convolution operation capacity provided by the chip, and at least one of the composition of the computing kernel of the computing device, wherein the policy server is configured to determine the to-be-verified model optimization strategy that optimizes the performance data of at least one computing device as the current optimal model optimization strategy suitable for at least one computing device having the attributes.

[0128] The generation device 701 of the policy server 700 is configured to perform one or more changes of operator type conversion, operator replacement, operator splitting, and operator merging on one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified.

[0129] The generating device 701 of the policy server 700 is configured to remove unsuitable model optimization strategies to be verified from the generated one or more model optimization strategies to be verified based on the attributes of at least one computing device.

[0130] The solution device 702 of the policy server 700 is configured to: perform the solution step regularly; or perform the solution step in response to receiving new performance data; or a combination of the above two.

[0131] In this way, the optimization strategy is liberated to the policy server for offline execution, and the optimization strategy solution in the model compilation process and the optimized model compilation and operation at the client can be separated and decoupled to a certain extent, simplifying the implementation of the two parts, centralizing the calculation at the policy server, and using large computing power to solve the optimization strategy offline, reducing the constraints on the performance requirements of the client when running alone, and reducing the model optimization burden at the client. The runtime calculation of the traditional model dynamic optimization can be converted into a query initiated by the client and a static optimization at the server, and the query and static optimization are energy-friendly and efficient, so this technology can achieve low energy consumption and high-efficiency strategy optimization. Moreover, the policy server can gradually receive and improve performance data and solve the optimal strategy from the client offline, and has more time and probability to solve the optimal strategy without affecting the workload of the client. In addition, the policy server can combine the specific attributes of the computing device and the performance data obtained by the computing device running the model to solve the current optimal model optimization strategy suitable for the computing device, so as to send the optimal model optimization strategy that best suits the attributes of the client to the client that issues the request later. Moreover, since the policy server solves and upgrades the current optimal model optimization strategy by itself, there is no need to upgrade the model optimization strategy for each client, which reduces a lot of workload. The policy server can also coordinate the relationship between the attributes of different clients and different model optimization strategies, understand which attributes are suitable for running which optimization strategy (operator change), and thus coordinate the allocation of optimization strategies to be verified to different clients and the generation of which optimization strategies to be verified.

[0132] Figure 8 A schematic flow chart of a method 800 for solving an optimization strategy for an artificial intelligence model according to an embodiment of the present application is shown.

[0133] like Figure 8As shown, method 800 includes: step 801, making one or more changes to one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified, and assigning them to at least one computing device for running; step 802, based on the properties of at least one computing device running one or more model optimization strategies to be verified and the performance data obtained by the at least one computing device running the one or more optimization models to be verified, solving the current optimal model optimization strategy suitable for at least one computing device in the one or more model optimization strategies to be verified.

[0134] In this way, the optimization strategy is liberated to the policy server for offline execution, and the computing device is only designed to run the optimized model and feedback performance data, simplifying the implementation of the two parts.

[0135] Fig. 9 A block diagram of a client 900 for applying an optimization strategy of an artificial intelligence model according to an embodiment of the present application is shown.

[0136] The client 900 includes: a sending device 901, configured to send a request for querying the optimal model optimization strategy and the client's attributes to the policy server; an application device 902, configured to receive from the policy server a model optimization strategy suitable for the client, which is sent to the client by the policy server based on the client's attributes, and apply the model optimization strategy.

[0137] In one embodiment, the client further includes a receiving device configured to receive one or more policies to be verified distributed from a policy server, and the application device of the client runs one or more optimization models to be verified optimized by the one or more policies to be verified.

[0138] In one embodiment, the client further includes a sending device configured to send performance data obtained by running one or more optimization models to be verified to the policy server.

[0139] In one embodiment, the policy server is configured to determine whether there is a current optimal model optimization policy suitable for the client in one or more policies to be verified based on the client's attributes sent by the client and the obtained performance data.

[0140] Alternatively, in one embodiment, the client is configured not to send the performance data obtained by running one or more policies to be verified to the policy server when confidentiality is set.

[0141] In one embodiment, the client also includes a determination device, which is configured to determine whether the most recent model optimization strategy previously allocated to the client is cached, when the policy server has not yet solved the current optimal model optimization strategy suitable for the client and does not have one or more model optimization strategies to be verified or does not assign a model optimization strategy to be verified to the client: if it is determined to be cached, the application device is configured to apply the cached most recent model optimization strategy, and if it is determined to be not cached, the sending device is configured to send the previously cached performance data to the policy server.

[0142] In this way, the client only needs to request the policy server when it needs to optimize the strategy. The policy server centrally calculates and uses large computing power to solve the optimization strategy offline, reducing the constraints on performance requirements when the client runs alone, and reducing the model optimization burden at the client. The runtime calculation of traditional model dynamic optimization can be converted into queries initiated by the client and static optimization at the server. Query and static optimization are energy-friendly and efficient, so this technology can achieve lower energy consumption and efficient policy optimization. Therefore, there is no need to upgrade the model optimization strategy for each client, which reduces a lot of workload.

[0143] Fig.10 A schematic flow chart of a method 1000 for applying an optimization strategy of an artificial intelligence model according to an embodiment of the present application is shown.

[0144] The method 1000 includes: step 1001, sending a request for querying the best model optimization strategy and the attributes of the client to the policy server; step 1002, receiving from the policy server a model optimization strategy suitable for the client sent to the client by the policy server based on the attributes of the client, and applying the model optimization strategy.

[0145] In this way, the calculation is centralized at the policy server, and the optimization strategy is solved offline using large computing power, which reduces the constraints on the performance requirements of the client when running alone, and reduces the model optimization burden at the client. The runtime calculation of the traditional model dynamic optimization can be converted into queries initiated by the client and static optimization at the server, and queries and static optimization are energy-friendly and efficient. Therefore, this technology can achieve low energy consumption and efficient strategy optimization. Therefore, there is no need to upgrade the model optimization strategy for each client, which reduces a lot of workload.

[0146] Fig.11 A block diagram of an exemplary electronic device suitable for implementing the embodiments of the present application is shown.

[0147] The electronic device may include a processor (H1); a storage medium (H2) coupled to the processor (H1) and storing computer executable instructions therein for performing the steps of each method of an embodiment of the present application when executed by the processor.

[0148] The processor (H1) may include, but is not limited to, one or more processors or microprocessors, etc.

[0149] The storage medium (H2) may include, but is not limited to, for example, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (such as hard disk, floppy disk, solid state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disc, etc.).

[0150] In addition, the electronic device may also include a data bus (H3), an input / output (I / O) bus (H4), a display (H5), and input / output devices (H6) (eg, keyboard, mouse, speaker, etc.).

[0151] The processor (H1) may communicate with external devices (H5, H6, etc.) through an I / O bus (H4) via a wired or wireless network (not shown).

[0152] The storage medium (H2) may also store at least one computer executable instruction for executing the various functions and / or method steps in the embodiments described in the present technology when the instruction is executed by the processor (H1).

[0153] In one embodiment, the at least one computer executable instruction may also be compiled into or constitute a software product, wherein one or more computer executable instructions are executed by a processor to perform the various functions and / or method steps in the embodiments described in the present technology.

[0154] Fig.12 A schematic diagram of a non-transitory computer-readable storage medium according to an embodiment of the present disclosure is shown.

[0155] like Fig.12As shown, instructions are stored on the computer-readable storage medium 1220, and the instructions are, for example, computer-readable instructions 1210. When the computer-readable instructions 1210 are executed by the processor, the various methods described above can be executed. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, the computer-readable storage medium 1220 may be connected to a computing device such as a computer, and then, when the computing device runs the computer-readable instructions 1210 stored on the computer-readable storage medium 1220, the various methods described above may be performed.

[0156] Of course, the above-mentioned specific embodiments are merely examples rather than limitations, and those skilled in the art can, according to the concept of the present application, merge and combine some steps and devices from the various embodiments described separately above to achieve the effects of the present application. Such merged and combined embodiments are also included in the present application, and such merges and combinations are not described one by one here.

[0157] Note that the advantages, strengths, effects, etc. mentioned in this disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of this application. In addition, the specific details disclosed above are only for the purpose of illustration and facilitation of understanding, not limitation, and the above details do not limit this application to being implemented by adopting the above specific details.

[0158] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0159] The step flow charts and the above method descriptions in this disclosure are only illustrative examples and are not intended to require or imply that the steps of each embodiment must be performed in the order given. As will be appreciated by those skilled in the art, the order of the steps in the above embodiments can be performed in any order. Words such as "thereafter", "then", "next", etc. are not intended to limit the order of steps; these words are only used to guide the reader through the description of these methods. In addition, any reference to a singular element, such as using the article "one", "one", or "the" is not to be construed as limiting the element to the singular.

[0160] In addition, the steps and devices in the various embodiments of this document are not limited to being implemented in a certain embodiment. In fact, according to the concept of this application, relevant partial steps and partial devices in the various embodiments of this document can be combined to conceive new embodiments, and these new embodiments are also included in the scope of this application.

[0161] Each operation of the method described above may be performed by any suitable means capable of performing the corresponding functions. The means may include various hardware and / or software components and / or modules, including but not limited to hardware circuits, application specific integrated circuits (ASICs) or processors.

[0162] The various illustrated logic blocks, modules and circuits may be implemented or described using a general purpose processor, a digital signal processor (DSP), an ASIC, a field programmable gate array signal (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but as an alternative, the processor may be any commercially available processor, controller, microcontroller or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, a microprocessor cooperating with a DSP core, or any other such configuration.

[0163] The steps of the method or algorithm described in conjunction with the present disclosure can be directly embedded in hardware, in a software module executed by a processor, or in a combination of the two. A software module can exist in any form of tangible storage medium. Some examples of storage media that can be used include random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, etc. A storage medium can be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. In an alternative manner, the storage medium can be integral with the processor. A software module can be a single instruction or many instructions, and can be distributed over several different code segments, between different programs, and across multiple storage media.

[0164] The methods disclosed herein include actions for implementing the described methods. Methods and / or actions may be interchangeable with each other without departing from the scope of the claims. In other words, unless a specific order of actions is specified, the order and / or use of specific actions may be modified without departing from the scope of the claims.

[0165] The above functions can be implemented by hardware, software, firmware or any combination thereof. If implemented in software, the functions can be stored as instructions on a tangible computer-readable medium. The storage medium can be any available tangible medium that can be accessed by a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage device or any other tangible medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer. As used herein, a disk and a disc include a compact disc (CD), a laser disc, an optical disc, a digital versatile disc (DVD), a floppy disc and a blue disc, wherein the disc usually reproduces data magnetically, and the disc reproduces data optically using a laser.

[0166] Thus, a computer program product can perform the operations presented herein. For example, such a computer program product can be a computer-readable tangible medium having instructions tangibly stored (and / or encoded) thereon, which instructions can be executed by a processor to perform the operations described herein. The computer program product can include packaging materials.

[0167] Software or instructions may also be transmitted via a transmission medium. For example, the software may be transmitted from a website, server or other remote source using a transmission medium such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio or microwave.

[0168] In addition, the module and / or other appropriate means for carrying out the methods and techniques described herein can be downloaded and / or otherwise obtained by the user terminal and / or base station when appropriate. For example, such a device can be coupled to a server to facilitate the transmission of the means for carrying out the methods described herein. Alternatively, the various methods described herein can be provided via a storage component (e.g., RAM, ROM, a physical storage medium such as a CD or a floppy disk, etc.) so that the user terminal and / or base station can obtain the various methods when being coupled to the device or providing a storage component to the device. In addition, any other appropriate technology for providing the methods and techniques described herein to the device can be utilized.

[0169] Other examples and implementations are within the scope and spirit of the present disclosure and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hard wiring, or any combination of these. Features that implement the functions can also be physically located in various locations, including being distributed so that parts of the functions are implemented at different physical locations. Moreover, as used herein, including as used in the claims, "or" used in a list of items that begin with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). In addition, the wording "exemplary" does not mean that the example described is preferred or better than other examples.

[0170] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0171] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0172] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A policy server for solving an optimization strategy for an artificial intelligence model, comprising a generating device, a solving device, and a receiving device: The generating device is configured to make one or more changes to one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified, and distribute them to at least one client for operation, wherein: The generating device is configured to: perform one or more changes of operator type conversion, operator replacement, operator splitting, and operator merging on one or more operators in the original artificial intelligence model to generate the one or more model optimization strategies to be verified, wherein the generating device is configured to assign different model optimization strategies to be verified to different clients so that the solving device determines whether the attributes of the client are suitable for running the operator changes in the assigned model optimization strategy to be verified, wherein if it is determined that the attributes of the client are suitable for running the operator changes in the assigned model optimization strategy to be verified, the generating device is configured to assign the model optimization strategy to be verified with the operator change to the client with the attribute; The solving device is configured to solve the current optimal model optimization strategy suitable for the at least one client among the one or more model optimization strategies to be verified based on the properties of at least one client running the one or more model optimization strategies to be verified and the performance data obtained by the at least one client running the one or more optimized models to be verified to optimize the original artificial intelligence model using the one or more model optimization strategies to be verified; The receiving device is configured to receive a request from a client for querying an optimal model optimization strategy and attributes of the client, Among them, when the policy server has not yet solved the current optimal model optimization strategy suitable for the client: the generating device is configured to allocate one or more to-be-verified strategies of the one or more to-be-verified model optimization strategies to the client based on the request so that the client can run one or more to-be-verified optimization models optimized by the one or more to-be-verified strategies, and the client is configured to send performance data obtained by running the one or more to-be-verified optimization models to the policy server.

2. The policy server according to claim 1, wherein: In the case where the policy server has solved the current optimal model optimization strategy suitable for the client, the solving device of the policy server is configured to send the current optimal model optimization strategy suitable for the client to the at least one client.

3. The policy server according to claim 1, wherein: The solving device is configured to determine whether there is a current optimal model optimization strategy suitable for the client among the one or more strategies to be verified based on the client's attributes sent by the client and the obtained performance data.

4. The policy server according to claim 1, wherein: The solving device is configured to partially solve the current optimal model optimization strategy when receiving partial performance data, and cache the current optimal model optimization strategy of the client, the current time associated therewith, and the partial performance data in a memory.

5. The policy server according to claim 1, wherein: The performance data includes at least one of running time, clock count, memory usage, data transmission volume, number of errors, and temperature; the attributes of the client include: the model of the client's chip, the type of chip, the amount of chip memory of the client, the supported bandwidth, the convolution operation capacity provided by the chip, and at least one of the composition of the client's computing core, wherein the policy server is configured to determine the to-be-verified model optimization strategy that optimizes the performance data of at least one client as the current optimal model optimization strategy suitable for at least one client having the attributes.

6. The policy server according to claim 1, wherein: The generating device is configured to eliminate unsuitable model optimization strategies to be verified from the generated one or more model optimization strategies to be verified based on the attributes of the at least one client.

7. The policy server according to claim 1, wherein: The solution device is configured to: perform the solution step regularly; or perform the solution step in response to receiving new performance data; or a combination of the two.

8. A method for solving an optimization strategy for an artificial intelligence model, comprising: One or more changes are made to one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified, and the strategies are assigned to at least one client for running, wherein different model optimization strategies to be verified are assigned to different clients in order to determine whether the attributes of the client are suitable for running the operator changes in the assigned model optimization strategy to be verified, wherein if it is determined that the attributes of the client are suitable for running the operator changes in the assigned model optimization strategy to be verified, the model optimization strategy to be verified with the operator changes is assigned to the client with the attributes; Based on the properties of at least one client running the one or more model optimization strategies to be verified and the performance data of one or more optimized models to be verified obtained by the at least one client running the one or more optimized models to be verified to optimize the original artificial intelligence model using the one or more model optimization strategies to be verified, solving the current optimal model optimization strategy suitable for the at least one client among the one or more model optimization strategies to be verified; receiving a request from a client for querying an optimal model optimization strategy and attributes of the client, Wherein, when the current optimal model optimization strategy suitable for the client has not yet been solved: based on the request, one or more of the one or more model optimization strategies to be verified are allocated to the client so that the client runs one or more optimization models to be verified optimized by the one or more optimization models to be verified, and the client is configured to send performance data obtained by running the one or more optimization models to be verified.

9. A client for applying an optimization strategy for an artificial intelligence model, comprising: A sending device configured to send a request for querying the optimal model optimization strategy and the attributes of the client to the policy server; An application device is configured to receive from the policy server a model optimization strategy suitable for the client, which is sent to the client by the policy server according to the attributes of the client, and apply the model optimization strategy, wherein the model optimization strategy is used to optimize the original artificial intelligence model to obtain an optimized artificial intelligence model, wherein different clients are assigned different to-be-verified model optimization strategies in order to determine whether the attributes of the client are suitable for running the operator change in the assigned to-be-verified model optimization strategy, wherein if it is determined that the attributes of the client are suitable for running the operator change in the assigned to-be-verified model optimization strategy, the client having the attribute is assigned the to-be-verified model optimization strategy having the operator change, The client further includes a receiving device configured to receive one or more strategies to be verified allocated from the strategy server, and the application device of the client runs one or more optimization models to be verified optimized by the one or more strategies to be verified, wherein one or more operators in the original artificial intelligence model are subjected to one or more changes of operator type conversion, operator replacement, operator splitting, and operator merging to generate the one or more strategies to be verified, The sending device is configured to send performance data obtained by running the one or more optimization models to be verified to the policy server, Wherein, the client further includes a determination device, which is configured to determine whether the most recent model optimization strategy previously allocated to the client is cached when the policy server has not yet solved the current optimal model optimization strategy suitable for the client and does not have one or more model optimization strategies to be verified or does not allocate a model optimization strategy to be verified to the client: and when it is determined to be cached, the application device is configured to apply the cached most recent model optimization strategy, and when it is determined to be not cached, the sending device is configured to send the previously cached performance data to the policy server.

10. The client according to claim 9, wherein: The policy server is configured to determine whether there is a current optimal model optimization policy suitable for the client among the one or more policies to be verified based on the attributes of the client sent by the client and the obtained performance data.

11. A method for applying an optimization strategy for an artificial intelligence model, comprising: Send a request to query the optimal model optimization strategy and the client's attributes to the strategy server; Receiving from the policy server a model optimization strategy suitable for the client, which is sent by the policy server to the client according to the attributes of the client, and applying the model optimization strategy, wherein the model optimization strategy is used to optimize the original artificial intelligence model to obtain an optimized artificial intelligence model, wherein different clients are assigned different to-be-verified model optimization strategies in order to determine whether the attributes of the client are suitable for running the operator change in the assigned to-be-verified model optimization strategy, wherein if it is determined that the attributes of the client are suitable for running the operator change in the assigned to-be-verified model optimization strategy, the client having the attribute is assigned the to-be-verified model optimization strategy having the operator change; The method also includes receiving one or more strategies to be verified assigned from the strategy server, and the application device of the client runs one or more optimization models to be verified optimized by the one or more strategies to be verified, wherein one or more operators in the original artificial intelligence model are subjected to one or more changes in operator type conversion, operator replacement, operator splitting, and operator merging to generate the one or more strategies to be verified, and sending performance data obtained by running the one or more optimization models to be verified to the strategy server. The method also includes, when the policy server has not yet solved the current optimal model optimization strategy suitable for the client and does not have one or more model optimization strategies to be verified or does not assign a model optimization strategy to be verified to the client: determining whether the most recent model optimization strategy previously assigned to the client is cached, and if it is determined to be cached, applying the cached most recent model optimization strategy, and if it is determined not to be cached, sending the previously cached performance data to the policy server.

12. A system for solving and applying an optimization strategy for an artificial intelligence model, comprising: The policy server is configured as: One or more changes are made to one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified, and the strategies are assigned to at least one client for running, wherein the generation includes: one or more changes of operator type conversion, operator replacement, operator splitting, and operator merging are made to one or more operators in the original artificial intelligence model to generate the one or more model optimization strategies to be verified, wherein different model optimization strategies to be verified are assigned to different clients in order to determine whether the attributes of the client are suitable for running the operator changes in the assigned model optimization strategy to be verified, wherein if it is determined that the attributes of the client are suitable for running the operator changes in the assigned model optimization strategy to be verified, the model optimization strategy to be verified with the operator changes is assigned to the client with the attributes; Based on the properties of at least one computing device running the one or more model optimization strategies to be verified and the performance data of one or more optimized models to be verified obtained by the at least one client running the one or more optimized models to be verified to optimize the original artificial intelligence model using the one or more model optimization strategies to be verified, solving the current optimal model optimization strategy suitable for the at least one client among the one or more model optimization strategies to be verified; receiving a request from a client for querying an optimal model optimization strategy and attributes of the client, Wherein, in the case where the current optimal model optimization strategy suitable for the client has not yet been solved: based on the request, one or more to-be-verified strategies of the one or more to-be-verified model optimization strategies are allocated to the client so that the client runs one or more to-be-verified optimization models optimized by the one or more to-be-verified strategies, and the client is configured to send performance data obtained by running the one or more to-be-verified optimization models, The client is configured to send a request for querying the optimal model optimization strategy and the attributes of the client to the policy server, and to send performance data obtained by running the one or more optimization models to be verified to the policy server, wherein the policy server is configured to send a model optimization strategy suitable for the client to the client based on the attributes of the client.

13. A method for solving and applying an optimization strategy for an artificial intelligence model, comprising: By the policy server: One or more changes are made to one or more operators in the original artificial intelligence model to generate one or more model optimization strategies to be verified, and the strategies are assigned to at least one client for running, wherein the generation includes: one or more changes of operator type conversion, operator replacement, operator splitting, and operator merging are made to one or more operators in the original artificial intelligence model to generate the one or more model optimization strategies to be verified, wherein different model optimization strategies to be verified are assigned to different clients in order to determine whether the attributes of the client are suitable for running the operator changes in the assigned model optimization strategy to be verified, wherein if it is determined that the attributes of the client are suitable for running the operator changes in the assigned model optimization strategy to be verified, the model optimization strategy to be verified with the operator changes is assigned to the client with the attributes; Based on the properties of at least one client running the one or more model optimization strategies to be verified and the performance data of one or more optimized models to be verified obtained by the at least one client running the one or more optimized models to be verified to optimize the original artificial intelligence model using the one or more model optimization strategies to be verified, solving the current optimal model optimization strategy suitable for the at least one client among the one or more model optimization strategies to be verified; receiving a request from a client for querying an optimal model optimization strategy and attributes of the client, Wherein, in the case where the current optimal model optimization strategy suitable for the client has not yet been solved: based on the request, one or more to-be-verified strategies of the one or more to-be-verified model optimization strategies are allocated to the client so that the client runs one or more to-be-verified optimization models optimized by the one or more to-be-verified strategies, and the client is configured to send performance data obtained by running the one or more to-be-verified optimization models, By the client: A request for querying the optimal model optimization strategy and the attributes of the client are sent to the policy server, and performance data obtained by running the one or more optimization models to be verified are sent to the policy server, wherein the policy server is configured to send a model optimization strategy suitable for the client to the client based on the attributes of the client.

14. An electronic device comprising: A memory for storing instructions; A processor, configured to read instructions in the memory and execute the method as claimed in any one of claims 8, 11, and 13.

15. A non-transitory storage medium having stored thereon instructions, in, When the instructions are read by a processor, the processor is caused to execute the method according to any one of claims 8, 11, and 13.