Multi-target execution plan optimization method, system and product for intelligent application

Abstract

The invention discloses a multi-target execution plan optimization method, system and product oriented to intelligent application, and belongs to the technical field of intelligent application. A logic application diagram and context information are obtained, and the context information comprises a service level target and a load feature; the method comprises the steps that a logic application diagram, a service level target, load characteristics and a pre-constructed operator performance model are integrated to obtain a constraint optimization problem, a multi-target constraint optimization solver is used for solving the constraint optimization problem, an optimal physical execution plan is generated, and an execution engine is used for executing the optimal physical execution plan. According to the method, the multi-dimensional service target can be understood and adjusted in real time by obtaining the service level target set by the user, the hysteresis is avoided by generating the optimal physical execution plan before plan execution, the self-adaptive capability and the quality of the generated result are improved by using the pre-constructed operator performance model and the multi-target constraint optimization solver, and the method has the advantages of being high in practicability and the like. And the scientificity, the globality and the reproducibility of a scheduling decision are ensured.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent application, and particularly relates to a multi-target execution plan optimization method, system and product for intelligent application. BACKGROUND

[0002] Intelligent application is an intelligent solution built by combining artificial intelligence technology as the core and digital means such as big data. It is widely used in various fields such as intelligent manufacturing, smart medical care and smart city. In recent years, with the deep integration of AI technology and business, intelligent application is no longer a simple task concatenation, but more and more contains "heterogeneous operators" with huge differences in computing resources, money cost and result quality requirements, such as containing light format conversion and calling expensive GPU resources in a large model inference. How to maximize efficiency while meeting user requirements for time, cost and quality has become one of the core challenges of existing technology.

[0003] The implementation of AI application in the prior art mainly includes the following ways, (1) developer manually hard-coded decision: mainly using AI application frameworks represented by LangChain and AutoGen, which can greatly simplify the complexity of building AI application; (2) reactive resource scheduling based on technical indicators: mainly represented by HPA (Horizontal Pod Autoscaler) of Kubemetes or higher level event-driven automatic scaling, by monitoring real-time technical indicators (such as CPU usage, memory consumption and queue length, etc.), dynamically increasing or reducing the number of service instances; (3) query plan optimization in traditional database and big data field: mainly represented by Spark SQL, PrestoDB or cost optimizer in traditional relational database, by exploring multiple physical execution paths and estimating the cost of each path, selecting a physical plan with the lowest cost to execute; (4) Pareto frontier exploration based on heuristic algorithm: using meta-heuristic algorithms to solve multi-objective optimization problems, such as genetic algorithm, particle swarm optimization algorithm or ant colony algorithm, to calculate a Pareto optimal frontier, which is a set of multiple non-dominated solutions, showing all possible and optimal trade-offs to the user.

[0004] But the above-mentioned mode also has corresponding defects, such as (1) unable to understand multi-dimensional business target: the optimization target almost focuses on two technical indexes of execution time and calculation resource consumption, and cannot understand and handle a multi-dimensional business target containing money cost and result quality; (2) mechanism has hysteresis: the reactive resource scheduling mechanism based on technical indexes is "reactive", and the expansion action is triggered only after the load pressure is observed, so when the load peak comes, the user experience has been damaged, the execution of the expansion action is late, and it is completely ineffective for one-time and non-continuous running intelligent application; (3) lack of adaptive ability and quality depends on the personal experience of experts: due to dynamic changes of business demand, when the customer proposes new demand, the system lacks the adaptive ability to business changes, needs the developer to modify repeatedly through in-depth code testing, and the number of potential execution plan combinations in the intelligent application is very large, and the generated decision quality completely depends on the personal experience of experts.

[0005] Therefore, how to provide an effective technical solution to solve the problems of unable to understand multi-dimensional business target, mechanism has hysteresis and lack of adaptive ability and quality depends on the personal experience of experts in the prior art has become a difficult problem to be solved in the prior art. SUMMARY

[0006] The purpose of the present application is to provide a multi-target execution plan optimization method and system for intelligent application, which solves the above-mentioned problems in the prior art.

[0007] In order to achieve the above-mentioned purpose, the present application adopts the following technical solutions: In a possible design, before obtaining the logical application graph and the context information, the method further includes: Obtaining a plurality of logical operators in a flow, defining each logical operator in the flow to obtain a physical operator corresponding to each logical operator, the physical operator being used to represent a physical implementation manner of the logical operator, and constructing a physical operator list based on the physical operator; Integrating the logical application graph, the service level target, the load characteristic and the pre-constructed operator performance model to obtain a constraint optimization problem; Solving the constraint optimization problem by using a multi-objective constraint optimization solver to generate an optimal physical execution plan; Executing the optimal physical execution plan by using an execution engine.

[0008] In a possible design, before obtaining the logical application graph and the context information, the method further includes: Obtaining a plurality of logical operators in a flow, defining each logical operator in the flow to obtain a physical operator corresponding to each logical operator, the physical operator being used to represent a physical implementation manner of the logical operator, and constructing a physical operator list based on the physical operator; Benchmark each physical operator in the physical operator list to obtain an operator performance model.

[0009] In a possible design, benchmarking each physical operator in the physical operator list to obtain an operator performance model comprises the following steps. Testing each physical operator in the physical list using different resource configurations and different input loads to obtain execution time and execution cost; Scoring each physical operator using an evaluation model or manual annotation to obtain a result quality score; Constructing an operator performance model based on the execution time, the execution cost, and the result quality score.

[0010] In a possible design, integrating the logical application graph, the service level objective, the load characteristics, and the pre-constructed operator performance model to obtain a constraint optimization problem comprises the following steps. Obtaining a physical implementation of each logical operator in the logical application graph, and generating a decision variable according to the physical implementation of each logical operator in the logical application graph; Obtaining a constraint condition and an objective function according to the service level objective; Integrating the pre-constructed operator performance model, the decision variable, the constraint condition, the objective function, and the load characteristics to generate the constraint optimization problem.

[0011] In a possible design, solving the constraint optimization problem using a multi-objective constraint optimization solver to generate an optimal physical execution plan comprises the following steps. Using the pre-constructed operator performance model to provide quantitative data for the constraint optimization problem; Based on the quantitative data, solving the decision variable using the multi-objective constraint optimization solver to obtain the optimal physical execution plan.

[0012] In a possible design, the multi-objective constraint optimization solver comprises a mixed integer programming solver, a constraint satisfaction problem solver, and / or a heuristic algorithm solver.

[0013] In a possible design, the service level objective comprises at least one of the following. A constraint or optimization objective for the total execution time of the entire logical application graph; A constraint or optimization objective for the total execution cost of the entire logical application graph; A constraint for the minimum result quality score of the output result of the entire logical application graph.

[0014] In a second aspect, the present application provides a multi-objective execution plan optimization system for intelligent applications, comprising: an acquisition module, configured to acquire a logical application graph and context information, the context information including a service level objective and load characteristics; an integration module, configured to integrate the logical application graph, the service level objective, the load characteristics, and a pre-built operator performance model to obtain a constrained optimization problem; a solution module, configured to solve the constrained optimization problem using a multi-objective constrained optimization solver to generate an optimal physical execution plan; an execution module, configured to execute the optimal physical execution plan using an execution engine.

[0015] In a possible design, the apparatus further includes a storage module, configured to store the operator performance model.

[0016] In a third aspect, the present application provides a multi-objective execution plan optimization apparatus for intelligent applications, including a memory, a processor, and a transceiver connected in sequence and in communication, wherein the memory is configured to store a computer program, the transceiver is configured to receive and send messages, and the processor is configured to read the computer program and execute the multi-objective execution plan optimization method for intelligent applications according to the first aspect.

[0017] In a fourth aspect, the present application provides a computer program product containing instructions, which, when executed on a computer, cause the computer to execute the multi-objective execution plan optimization method for intelligent applications according to the first aspect.

[0018] The present application has the following beneficial effects: (1) The present application discloses a multi-objective execution plan optimization method, system, and product for intelligent applications. The logical application graph and context information are acquired, wherein the context information includes a service level objective and load characteristics. The logical application graph, the service level objective, the load characteristics, and a pre-built operator performance model are integrated to obtain a constrained optimization problem. The constrained optimization problem is solved using a multi-objective constrained optimization solver to generate an optimal physical execution plan. The optimal physical execution plan is executed using an execution engine. (2) The present application allows users to directly define service level objectives using business language and generates an optimal physical execution plan according to the logical application graph and the service level objective. The business objectives can be adjusted in real time, and the adjustment is automatically reflected in the technical implementation, improving execution efficiency and shortening the execution cycle. (3) The present application obtains an optimal execution plan before plan execution, ensuring that the execution time, execution cost, and execution quality meet the requirements of the service level objective. It can completely change the passive mode of existing technologies, which expands after an incident occurs, and avoid cost waste caused by excessive configuration or incorrect selection of implementation schemes from a global perspective. (4) The application models operator data, finds an optimal physical execution plan by using a multi-objective constraint optimization solver, improves self-adaptive capability and quality, avoids problems of low efficiency and easy errors, and ensures scientificity, globality and reproducibility of scheduling decisions, facilitating application and promotion. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A flowchart of a multi-objective execution plan optimization method provided by the first aspect of the embodiment of the application is shown in the figure. Figure 2 A module diagram of a multi-objective execution plan optimization system provided by the second aspect of the embodiment of the application is shown in the figure. Figure 3 A structure diagram of a multi-objective execution plan optimization device provided by the third aspect of the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the application or the prior art, the application will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the drawings is only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings. It should be noted that the description of these embodiments is used to help understand the application, but does not constitute a limitation on the application.

[0021] It should be understood that although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element can be called a second element, and similarly, a second element can be called a first element without departing from the scope of the example embodiments of the application.

[0022] It should be understood that for the term "and / or" that may appear in the present text, it is only a description of the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, A and / or B, which means that there are three cases of A alone, B alone, and A and B together; for the term " / and" that may appear in the present text, it is another description of the relationship of another associated object, which means that there can be two kinds of relationships, for example, A / and B, which means that there are two cases of A alone and A and B together; in addition, for the character " / " that may appear in the present text, it generally means that the associated objects before and after are an "or" relationship.

[0023] Embodiment: As Figure 1As shown, the first aspect of the embodiment provides a multi-objective execution plan optimization method for intelligent applications, which can be executed by a computer device or a virtual machine with certain computing resources, such as an electronic device such as a personal computer or a smart phone, or a virtual machine; the multi-objective execution plan optimization method includes but is not limited to the following steps: S1. Obtain a logical application graph and context information, wherein the context information includes service level objectives and load characteristics; Wherein, the logical application is a series of AI operators and platform computing tasks arranged to achieve a specific business objective, and the logical application graph in the embodiment includes but is not limited to a directed acyclic graph (DAG); the service level objectives are a combination of constraints and / or optimization objectives specified by the user, when the service level objectives are a combination of one or more constraints, examples can be {total execution time < 60s, total execution cost < 0.5 yuan, minimum quality score greater than 0.8}; when the service level objectives are an optimization objective, an example can be {under the constraint of total execution time < 120s, try to reduce the cost as much as possible}.

[0024] In the embodiment, the context information further includes load characteristics, which are used to describe the key characteristics of the input data to be processed this time, such as {text length: 1.5MB}.

[0025] In specific implementation, the service level objectives at least include any one of the following: a constraint or optimization objective for the total execution time of the entire logical application graph; a constraint or optimization objective for the total execution cost of the entire logical application graph; a constraint for the minimum result quality score of the output result of the entire logical application graph.

[0026] In a possible design, before obtaining the logical application graph and the context information, further comprising: obtaining a plurality of logical operators in a flow, defining each logical operator in the flow to obtain a physical operator corresponding to each logical operator, the physical operator being used to represent a physical implementation of the logical operator, and constructing a physical operator list based on the physical operator; performing a benchmark test on each physical operator in the physical operator list to obtain an operator performance model.

[0027] Wherein, the benchmark test on each physical operator in the physical operator list to obtain the operator performance model specifically includes: testing each physical operator in the physical list using different resource configurations and different input loads to obtain execution time and execution cost; scoring each physical operator using an evaluation model or manual annotation to obtain a result quality score; and constructing an operator performance model based on the execution time, the execution cost and the result quality score.

[0028] It should be noted that in this embodiment, the logical operator is a node of the logical application graph, which defines a functional unit that needs to be executed, such as the "text summary" logical operator and the "speech recognition" logical operator; the physical operator is a specific implementation of the logical operator, and one logical operator can correspond to multiple physical operators, for example, the "text summary" logical operator can be implemented by the "GPT-4 API" physical operator, the "GPT-3.5 API" or the "local Llama3 model" physical operator.

[0029] In specific implementation, by benchmarking each physical operator in the physical operator list, the corresponding performance portrait, i.e., the operator performance model, is established, in other words, for each specific physical operator, its performance is tested under different resource configurations and different input loads, and is recorded to form a multi-dimensional performance data model. The performance data model should at least include the following relationships Taking the "local Llama3 model" physical operator as an example, Llama3 (load = 10KB, resource configuration = {GPU: A100}) (execution time = 5s, execution cost = 0.01 yuan, result quality score = 0.85), wherein the result quality score can be calculated by other evaluation models or manual annotation, and the other evaluation models include but are not limited to the GPT-4 (Generative Pre-trained Transformer-4) scorer, and the manual annotation can calculate the score of the result according to the preset scoring rule, and the preset scoring rule can be the keyword score, which is artificially preset.

[0030] S2. Integrating the logical application graph, the service level target, the load characteristics and the pre-built operator performance model to obtain a constraint optimization problem; Specifically, in step S2, the logical application graph, the service level target, the load characteristics and the pre-built operator performance model are integrated to obtain a constraint optimization problem, including: S2.1. Obtaining the physical implementation of each logical operator in the logical application graph, and generating a decision variable according to the physical implementation of each logical operator in the logical application graph; S2.2. Obtaining a constraint condition and an objective function according to the service level target; S2.3. Integrating the pre-built operator performance model, the decision variable, the constraint condition, the objective function and the load characteristics to generate a constraint optimization problem.

[0031] It should be noted that the decision variable takes the value of 1 or 0; the service level target is defined to obtain the objective function, for example, if the service level target is to minimize the cost, then the objective function is , wherein a logical operator, a logical operator a physical implementation of a decision variable; the constraints include but are not limited to SLO (Service Level Objective) constraints and logical constraints, the SLO constraints can be obtained from pre-built operator performance models, for example, the SLO constraints are , , wherein , and are obtained from pre-built operator performance models; the logical constraints can be exemplified as for each logical operator, and only one physical implementation can be selected, that is, it can be expressed as .

[0032] S3. using a multi-objective constrained optimization solver to solve the constrained optimization problem to generate an optimal physical execution plan; Specifically, in step S3, the multi-objective constrained optimization solver is used to solve the constrained optimization problem to generate an optimal physical execution plan, including: S3.1. using pre-built operator performance models to provide quantitative data for the constrained optimization problem; S3.2. based on the quantitative data, using a multi-objective constrained optimization solver to solve the decision variable to obtain an optimal physical execution plan.

[0033] It should be noted that the multi-objective constrained optimization solver includes but is not limited to a mixed integer programming solver, a constraint satisfaction problem solver, and / or a heuristic algorithm solver. The mixed integer programming solver (Mixed Integer Programming, MIP) is often used to solve linear programming problems containing continuous variables and integer variables, and its core principle is to use the branch and bound method as the basic framework, combined with the cut plane method and heuristic algorithm for solving; the principle of the constraint satisfaction problem solver is to search and infer in combination to find solutions or prove mistakes, and its core methods include backtracking search, constraint propagation and heuristic strategy; in the heuristic algorithm solver, the heuristic algorithm includes but is not limited to genetic algorithm, simulated annealing algorithm and / or local search algorithm.

[0034] Specifically, the optimal physical execution plan is a specific and directly executable configuration list, which explicitly specifies the physical operator bound to each logical operator and its resource requirements, that is, each logical operator specifies a specific physical implementation and its required resource configuration from its multiple available physical implementations, for example, logical operator 1 (ASR) Binding 'local Whisper model' with resource requirement = '{GPU: V100}' indicates that the ASR logic operator is implemented using the local Whisper model, and the required resource configuration is a V100 GPU; Logic operator 2 (summary) Binding to 'GPT-4 API' with resource requirement = 'N / A' indicates that the digest logic operator is implemented using the GPT-4 API and no resource configuration is required.

[0035] S4. Use the execution engine to execute the optimal physical execution plan.

[0036] It should be noted that the execution engine can be Kubernetes or Argo Workflows, etc. In practice, the execution engine instantiates resources and executes the application according to the optimal physical execution plan.

[0037] In this embodiment, steps S1-S4 are online execution steps, while constructing the OperatorPerformance Profile (OPP) is an offline execution step.

[0038] Based on the above disclosure, this embodiment discloses a multi-objective execution plan optimization method for intelligent applications. By unifying unrelated business indicators, such as execution time, execution cost, and result quality, into mathematical constraints and objectives, high-level business concepts are mapped to low-level solvable problems, and abstract result quality is included in the optimization scope. This method can solve the scheduling problems of complex applications in the AI ​​era. Furthermore, the essence of traditional mechanisms is ex-post compensation, while the essence of this invention is ex-ante calculation. By forming an optimization closed loop through "offline performance modeling + online constraint solving," expansion does not begin only when resources are exhausted, but the optimal physical implementation combination is calculated before the task begins, thereby completely avoiding performance problems. This embodiment allows users to directly issue an instruction containing specific business constraints, namely the service level objective of this application. This instruction is automatically translated into a formalized constraint optimization problem, and a solver is used to calculate the unique optimal answer that satisfies the instruction. This transforms the exploration problem into a solution problem, achieving a higher level of automation, and can be directly aligned with business objectives, thus having broad application prospects.

[0039] like Figure 2 As shown, the second aspect of this embodiment provides a multi-objective execution plan optimization system for intelligent applications, including: The acquisition module is used to acquire logical application graphs and context information, including service level targets and load characteristics. The integration module is used to integrate the logical application graph, service level objectives, load characteristics, and pre-built operator performance models to obtain a constrained optimization problem; a solving module, configured to solve the constraint optimization problem using a multi-objective constraint optimization solver to generate an optimal physical execution plan; an executing module, configured to execute the optimal physical execution plan using an execution engine.

[0040] In a possible design, the system further includes a storage module, configured to store the operator performance model.

[0041] The working process, working details and technical effects of the multi-objective execution plan optimization system for intelligent applications provided in the second aspect of the embodiment can be referred to the multi-objective execution plan optimization method for intelligent applications described in the first aspect, and will not be repeated here.

[0042] As shown in Figure 3 The third aspect of the embodiment provides a multi-objective execution plan optimization apparatus for intelligent applications, which includes a memory, a processor and a transceiver connected in sequence, wherein the memory is configured to store a computer program, the transceiver is configured to receive and send messages, and the processor is configured to read the computer program and execute the multi-objective execution plan optimization method for intelligent applications described in the first aspect. Specifically, the memory can include, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a flash memory, a first input first output (FIFO) memory and / or a first input last output (FILO) memory, and the like; and the processor can be, but is not limited to, a microprocessor with a model number of STM32F105 series. In addition, the computer device can further include, but is not limited to, a power module, a display screen and other necessary components.

[0043] The working process, working details and technical effects of the multi-objective execution plan optimization apparatus for intelligent applications provided in the third aspect of the embodiment can be referred to the multi-objective execution plan optimization method for intelligent applications described in the first aspect, and will not be repeated here.

[0044] The fourth aspect of the embodiment provides a computer program product, which includes a computer program or instructions, and the computer program or the instructions are configured to implement the multi-objective execution plan optimization method for intelligent applications described in the first aspect when executed by a computer.

[0045] The working process, working details and technical effects of the computer program product provided in the fourth aspect of the embodiment can be referred to the multi-objective execution plan optimization method for intelligent applications described in the first aspect, and will not be repeated here.

[0046] It should be pointed out finally that the above only describes the preferred embodiments of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A multi-objective execution plan optimization method for intelligent applications, characterized in that, The method comprises the following steps: obtaining a logical application graph and context information, wherein the context information comprises a service level objective and load characteristics; integrating the logical application graph, the service level objective, the load characteristics and a pre-built operator performance model to obtain a constraint optimization problem; solving the constraint optimization problem by using a multi-objective constraint optimization solver to generate an optimal physical execution plan; executing the optimal physical execution plan by using an execution engine. 2.The multi-objective execution plan optimization method for intelligent application oriented application according to claim 1, wherein, Before the step of obtaining the logical application graph and the context information, the method further comprises the following steps: obtaining a plurality of logical operators in a flow, defining each logical operator in the flow to obtain a physical operator corresponding to each logical operator, wherein the physical operator is used to represent a physical implementation manner of the logical operator, and a physical operator list is constructed based on the physical operator; benchmark testing each physical operator in the physical operator list to obtain an operator performance model.

3. The method of claim 2, wherein, The step of benchmark testing each physical operator in the physical operator list to obtain an operator performance model comprises the following steps: testing each physical operator in the physical list by using different resource configurations and different input loads to obtain execution time and execution cost; scoring each physical operator by using an evaluation model or manual annotation to obtain a result quality score; constructing an operator performance model based on the execution time, the execution cost and the result quality score.

4. The method of claim 1, wherein, The step of integrating the logical application graph, the service level objective, the load characteristics and the pre-built operator performance model to obtain a constraint optimization problem comprises the following steps: obtaining a physical implementation manner of each logical operator in the logical application graph, and generating a decision variable according to the physical implementation manner of each logical operator in the logical application graph; obtaining a constraint condition and an objective function according to the service level objective; integrating the pre-built operator performance model, the decision variable, the constraint condition, the objective function and the load characteristics to generate a constraint optimization problem.

5. The method of claim 1, wherein, The multi-objective constraint optimization solver comprises a mixed integer programming solver, a constraint satisfaction problem solver and / or a heuristic algorithm solver.

6. The method of claim 1, wherein, The service level objective at least comprises any one of the following: a constraint or optimization objective for a total execution time of the entire logical application graph; a constraint or optimization objective for a total execution cost of the entire logical application graph; a constraint for a minimum result quality score of a result output by the entire logical application graph.

7. A smart application oriented multi-objective execution plan optimization system for implementing the method of any one of claims 1 to 6, characterized by The method comprises the following steps: an obtaining module, configured to obtain a logical application graph and context information, wherein the context information comprises a service level objective and load characteristics; an integrating module, configured to integrate the logical application graph, the service level objective, the load characteristics and a pre-built operator performance model to obtain a constraint optimization problem; a solving module, configured to solve the constraint optimization problem by using a multi-objective constraint optimization solver to generate an optimal physical execution plan; an executing module, configured to execute the optimal physical execution plan by using an execution engine. 8.The multi-objective execution plan optimization system for intelligent application oriented application, according to claim 7, wherein, The method further comprises a storage module, configured to store the operator performance model.

9. A multi-objective execution plan optimization apparatus for intelligent applications, characterized by, The method comprises the following steps: a memory, a processor and a transceiver connected in sequence, wherein the memory is configured to store a computer program, the transceiver is configured to receive and send messages, and the processor is configured to read the computer program and execute the method for multi-objective execution plan optimization for intelligent applications according to any one of claims 1 to 6.

10. A computer program product comprising computer programs or instructions, characterized in that, The computer program or the instructions, when executed by a computer, implement the intelligent application-oriented multi-objective execution plan optimization method as claimed in any one of claims 1-6.

Images