Inference task execution method and device and electronic equipment

Abstract

The invention relates to an inference task execution method and apparatus, and an electronic device. The method comprises the steps of obtaining a deep learning model to generate an original calculation graph for an inference task; analyzing the original calculation graph to determine a preliminary analysis result of the original calculation graph; determining a target optimization strategy from preset multi-scale optimization strategies according to the preliminary analysis result; the target optimization strategy is one or combination of multi-scale optimization strategies; adjusting the original calculation graph according to the target optimization strategy to obtain an optimized calculation graph; comparing and displaying the original calculation graph and the optimized calculation graph through a graphical user interface; in response to an optimization adjustment parameter input by a user through the graphical user interface, adjusting the optimization calculation graph according to the optimization adjustment parameter to obtain a target calculation graph; and controlling the deep learning model to execute the reasoning task according to the target calculation graph. According to the optimized computational graph, the execution efficiency and the resource utilization rate can be improved, and therefore various complex scenes can be dealt with.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of artificial intelligence, and particularly relates to a reasoning task execution method and device and electronic equipment. BACKGROUND

[0002] Graph optimization technology is widely used in deep learning model reasoning optimization, compiler optimization, computer vision, natural language processing and other fields. The main goal of graph optimization technology is to optimize the computation graph to reduce computational complexity, memory occupation, improve execution efficiency, and thus accelerate the deep learning model reasoning process.

[0003] However, the current graph optimization technology often focuses on automatic or traditional rule-driven methods, resulting in optimized computation graphs that cannot adapt to various complex scenarios. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a reasoning task execution method, device and electronic equipment to overcome the above problems or at least partially solve the above problems.

[0005] In order to solve the above technical problems, the present application is implemented as follows: A reasoning task execution method, the method comprising: obtaining a deep learning model to generate an original computation graph for a reasoning task; the original computation graph comprising nodes and edges between the nodes, the nodes representing operations, and the edges representing data transfer between operations; analyzing the original computation graph to determine a preliminary analysis result of the original computation graph; determining a target optimization strategy from a plurality of preset multi-scale optimization strategies according to the preliminary analysis result; the target optimization strategy being one or a combination of the multi-scale optimization strategies; adjusting the original computation graph according to the target optimization strategy to obtain an optimized computation graph; comparing and displaying the original computation graph and the optimized computation graph through a graphical user interface; adjusting the optimized computation graph according to optimization adjustment parameters input by a user through the graphical user interface to obtain a target computation graph; controlling the deep learning model to execute the reasoning task according to the target computation graph.

[0006] In the above embodiments, after the system optimizes the original calculation graph according to the multi-scale optimization strategy, the original calculation graph and the optimized calculation graph can be compared and displayed through a graphical user interface. The user can input corresponding optimization adjustment parameters to further iteratively optimize the calculation graph, which can effectively improve the overall optimization effect of the calculation graph. The optimized calculation graph can improve the execution efficiency and resource utilization, thereby coping with various complex scenarios.

[0007] In an embodiment of the present application, the analyzing the original calculation graph to determine the preliminary analysis result of the original calculation graph comprises: extracting metadata of the nodes in the original calculation graph; the metadata at least includes operator type, input / output tensor shape, data type, whether it is a constant, and whether it includes trainable parameters; constructing a predecessor-successor dependency graph according to the original calculation graph, and determining the matchable optimization mode of the predecessor-successor dependency graph; determining the node type of the node according to a pre-defined operator semantic rule library; the node type at least includes basic operator class, activation function class, normalization class, control flow class, and redundancy / prunable class; obtaining characteristic information of a target hardware platform on which the deep learning model is deployed; the characteristic information at least includes memory bandwidth, computing unit architecture, and supported low-precision instruction set; inputting the metadata, the optimization mode, the node type, and the characteristic information into a prediction model to obtain the estimated calculation intensity and potential bottleneck area of the operators in the original calculation graph by the prediction model; determining the core target of the original calculation graph according to the calculation intensity and the potential bottleneck area; taking the core target as the preliminary analysis result of the original calculation graph.

[0008] In the above embodiments, the system performs in-depth analysis on the original calculation graph, so as to input the metadata, optimization mode, node type, and characteristic information in the original calculation graph into a determination prediction model to obtain the preliminary analysis result of the original calculation graph, and determine the corresponding target optimization strategy from the multi-scale optimization strategy. In this way, the optimization strategy can be flexibly adjusted according to different calculation graph structures and hardware requirements, so as to ensure that the optimized calculation graph can achieve optimal performance in various scenarios, including complex scenarios.

[0009] In an embodiment of the present application, the determining the target optimization strategy from the pre-defined multi-scale optimization strategy according to the preliminary analysis result comprises: In response to an optimization target input by a user through the graphical user interface, a target optimization strategy is determined from a plurality of preset multi-scale optimization strategies according to the preliminary analysis result and the optimization target.

[0010] In the above embodiment, the preliminary analysis result and the optimization target set by the user are combined to determine the target optimization strategy, so that the calculation graph optimized according to the target optimization strategy can not only improve the execution efficiency and resource utilization to cope with various complex scenarios, but also better meet the user's needs and overcome the optimization bottleneck that the system cannot achieve due to its own limitations.

[0011] In an embodiment of the present application, the comparison and display of the original calculation graph and the optimized calculation graph through the graphical user interface comprises: The topological structure of the original calculation graph and the optimized calculation graph is compared and displayed through the graphical user interface, wherein the topological structure of the optimized calculation graph is marked with a preset visual element to identify the nodes that are modified, merged or deleted relative to the original calculation graph. The change log is displayed through the graphical user interface, and the change log at least includes the operation description of the optimization operation performed on the original calculation graph and the performance impact estimation of the optimization operation.

[0012] In the above embodiment, the system can compare and display the topological structure of the original calculation graph and the optimized calculation graph through the graphical user interface, as well as the change log of the optimized calculation graph, etc., so that the user can more intuitively understand the changes of the optimized calculation graph compared with the original calculation graph, and then the user can adaptively and quickly determine the optimization adjustment parameters for the optimized calculation graph according to the user's own needs, so that the target calculation graph can be further adjusted according to the optimization adjustment parameters, improving the optimization efficiency of the target calculation graph.

[0013] In an embodiment of the present application, the comparison and display of the original calculation graph and the optimized calculation graph through the graphical user interface further comprises: When the optimization operation is identified as having risks, a warning prompt corresponding to the optimization operation is provided in the graphical user interface.

[0014] In the above embodiment, when the optimization operation for the calculation graph is identified as having risks, a warning prompt corresponding to the optimization operation is provided in the graphical user interface to assist the user to make a corresponding decision quickly, improving the optimization efficiency of the target calculation graph.

[0015] In an embodiment of the present application, after the target calculation graph is obtained by adjusting the optimized calculation graph according to the optimization adjustment parameters input by the user through the graphical user interface, the method further comprises: The target computation graph is evaluated according to preset evaluation criteria to obtain an evaluation result; the evaluation criteria at least include execution efficiency, resource consumption, and delay time; The evaluation result is displayed through the graphical user interface; In response to an iterative optimization adjustment parameter input by a user through the graphical user interface, a new target computation graph is obtained by iteratively adjusting the optimization computation graph of the last round according to the iterative optimization adjustment parameter.

[0016] In the above embodiments, based on the real-time evaluation result and the iterative optimization adjustment parameter fed back by the user, the system can iteratively optimize the computation graph, and continuously optimize the computation graph through a closed-loop feedback mechanism, to ensure that the target computation graph with the best optimization effect under different conditions is obtained.

[0017] In an embodiment of the present application, the control of the deep learning model to perform the inference task according to the target computation graph comprises: Converting the target computation graph into a format suitable for a target hardware platform; Deploying the converted target computation graph to the target hardware platform, so that the deep learning model deployed in the target hardware platform performs the inference task according to the target computation graph.

[0018] In the above embodiments, by converting the target computation graph into a format suitable for the target hardware platform, the target computation graph can be directly deployed to the target hardware platform, and then the deep learning model of the target hardware platform can quickly perform the inference task according to the target computation graph.

[0019] An inference task execution apparatus, the apparatus comprising: An original computation graph generation module configured to obtain an original computation graph for a deep learning model performing an inference task; the original computation graph comprises nodes and edges between the nodes, the nodes are configured to represent operations, and the edges are configured to represent data transmission between operations; A preliminary analysis result determination module configured to analyze the original computation graph to determine a preliminary analysis result of the original computation graph; An optimization strategy determination module configured to determine a target optimization strategy from a plurality of preset multi-scale optimization strategies according to the preliminary analysis result; the target optimization strategy is one or a combination of the multi-scale optimization strategies; A computation graph optimization module configured to adjust the original computation graph according to the target optimization strategy to obtain an optimized computation graph; A comparison and display module configured to compare and display the original computation graph and the optimized computation graph through a graphical user interface; a target computation graph determination module, configured to adjust the optimization computation graph to obtain a target computation graph according to an optimization adjustment parameter input by a user through the graphical user interface in response to the optimization adjustment parameter; a reasoning task execution module, configured to control the deep learning model to execute the reasoning task according to the target computation graph.

[0020] An electronic device, comprising: a processor; a memory for storing processor-executable instructions; The processor is configured to execute the instructions to implement the reasoning task execution method described above.

[0021] A computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal can execute the reasoning task execution method described above.

[0022] The embodiments of the present application have at least the following beneficial effects: In the embodiments of the present application, the system obtains a deep learning model to generate an original computation graph for a reasoning task; the original computation graph includes nodes and edges between the nodes, the nodes are used to represent operations, and the edges are used to represent data transmission between operations; the original computation graph is analyzed to determine a preliminary analysis result of the original computation graph, to determine a target optimization strategy from a plurality of preset optimization strategies, wherein the target optimization strategy is one or a combination of the plurality of optimization strategies; then, the original computation graph can be adjusted according to the target optimization strategy to obtain an optimization computation graph, the original computation graph and the optimization computation graph are compared and displayed through a graphical user interface, in response to an optimization adjustment parameter input by a user through the graphical user interface, the optimization computation graph is adjusted according to the optimization adjustment parameter to obtain a target computation graph, and finally, the deep learning model can be controlled to execute the reasoning task according to the target computation graph. In the embodiments of the present application, after the original computation graph is optimized according to the plurality of optimization strategies, the original computation graph and the optimization computation graph can be compared and displayed through the graphical user interface, the user can input corresponding optimization adjustment parameters to further iteratively optimize the computation graph, which can effectively improve the overall optimization effect of the computation graph, and the optimized computation graph can improve the execution efficiency and resource utilization, thereby coping with various complex scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 A step flowchart of a reasoning task execution method provided in the embodiments of the present application; Figure 2 A multi-scale interactive computation graph optimization flowchart provided in the embodiments of the present application; Figure 3 A structural schematic diagram of a reasoning task execution apparatus provided in the embodiments of the present application; Figure 4A structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] Other advantages and benefits of the present application will become apparent to those skilled in the art upon consideration of the disclosure or can be learned by practice of the application. The advantages and benefits of the present application can be realized and attained by practical performance or application of various embodiments of the method and techniques disclosed in this specification. Various modifications can be made to the embodiments of the present application in terms of the details of the specification without departing from the spirit of the application. It is to be understood that the preferred embodiments are only for illustrating the present application and are not intended to limit the scope of protection of the present application.

[0025] It should be noted that the embodiments of the present application can involve the use of user data. In actual applications, user-specific personal data can be used in the schemes described herein in a manner that complies with applicable laws and regulations of the country (for example, with the explicit consent of the user, with the actual notification of the user, etc.) and within the scope permitted by applicable laws and regulations.

[0026] At present, related graph optimization methods include constant folding, redundant node elimination, operator fusion, operator replacement, and operator front movement. Specifically, constant folding is to calculate constant expressions in the graph in advance during the construction phase of the computation graph, thereby reducing the amount of computation at runtime; redundant node elimination is to delete nodes and operations in the computation graph that do not affect the final output; operator fusion is to fuse multiple consecutive operations into a composite operation, thereby reducing the number of nodes and execution time of the computation graph; operator replacement is to replace an original operator with a more efficient operator in the computation graph; and operator front movement is to move an operator to the front if the input of the operator is independent of other parts of the program, thereby reducing the computational burden at runtime.

[0027] Related graph optimization methods aim to improve computation performance and reduce memory occupancy by performing static or semi-static analysis and modification of the computation graph. These graph optimization methods are mostly based on classical compilation optimization techniques and usually simplify, adjust, and optimize the computation graph through certain rules or heuristic algorithms. However, the optimization range of these methods is limited, and they cannot handle complex graph structures, have large hardware adaptation and complex dependency problems, and are difficult to achieve global optimization.

[0028] With the continuous complication of deep neural network models and the diversification of hardware computing power, traditional manual graph optimization methods gradually cannot meet the needs of efficient inference. Therefore, deep learning driven graph optimization methods emerge as the times require. Generally, it can include automatic differentiation, reinforcement learning, neural architecture search (NAS, Neural Architecture Search) and other technologies. Through dynamically adjusting the computation graph structure, it can optimize for specific hardware platforms or inference tasks, improve inference speed, reduce memory occupation and improve resource utilization efficiency. Specifically, automatic differentiation technology is to calculate the gradient in the training process and go through the back propagation algorithm to optimize the parameters, operations and structures in the graph; neural architecture search is a deep learning based method that explores different network topology structures, connection methods, activation functions and other factors to find the optimal network architecture or computation graph structure; the idea of reinforcement learning for graph optimization is to learn an optimization strategy through interaction with the environment to gradually improve the computation graph. Deep learning based graph optimization methods have strong compatibility and adaptability, but at the same time, they also bring additional computational overhead. Training a reinforcement deep learning model requires a large number of sample interactions and feedback, and faces the problems of stability and convergence.

[0029] Current graph optimization methods often focus on automatic or traditional rule driven methods, but generally have the problem of insufficient interactivity. Most optimization methods are black box operations for users, lack intuitive feedback mechanisms, and the transparency and operability of the optimization process are poor, making it difficult for users to accurately control or adjust the optimization strategy for the computation graph. In addition, current graph optimization methods often apply a single method, lack of combination and collaborative optimization of multiple methods, and a single method often cannot cope with the challenges of various complex scenarios.

[0030] To solve the above problems, the embodiment of the present application provides a method for executing an inference task. The method specifically optimizes the computation graph based on a multi-scale optimization strategy, and the user can also observe the optimized computation graph through a graphical user interface, and then input the corresponding computation graph for further iterative optimization, which can effectively improve the overall optimization effect of the computation graph. In this way, the deep learning model can execute the inference task through the multi-scale optimization strategy and the user-optimized computation graph. The embodiment of the present application can cope with performance bottlenecks in complex computation graphs by combining graph optimization methods and deep learning driven adaptive optimization strategies, improve execution efficiency and resource utilization, and thus cope with various complex scenarios.

[0031] Reference Figure 1 , a step flowchart of a method for executing an inference task provided in an embodiment of the present application is shown, specifically including the following steps: In step 101, a deep learning model generates an original computation graph for an inference task; the original computation graph includes nodes and edges between the nodes, the nodes are used to represent operations, and the edges are used to represent data transmission between operations.

[0032] The computation graph is a graph structure composed of nodes (representing computing operations) and edges (representing data flow and dependency relationships), commonly used to represent the computing process of a deep learning model. Each node represents an operation (such as matrix multiplication, convolution, activation, etc.), and the edge represents data transmission between operations, indicating data flow and dependency relationships, i.e., computing order and data transmission path. In practical applications, the basic execution unit of the computation graph is an operator.

[0033] The method of the embodiments of the present application can be applied to a system, specifically, the system can refer to a software platform or tool capable of performing computation graph analysis, multi-scale optimization, graphical interaction, and evaluation functions, etc. The embodiments of the present application do not need to be limited to this, in the embodiments of the present application, the system can receive an original computation graph (computation graph) generated by a deep learning model of a deep learning framework (such as TensorFlow, PyTorch, ONNX, etc.), and then optimize the original computation graph, so that the deep learning model can complete the inference task through the optimized computation graph, improving the execution efficiency and resource utilization of the deep learning model.

[0034] In step 102, the original computation graph is analyzed to determine a preliminary analysis result of the original computation graph.

[0035] In the embodiments of the present application, the system first converts the input original computation graph into a unified intermediate representation (Intermediate Representation, IR), for example, using an MLIR (Multi-Level Intermediate Representation) style graph structure, and performs a deep analysis on the topology structure of the converted computation graph, including structure analysis, operator type identification, and dependency relationship evaluation, etc. of the computation graph, to determine the optimization target and potential optimization space of the computation graph, thereby obtaining the preliminary analysis result of the original computation graph.

[0036] In step 103, a target optimization strategy is determined from a plurality of preset multi-scale optimization strategies according to the preliminary analysis result; the target optimization strategy is one or a combination of the multi-scale optimization strategies.

[0037] In a specific implementation, a plurality of multi-scale optimization strategies of different levels can be pre-deployed in the system. For example, the multi-scale optimization strategies can include operator-level optimization, module-level optimization, global optimization, and deep learning driven optimization, etc. These multi-scale optimization strategies can be applied to the computation graph optimization individually or in combination. The embodiments of the present application do not need to be limited in this regard.

[0038] Specifically, the operator-level optimization can include operator fusion, constant folding, subgraph elimination, etc. The module-level optimization uses graph pruning to remove parts of the computation graph that have no effect on the final result. Graph pruning is an optimization technique that removes parts of the computation graph that have no effect on the final inference result (such as redundant operators or nodes), thereby reducing the computational and storage burden. The global optimization rearranges the execution order of operators to minimize memory access conflicts and data dependency bottlenecks. The deep learning driven optimization uses neural architecture search to automatically generate the optimal subgraph structure by selecting architectures in the search space, in order to improve computational efficiency and performance.

[0039] In the embodiments of the present application, the system can determine a target optimization strategy from the multi-scale optimization strategies according to the preliminary analysis result of the original computation graph. The target optimization strategy can be one or a combination of the multi-scale optimization strategies. For example, it can be one multi-scale optimization strategy such as global optimization, or a combination of two multi-scale optimization strategies such as operator-level optimization and module-level optimization. The embodiments of the present application intelligently select the target optimization strategy by comprehensively using a plurality of multi-scale optimization strategies such as operator-level optimization, module-level optimization, global optimization, and deep learning driven optimization (such as neural architecture search and reinforcement learning), in order to flexibly adjust the execution order of operators in the original computation graph through the target optimization strategy, thereby effectively improving the overall optimization effect of the computation graph.

[0040] In step 104, an optimized computation graph is obtained by adjusting the original computation graph according to the target optimization strategy.

[0041] In the embodiments of the present application, after the target optimization strategy is determined, the original computation graph can be adaptively adjusted according to the target optimization strategy to obtain the optimized optimized computation graph. For example, assuming that the target optimization strategy is operator fusion in operator-level optimization, a plurality of consecutive operations in the original computation graph can be fused into a composite operation, such as a composite operation of convolution, normalization, and activation. The nodes and edges corresponding to the convolution, normalization, and activation operations in the original computation graph can be deleted, thereby obtaining the optimized computation graph with reduced number of nodes and execution time of the computation graph. The original computation graph requires three kernel launches and twice intermediate result reading and writing, while the optimized computation graph only requires one kernel call and one data read and write. It can be seen that the optimized computation graph reduces the computation delay and memory access overhead, and improves the execution efficiency and resource utilization.

[0042] At step 105, the original computing graph and the optimized computing graph are compared and displayed through a graphical user interface.

[0043] In the embodiments of the present application, after obtaining the optimized computing graph, the system can provide a graphical user interface for the user. In this way, the user can interact with the system in real time through the graphical user interface. Specifically, the system can compare and display the original computing graph and the optimized computing graph through the graphical user interface. In this way, the user can intuitively observe the differences between the original computing graph and the optimized computing graph, and then determine whether the optimized computing graph is the computing graph needed by the user.

[0044] It should be noted that the specific content of comparing and displaying the original computing graph and the optimized computing graph through the graphical user interface can be pre-set by the system or set by the user according to the user's own needs. The embodiments of the present application do not need to be limited in this regard.

[0045] At step 106, in response to the optimization adjustment parameters input by the user through the graphical user interface, the optimization adjustment parameters are used to adjust the optimized computing graph to obtain a target computing graph.

[0046] In the embodiments of the present application, the user determines the places that need to be further optimized in the optimized computing graph by observing the original computing graph and the optimized computing graph compared and displayed through the graphical user interface. For example, the user can further merge multiple operations or delete a redundant operation. The user can input optimization adjustment parameters through configurable optimization options provided by the graphical user interface. After obtaining the optimization adjustment parameters, the system can further adjust the optimized computing graph according to the optimization adjustment parameters to obtain a target computing graph. The optimized target computing graph can improve the execution speed, reduce the memory occupation, and reduce the delay of the inference task, thereby meeting the requirements of efficient computing and low-delay inference.

[0047] The user of the embodiments of the present application can intuitively view the changes of the computing graph before and after optimization through the graphical user interface, adjust the optimization mode in real time, and dynamically adjust optimization parameters such as pruning ratio and operator fusion threshold, thereby enhancing the user's control and feedback on the optimization process and improving the transparency and operation flexibility of the optimization process.

[0048] It should be noted that the optimization options configurable by the user in the graphical user interface can be specific instantiations of four large-scale optimization strategies, namely operator-level optimization, module-level optimization, global optimization, and deep learning-driven optimization. In addition, the optimization options support fine-grained combination and parameter tuning, thereby realizing precise intervention and personalized control of the optimization process.

[0049] At step 106, the deep learning model is controlled to execute the inference task according to the target computing graph.

[0050] In the embodiments of the present application, the system can control the deep learning model to perform an inference task according to the target computation graph adjusted by the multiple optimizations after obtaining the target computation graph, where the inference task can be a task of image classification, a task of speech recognition, a task of target detection of an image, etc.

[0051] In the above embodiments, the system can display the original computation graph and the optimized computation graph through a graphical user interface after optimizing the original computation graph according to the multi-scale optimization strategy, the user can input corresponding optimization adjustment parameters to further iteratively optimize the computation graph, which can effectively improve the overall optimization effect of the computation graph, and the optimized computation graph can improve the execution efficiency and resource utilization, thereby coping with various complex scenarios.

[0052] In an embodiment of the present application, the analyzing the original computation graph to determine the preliminary analysis result of the original computation graph can include: extracting metadata of the nodes in the original computation graph; the metadata at least includes operator type, input / output tensor shape, data type, whether it is a constant, and whether it includes trainable parameters; constructing a predecessor-successor dependency graph according to the original computation graph, and determining an optimizable mode matched by the predecessor-successor dependency graph; determining a node type of the node according to a pre-defined operator semantic rule library; the node type at least includes a basic operator class, an activation function class, a normalization class, a control flow class, and a redundancy / prunable class; obtaining characteristic information of a target hardware platform on which the deep learning model is deployed; the characteristic information at least includes memory bandwidth, computing unit architecture, and supported low-precision instruction set; inputting the metadata, the optimizable mode, the node type, and the characteristic information into a prediction model to obtain a computation intensity and a potential bottleneck area of an operator in the original computation graph estimated by the prediction model; determining a core target of the original computation graph according to the computation intensity and the potential bottleneck area; taking the core target as the preliminary analysis result of the original computation graph.

[0053] In the embodiments of the present application, the preliminary deep analysis process of the original computation graph can include: extracting the metadata of each node in the original computation graph (such as operator type, input / output tensor shape, data type, whether it is a constant or contains trainable parameters). A complete predecessor-successor dependency graph is constructed according to the original computation graph and the metadata, and a subgraph matching algorithm is used to identify common optimization patterns (such as Conv (convolution), BatchNorm (normalization), ReLU (activation) which can be fused into one operation) based on the predecessor-successor dependency graph. Based on the pre-defined operator semantic rule library, the system classifies the nodes in the original computation graph into basic operator class, activation function class, normalization class, control flow class, and redundant / prunable class, etc. The characteristics of the target hardware platform on which the deep learning model is deployed are obtained, such as memory bandwidth, computing unit architecture, supported low-precision instruction set, etc.

[0054] Subsequently, the system can determine the computing intensity and potential bottleneck area of each operator in the original computation graph according to the metadata, optimization patterns, node types and characteristic information through a prediction model such as a static performance model (such as Roofline model) or a lightweight delay predictor, thereby determining the core target of the initial optimization of the original computation graph, so that the core target can be taken as the preliminary analysis result of the original computation graph, wherein the preliminary analysis result can include graph complexity indicators, operator distribution characteristics, critical path composition, resource hotspot markers and hardware adaptation suggestions, etc., and then the target optimization strategy is determined from the multi-scale optimization strategies according to the preliminary analysis result, which is used to optimize the original computation graph. Exemplarily, the core target can be to shorten the critical path length in delay-sensitive tasks, to reduce the peak memory occupancy in memory-limited scenarios, or to enable quantization and sparsification-friendly strategies in energy efficiency-oriented deployment.

[0055] The system determines the target optimization strategy from the multiple scale optimization strategies intelligently according to the preliminary analysis result, such as the figure complexity index (such as the total number of nodes, the maximum depth), the operator distribution characteristics (such as the convolution intensive type or the attention dominant type), the key path composition, the resource hotspot marking (such as the high memory consumption node) and the hardware adaptation suggestion (such as whether to support Tensor Core or NPU special operator), through the built-in hierarchical strategy scheduler. For example, if a large number of continuous Element-wise (element-wise operation) operations are detected, operator fusion (operator-level optimization) is automatically enabled; if there is a subgraph that is not used by the downstream, module-level pruning (module-level optimization) is triggered; for the small operators scattered on the key path, global rearrangement and fusion are preferentially performed to reduce the kernel launch overhead (global optimization); for the model with obvious structural redundancy, light-weight NAS (neural architecture search) fine-tunes the local subgraph (deep learning driven optimization).

[0056] In the above embodiment, the system performs depth analysis on the original computation graph, so that the preliminary analysis result of the original computation graph is obtained by inputting the metadata, the optimizable mode, the node type and the characteristic information in the original computation graph into the determination prediction model, and the corresponding target optimization strategy is determined from the multiple scale optimization strategies. In this way, the optimization strategy can be flexibly adjusted according to different computation graph structures and hardware requirements, so that the optimized computation graph can achieve optimal performance in various scenarios (including complex scenarios).

[0057] In an embodiment of the present application, the determining the target optimization strategy from the multiple scale optimization strategies according to the preliminary analysis result can include: In response to the optimization target input by the user through the graphical user interface, the target optimization strategy is determined from the multiple scale optimization strategies according to the preliminary analysis result and the optimization target.

[0058] In the embodiment of the present application, the system allows the user to decide the target optimization strategy. Specifically, the system adopts a weighted scoring mechanism to dynamically adjust the priority and execution order of the multiple scale optimization strategies according to the optimization target set by the user (such as minimizing the end-to-end delay or the peak memory), so as to realize the synergistic effect of the multiple scale optimization strategies. In this way, the target optimization strategy is determined by combining the preliminary analysis result and the optimization target set by the user through the graphical user interface.

[0059] In the above embodiments, by combining the preliminary analysis result and the optimization target set by the user to determine the target optimization strategy, thus, the calculation graph optimized according to the target optimization strategy can not only improve the execution efficiency and resource utilization to cope with various complex scenarios, but also better meet the user demand and overcome the optimization bottleneck that the system cannot achieve due to its own limitations.

[0060] In an embodiment of the present application, the comparison and display of the original calculation graph and the optimized calculation graph through the graphical user interface can include: comparing and displaying the topologies of the original calculation graph and the optimized calculation graph through the graphical user interface, wherein the topology of the optimized calculation graph identifies the nodes that are modified, fused or deleted relative to the original calculation graph through preset visual elements; displaying a change log through the graphical user interface, wherein the change log at least includes the operation description of the optimization operation performed on the original calculation graph and the performance impact estimation of the optimization operation.

[0061] In the embodiments of the present application, the user interacts with the system in real time through the graphical user interface, specifically, the original calculation graph and the optimized calculation graph are visually displayed through the graphical user interface, and detailed feedback information is provided for each optimization operation, thus, the user can input the corresponding optimization adjustment parameters through the graphical user interface to iteratively optimize the optimized calculation graph according to the user's own needs.

[0062] Specifically, the graphical user interface synchronously displays the topologies of the original calculation graph and the optimized calculation graph in a visual manner, and can also identify the nodes that are modified, fused or deleted in the optimized calculation graph compared with the original calculation graph through highlighting and other visual elements.

[0063] In addition, the graphical user interface can provide a structured change log, wherein the change log can include the operation description of the optimization operation performed on the original calculation graph and the performance impact estimation of each optimization operation during optimization. For example, the operation description of the optimization operation performed on the original calculation graph can be "fused Conv2D and BatchNorm nodes into FusedConvBN", and the performance impact estimation of each optimization operation during optimization can be the change of node number, the estimated delay reduction value, the memory peak drop amount, etc.

[0064] The user can intuitively view the topologies of the original computation graph and the optimized computation graph through the graphical user interface, and can intuitively compare the differences between the computation graphs before and after optimization, and dynamically adjust the optimization behavior based on the feedback information, for example, adjust the pruning ratio (0%~30%) through a slider, set the minimum continuous length threshold for operator fusion, enable or disable a specific optimization category (such as temporarily closing the NAS search to verify its necessity), or reorder the strategy execution priority, etc. In some embodiments, all optimization operations support one-key undo or multi-version saving, ensuring the safety and flexibility of the exploration process.

[0065] In the above embodiments, the system can display the topologies of the original computation graph and the optimized computation graph through the graphical user interface, and display the change log of the optimized computation graph, etc. In this way, the user can more intuitively understand the changes of the optimized computation graph compared with the original computation graph, and then the user can adaptively and quickly determine the optimization adjustment parameters for the optimized computation graph according to the user's own needs, so that the target computation graph can be further adjusted according to the optimization adjustment parameters, thereby improving the optimization efficiency of the target computation graph.

[0066] In an embodiment of the present application, the displaying the topologies of the original computation graph and the optimized computation graph through the graphical user interface can further include: When the optimization operation is identified as having a risk, a warning prompt corresponding to the optimization operation is provided in the graphical user interface.

[0067] In the embodiments of the present application, if the system identifies that a certain optimization operation for the computation graph (which can be for the original computation graph or for the optimized computation graph) has a risk, for example, if a certain optimization operation for the original computation graph may introduce a numerical precision risk (such as INT8 quantization or aggressive pruning), which will cause the accuracy of the deep learning model to decrease when performing an inference task based on the final target computation graph, the system will actively mark on the graphical user interface to provide a warning prompt to remind the user whether to perform the optimization operation to generate the final target computation graph.

[0068] In the above embodiments, when the optimization operation for the computation graph is identified as having a risk, a warning prompt corresponding to the optimization operation is provided in the graphical user interface to assist the user to make a corresponding decision quickly, thereby improving the optimization efficiency of the target computation graph.

[0069] In an embodiment of the present application, after the target computation graph is obtained by adjusting the optimized computation graph according to the optimization adjustment parameters input by the user through the graphical user interface, the method can further include: The target computation graph is evaluated according to preset evaluation criteria to obtain an evaluation result; the evaluation criteria at least include execution efficiency, resource consumption and delay time; The evaluation result is displayed through the graphical user interface; In response to an iterative optimization adjustment parameter input by a user through the graphical user interface, a new target computation graph is obtained by iteratively adjusting the optimization computation graph of the last round according to the iterative optimization adjustment parameter.

[0070] In the embodiments of the present application, the optimized optimization computation graph is evaluated by the built-in evaluation module in the system according to the preset evaluation criteria to obtain an evaluation result, wherein the evaluation criteria can include: execution efficiency (measuring the execution speed of the optimization computation graph, evaluating its computing ability), resource consumption (evaluating the memory usage, storage demand and bandwidth consumption of the optimization computation graph, etc.), delay time (evaluating the time delay of the inference task, including the time delay from input to output), and then the system gives feedback according to the evaluation result to help the user understand the optimization effect.

[0071] If the user thinks that the optimization effect of the optimization computation graph is not ideal, the iterative optimization adjustment parameter input through the graphical user interface can be based on the evaluation result, the system will obtain a new target computation graph by iteratively adjusting the optimization computation graph of the last round according to the iterative optimization adjustment parameter, and the new target computation graph can continue to be evaluated according to the preset evaluation criteria to obtain an evaluation result. In the optimization process of the optimization computation graph, the user can participate in the iterative optimization repeatedly to form a closed-loop feedback, and the optimization adjustment parameter is adjusted continuously until the target computation graph with the best optimization effect is obtained.

[0072] In the above embodiments, based on the real-time evaluation result and the iterative optimization adjustment parameter of the user feedback, the system can adaptively iteratively optimize the computation graph, and continuously optimize the computation graph through the closed-loop feedback mechanism to ensure that the target computation graph with the best optimization effect is obtained under different conditions.

[0073] In an embodiment of the present application, the control of the deep learning model to perform the inference task according to the target computation graph can include: Converting the target computation graph into a format suitable for the target hardware platform; Deploying the converted target computation graph to the target hardware platform, so that the deep learning model deployed in the target hardware platform performs the inference task according to the target computation graph.

[0074] In the embodiments of the present application, after obtaining the target computation graph through iterative optimization, the target computation graph can be provided as an output result to the user. The system supports outputting the target computation graph in a format suitable for deployment of the target hardware platform (such as ONNX, TensorFlow Lite, etc.), so that the converted target computation graph can be directly deployed to the target hardware platform, so that the deep learning model of the target hardware platform performs inference tasks according to the target computation graph. In the above embodiments, by converting the target computation graph into a format suitable for the target hardware platform, the target computation graph can be directly deployed to the target hardware platform, and then the deep learning model of the target hardware platform can quickly perform inference tasks according to the target computation graph.

[0075] In order for those skilled in the art to better understand the embodiments of the present application, the following specific examples are used for illustration, and the embodiments of the present application are described with reference to the accompanying drawings Figure 2 A multi-scale interactive computation graph optimization flowchart is provided in the embodiments of the present application, and the specific process can include: Receiving an input computation graph: the system first receives an input computation graph (original computation graph); Computation graph analysis: preliminary analysis is performed on the computation graph, including structure analysis of the computation graph, operator type identification, dependency relationship evaluation, etc., to determine the optimization target and potential optimization space. According to the structure of the computation graph and the optimization target, the system selects a suitable multi-scale optimization strategy, which can include operator-level optimization, module-level optimization, global optimization, deep learning-driven optimization, etc. On the basis of multi-scale optimization, the system provides a graphical user interface for the user to make real-time adjustments. The user can view the optimized computation graph and manually modify the optimization strategy or input optimization adjustment parameters as needed. The user can dynamically select different multi-scale optimization strategies, optimization adjustment parameters, enable or disable specific optimization operations, and obtain clear feedback during the optimization process.

[0076] Optimization result evaluation: the optimized computation graph is evaluated by the built-in evaluation module. The evaluation content includes the execution efficiency, resource consumption, and delay time of the computation graph. Among them, the user can further adjust the optimization strategy according to the evaluation result to form a closed-loop feedback mechanism and continuously improve the optimization process.

[0077] Computation graph output: finally, the system outputs the optimized computation graph and supports deploying the optimized computation graph to the target hardware platform for inference tasks.

[0078] It can be seen that the embodiments of the present application have at least the following effects: 1. Users can intuitively view the changes of the computation graph before and after optimization through the graphical user interface, adjust the optimization mode in real time, and improve the transparency and operation flexibility of the optimization process. 2. By combining multiple multi-scale optimization strategies (including deep learning driven optimization), the optimization mode can be flexibly adjusted according to different computation graph structures and hardware requirements, thereby ensuring that the computation graph can achieve optimal performance in various scenarios. 3. The optimized computation graph improves the execution speed, reduces the memory occupancy, and reduces the delay of the inference task, meeting the requirements of efficient computation and low-delay inference. 4. Based on real-time effect evaluation and user feedback, the system can adaptively adjust the optimization mode for the computation graph and continuously optimize the computation graph through a closed-loop feedback mechanism to ensure the best effect under different conditions.

[0079] It should be noted that for the method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited by the order of the described actions, because according to the embodiments of the present application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present application.

[0080] Referring to Figure 3 , a structural block diagram of a reasoning task execution device provided in an embodiment of the present application is shown, which includes the following modules: An original computation graph generation module 301 is configured to obtain an original computation graph for a reasoning task generated by a deep learning model; the original computation graph includes nodes and edges between the nodes, the nodes are used to represent operations, and the edges are used to represent data transmission between operations; A preliminary analysis result determination module 302 is configured to analyze the original computation graph to determine a preliminary analysis result of the original computation graph; An optimization strategy determination module 303 is configured to determine a target optimization strategy from a plurality of preset multi-scale optimization strategies according to the preliminary analysis result; the target optimization strategy is one or a combination of the multi-scale optimization strategies; A computation graph optimization module 304 is configured to adjust the original computation graph to obtain an optimized computation graph according to the target optimization strategy; A comparison and display module 305 is configured to compare and display the original computation graph and the optimized computation graph through a graphical user interface; A target computation graph determination module 306 is configured to adjust the optimized computation graph to obtain a target computation graph according to optimization adjustment parameters input by a user through the graphical user interface; The inference task execution module 307 is configured to control the deep learning model to execute the inference task according to the target computation graph.

[0081] In an embodiment of the present application, the preliminary analysis result determination module 302 is configured to: extract metadata of the nodes in the original computation graph; the metadata at least includes operator type, input / output tensor shape, data type, whether it is a constant, and whether it includes trainable parameters; construct a predecessor-successor dependency graph according to the original computation graph, and determine an optimizable mode matched by the predecessor-successor dependency graph; determine the node type of the node according to a pre-defined operator semantic rule library; the node type at least includes basic operator class, activation function class, normalization class, control flow class, and redundancy / prunable class; obtain characteristic information of a target hardware platform on which the deep learning model is deployed; the characteristic information at least includes memory bandwidth, computing unit architecture, and supported low-precision instruction set; input the metadata, the optimizable mode, the node type, and the characteristic information into a prediction model to obtain the computation intensity and potential bottleneck area of the operators in the original computation graph estimated by the prediction model; determine the core target of the original computation graph according to the computation intensity and the potential bottleneck area; use the core target as the preliminary analysis result of the original computation graph.

[0082] In an embodiment of the present application, the optimization strategy determination module 303 is configured to: determine a target optimization strategy from a plurality of pre-set multi-scale optimization strategies according to the preliminary analysis result and an optimization target input by a user through the graphical user interface.

[0083] In an embodiment of the present application, the comparison and display module 305 is configured to: compare and display the topology structures of the original computation graph and the optimized computation graph through a graphical user interface, wherein the topology structure of the optimized computation graph is marked with a pre-set visual element to identify the nodes that are modified, fused, or deleted relative to the original computation graph; display a change log through the graphical user interface, the change log at least including operation description of an optimization operation performed on the original computation graph and performance impact estimation of the optimization operation.

[0084] In an embodiment of the present application, the comparison and display module 305 is configured to: When the optimization operation is identified as being at risk, a warning prompt corresponding to the optimization operation is provided in the graphical user interface.

[0085] In an embodiment of the present application, the apparatus further comprises an evaluation module configured to: evaluate the target computation graph according to preset evaluation criteria to obtain an evaluation result, wherein the evaluation criteria at least include execution efficiency, resource consumption, and delay time; display the evaluation result through the graphical user interface; in response to an iterative optimization adjustment parameter input by a user through the graphical user interface, iteratively adjust the optimization computation graph of the last round according to the iterative optimization adjustment parameter to obtain a new target computation graph.

[0086] In an embodiment of the present application, the inference task execution module 307 is configured to: convert the target computation graph into a format suitable for a target hardware platform; deploy the converted target computation graph to the target hardware platform, so that the deep learning model deployed in the target hardware platform performs the inference task according to the target computation graph.

[0087] In an embodiment of the present application, the system obtains a deep learning model to generate an original computation graph for an inference task; the original computation graph includes nodes and edges between the nodes, the nodes are used to represent operations, and the edges are used to represent data transmission between operations; the original computation graph is analyzed to determine a preliminary analysis result of the original computation graph, so as to determine a target optimization strategy from preset multi-scale optimization strategies, wherein the target optimization strategy is one or a combination of the multi-scale optimization strategies; then, the original computation graph can be adjusted according to the target optimization strategy to obtain an optimized computation graph; the original computation graph and the optimized computation graph are compared and displayed through a graphical user interface; in response to an optimization adjustment parameter input by a user through the graphical user interface, the optimized computation graph is adjusted according to the optimization adjustment parameter to obtain a target computation graph; finally, the deep learning model can be controlled to perform the inference task according to the target computation graph. In an embodiment of the present application, after the original computation graph is optimized according to the multi-scale optimization strategies, the original computation graph and the optimized computation graph can be compared and displayed through the graphical user interface; the user can input corresponding optimization adjustment parameters to further iteratively optimize the computation graph, which can effectively improve the overall optimization effect of the computation graph; the optimized computation graph can improve execution efficiency and resource utilization, thereby coping with various complex scenarios.

[0088] For the apparatus embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the related parts are referred to the part of the method embodiment. The embodiments of the present application further provide an electronic device, such as a server, a personal computer, a mobile phone, a tablet computer, a wearable device, or the like. The embodiments of the present application further provide a computer-readable storage medium, which stores a computer program product.Figure 4 As shown, the device includes a processor 1001, a device interface 1002, a memory 1003 and a bus 1004; The memory 1003 is configured to store a computer program. The processor 1001 is configured to execute the program stored in the memory 1003 to implement the above steps.

[0089] The bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0090] The memory can include a Random Access Memory (RAM) and can also include a non-volatile memory, for example, at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0091] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; can also be a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0092] The application further provides a storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the inference task execution method of the foregoing embodiments.

[0093] The algorithms and displays presented herein are not inherently related to any particular computer, virtual apparatus, or other apparatus. Various aspects of the present disclosure are constructed and arranged to implement the techniques described herein. Additionally, the various aspects presented are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages can be used to implement the teachings of the present disclosure as described herein, and any references below to specific languages are provided for disclosure of enablement of the present disclosure.

[0094] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been described in detail in order not to obscure the understanding of this description.

[0095] Similarly, it is to be understood that the mechanical details of the various features of the application that are described in the exemplary embodiments of the application above are sometimes grouped together in a single embodiment, figure, or description of related features, for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various aspects of the application. This method of disclosure, however, is not to be interpreted as reflecting an intention that the application requires more features than are explicitly recited in each claim. Rather, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Following, the claims are in such form as is accepted by the patent office. Thus, this application includes all modifications and variations of the features taught by the exemplary embodiments discussed above as well as other available options and features that are not specifically disclosed above. The definitions, as typically interpreted by the patent office, are meant to be attached to the claims as filed.

[0096] Those skilled in the art will appreciate that the modules in the apparatuses of the embodiments can be adapted and placed in one or more apparatuses other than the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and further can be divided into more sub-modules or sub-units or sub-components. Any combination of all the features disclosed in the specification (including the accompanying claims, abstract and drawings), and any method or of the apparatuses so disclosed, can be made unless the contrary is clearly indicated by the context of the specification (including the accompanying claims, abstract and drawings), and any method or of the apparatuses so disclosed, can be made unless the contrary is clearly indicated by the context of the specification (including the accompanying claims, abstract and drawings). Each feature disclosed in the description (including the accompanying claims, abstract and drawings) can be replaced by alternative features that serve the same, equivalent or similar purpose, unless the contrary is explicitly stated.

[0097] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the sequencing device according to this application. This application can also be implemented as a device or apparatus program for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0098] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0099] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0100] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

[0101] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0102] It should be noted that the various data-related processes in the embodiments of this application are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.

Claims

1. A method for performing a reasoning task, characterized in that, The method includes: Obtain the original computation graph generated by the deep learning model for the inference task; the original computation graph includes nodes and edges between the nodes, the nodes are used to represent operations, and the edges are used to represent data transfer between operations; Analyze the original computation graph to determine the preliminary analysis results of the original computation graph; Based on the preliminary analysis results, a target optimization strategy is determined from the preset multi-scale optimization strategy; the target optimization strategy is one or a combination of the multi-scale optimization strategies. The optimized computational graph is obtained by adjusting the original computational graph according to the target optimization strategy. The original computation graph and the optimized computation graph are compared and displayed through a graphical user interface; In response to the optimization adjustment parameters input by the user through the graphical user interface, the optimization calculation graph is adjusted according to the optimization adjustment parameters to obtain the target calculation graph; The deep learning model is controlled to perform the inference task based on the target computation graph.

2. The method according to claim 1, characterized in that, The analysis of the original computation graph to determine the preliminary analysis results of the original computation graph includes: Extract the metadata of the nodes in the original computation graph; the metadata includes at least the operator type, input / output tensor shape, data type, whether it is a constant, and whether it includes trainable parameters; Construct a predecessor-successor dependency graph based on the original computation graph, and determine the optimizable patterns for matching the predecessor-successor dependency graph; The node type is determined based on a predefined operator semantic rule library; the node type includes at least basic operator class, activation function class, normalization class, control flow class, and redundant / pruned class; Obtain the characteristic information of the target hardware platform on which the deep learning model is deployed; the characteristic information includes at least memory bandwidth, computing unit architecture, and supported low-precision instruction set; The metadata, the optimizable mode, the node type, and the characteristic information are input into the prediction model to obtain the computational intensity and potential bottleneck regions of the operators in the original computation graph estimated by the prediction model. The core objectives of the original computational graph are determined based on the computational intensity and the potential bottleneck regions. The core objective is used as the preliminary analysis result of the original computation graph.

3. The method according to claim 2, characterized in that, The step of determining the target optimization strategy from the preset multi-scale optimization strategies based on the preliminary analysis results includes: In response to the optimization objective input by the user through the graphical user interface, a target optimization strategy is determined from a preset multi-scale optimization strategy based on the preliminary analysis results and the optimization objective.

4. The method according to claim 1, characterized in that, The step of comparing and displaying the original computation graph and the optimized computation graph through a graphical user interface includes: The topology of the original computation graph and the optimized computation graph is compared and displayed through a graphical user interface. In the topology of the optimized computation graph, nodes that have been modified, merged or deleted relative to the original computation graph are identified by preset visual elements. The change log is displayed through a graphical user interface. The change log includes at least a description of the optimization operations performed on the original computation graph and an estimated performance impact of the optimization operations.

5. The method according to claim 4, characterized in that, The method of comparing and displaying the original computation graph and the optimized computation graph through a graphical user interface also includes: When the optimization operation is identified as risky, a warning message corresponding to the optimization operation is provided in the graphical user interface.

6. The method according to claim 1, characterized in that, After adjusting the optimization computation graph according to the optimization adjustment parameters input by the user through the graphical user interface to obtain the target computation graph, the method further includes: The target computation graph is evaluated according to preset evaluation criteria to obtain evaluation results; the evaluation criteria include at least execution efficiency, resource consumption, and latency. The evaluation results are displayed through the graphical user interface. In response to the iterative optimization adjustment parameters input by the user through the graphical user interface, the previous round of optimization computation graph is iteratively adjusted according to the iterative optimization adjustment parameters to obtain a new target computation graph.

7. The method according to claim 1, characterized in that, The control of the deep learning model to perform the inference task according to the target computation graph includes: The target computation graph is converted into a format compatible with the target hardware platform; The transformed target computation graph is deployed to the target hardware platform so that the deep learning model deployed on the target hardware platform can perform the inference task based on the target computation graph.

8. A reasoning task execution device, characterized in that, The device includes: The original computation graph generation module is used to obtain the original computation graph generated by the deep learning model for the inference task; the original computation graph includes nodes and edges between the nodes, the nodes are used to represent operations, and the edges are used to represent data transfer between operations; The preliminary analysis result determination module is used to analyze the original computation graph to determine the preliminary analysis results of the original computation graph; The optimization strategy determination module is used to determine a target optimization strategy from a preset multi-scale optimization strategy based on the preliminary analysis results; the target optimization strategy is one or a combination of the multi-scale optimization strategies. The computation graph optimization module is used to adjust the original computation graph according to the target optimization strategy to obtain an optimized computation graph; The comparison and display module is used to compare and display the original computation graph and the optimized computation graph through a graphical user interface; The target computation graph determination module is used to adjust the optimization computation graph according to the optimization adjustment parameters input by the user through the graphical user interface to obtain the target computation graph; The inference task execution module is used to control the deep learning model to execute the inference task according to the target computation graph.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the instructions to implement the inference task execution method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal is able to execute the reasoning task execution method as described in any one of claims 1 to 7.

Images