Computational graph optimization method, device and data processing system

Abstract

Disclosed is a method, device and data processing system for optimizing a computational graph, which relate to the field of artificial intelligence. The method comprises: obtaining a computational graph corresponding to a neural network model, dividing the computational graph into a plurality of subgraphs according to the characteristics of the computational graph before running the computational graph, determining the scheduling mode of each subgraph, that is, the execution order of operators in the subgraph and at least one computing unit used to run the subgraph, and for each of the plurality of subgraphs, running the subgraph according to the scheduling mode of the subgraph. Thus, the computational graph is divided into a plurality of subgraphs, each subgraph is optimized separately, and the scheduling mode of each subgraph is sought. For large-scale computational graphs, the running time of the computational graph can be effectively shortened and resource occupation can be reduced, and the optimization complexity can be effectively reduced, thereby improving the efficiency of optimization.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a computational graph optimization method, device, and data processing system. Background Art

[0002] At present, neural network models are often represented as computational graphs. Optimizing the computational units used when running computational graphs and the execution order of operators in the computational graphs can shorten the running time and reduce the use of resources. However, current optimization algorithms have limitations. For example, greedy strategies and heuristic searches are weak in searching for global optimal solutions, and global optimization methods have significantly increased search time and low efficiency when processing large-scale computational graphs. Therefore, for the execution optimization problem of large-scale computational graphs, there is no optimization method that can quickly find solutions and ensure the quality of solutions. Summary of the invention

[0003] The present application provides a computational graph optimization method, device, and data processing system, thereby improving the optimization efficiency of the computational graph, shortening the running time of the computational graph, and reducing resource usage.

[0004] In a first aspect, a computational graph optimization method is provided, the method comprising: obtaining a computational graph corresponding to a neural network model, dividing the computational graph into multiple subgraphs according to the characteristics of the computational graph before running the computational graph, determining a scheduling mode for each subgraph, that is, the execution order of operators in the subgraph and at least one computing unit used to run the subgraph, and for each of the multiple subgraphs, running the subgraph according to the scheduling mode of the subgraph.

[0005] Compared with searching for the scheduling mode of the entire computational graph based on only one optimization algorithm, the optimization time of the computational graph is long, the optimization efficiency is low, the running time of the computational graph is long, and more resources are occupied. The computational graph optimization method provided in the present application divides a large-scale computational graph into small-scale subgraphs according to the characteristics of the computational graph. Since the subgraphs are small in size, each subgraph is optimized separately, and the scheduling mode of each subgraph is found as soon as possible, so that the optimization efficiency for the subgraph is high and the optimization time is short. Therefore, for large-scale computational graphs, it is possible to effectively reduce the optimization complexity, improve the optimization efficiency, shorten the running time of the computational graph, and reduce resource occupation.

[0006] In a possible implementation, partitioning the computational graph according to a partitioning strategy to obtain a plurality of subgraphs includes: partitioning the computational graph according to a feature of the computational graph and a feature of an optimization algorithm to obtain a plurality of subgraphs.

[0007] In this way, when optimizing the computational graph, subgraphs that conform to the structure of the computational graph and are compatible with the optimization algorithm are divided according to the characteristics of the computational graph and the characteristics of the optimization algorithm, so as to shorten the running time of the subgraph and reduce resource usage when running the subgraph, thereby shortening the running time of the computational graph and reducing resource usage.

[0008] In another possible implementation, the computation graph is partitioned according to a partitioning strategy to obtain multiple subgraphs, including: partitioning the computation graph according to characteristics of the computation graph, characteristics of the optimization algorithm, and subgraph sizes to obtain multiple subgraphs.

[0009] In this way, when optimizing the computational graph, subgraphs that conform to the structure of the computational graph and are compatible with the optimization algorithm are divided according to the characteristics of the computational graph, the characteristics of the optimization algorithm, and the target subgraph size, so that when running the subgraph, the running time of the subgraph can be shortened and the resource usage can be reduced, thereby shortening the running time of the computational graph and reducing the resource usage.

[0010] Among them, the characteristics of the computational graph include the control logic, computational logic and scale of the computational graph.

[0011] Therefore, when optimizing the computational graph, the computational graph is divided according to the control logic, computational logic, and the scale of the computational graph, so that the multiple subgraphs obtained by the division conform to the control logic and computational logic of the computational graph, thereby avoiding operational logic errors.

[0012] In another possible implementation, the multiple subgraphs include a first subgraph and a second subgraph, and the scheduling modes of the multiple subgraphs are determined, including: determining the scheduling mode of the second subgraph according to the resource usage status indicated by the optimization result of the first subgraph, the first subgraph is a subgraph that runs earlier than the second subgraph, and the resource usage status includes at least one of storage resource usage status, computing resource usage status, network resource usage status and energy consumption status.

[0013] When determining the scheduling mode of the current subgraph, refer to the resource usage status of the previous subgraph, so as to make full use of resources and improve resource utilization. For example, based on the computing resources and storage resources occupied by the previous subgraph, determine the computing resources and storage resources occupied by the next subgraph to avoid occupying too many resources, resulting in resource waste, and avoid insufficient resources causing the next subgraph to fail. In this way, each subgraph optimizes and searches for a scheduling mode based on the previous subgraph, improving the performance of the entire large-scale computing graph.

[0014] In another possible implementation, determining the scheduling mode of the second subgraph according to the resource usage status indicated by the optimization result of the first subgraph includes: determining the scheduling mode of the second subgraph according to the resource usage status indicated by the optimization result of the first subgraph by at least two optimization algorithms.

[0015] Optimize subgraphs based on multiple optimization algorithms to quickly find scheduling modes that shorten the running time of the computational graph and reduce resource usage.

[0016] For example, at least two optimization algorithms include greedy strategy, heuristic search and global optimization. The optimization effect for small and medium-sized computational graphs has been significantly improved, especially for medium-sized subgraphs, where the use of evolutionary algorithms for global optimization can achieve more than 30% optimization effect compared to local search algorithms.

[0017] In another possible implementation, the multiple subgraphs include a first subgraph and a second subgraph, and the subgraphs are run according to the scheduling mode of each subgraph in the multiple subgraphs, including: running the first subgraph according to the scheduling mode of the first subgraph; using the output data of the first subgraph as input data of the second subgraph, and running the second subgraph according to the scheduling mode of the second subgraph.

[0018] In a second aspect, a computation graph optimization device is provided, the computation graph optimization device comprising modules for executing the computation graph optimization method in the first aspect or any possible design of the first aspect. For example, the computation graph optimization device comprises a communication module, an optimization module, and an execution module.

[0019] A communication module is used to obtain a computational graph corresponding to a neural network model; an optimization module is used to partition the computational graph according to a partitioning strategy to obtain multiple subgraphs, where the partitioning strategy is used to indicate a method of partitioning the computational graph according to the characteristics of the computational graph; the optimization module is also used to determine a scheduling mode for multiple subgraphs, where the scheduling mode is used to indicate an execution order of operators in a subgraph and at least one computing unit used to run a subgraph; an execution module is used to run a subgraph according to the scheduling mode of each subgraph in a plurality of subgraphs.

[0020] In a possible implementation, when the optimization module partitions the computation graph according to the partitioning strategy to obtain multiple subgraphs, it is specifically used to: partition the computation graph according to the characteristics of the computation graph and the characteristics of the optimization algorithm to obtain multiple subgraphs.

[0021] In another possible implementation, when the optimization module partitions the computational graph according to the partitioning strategy to obtain multiple subgraphs, it is specifically used to: partition the computational graph according to the characteristics of the computational graph, the characteristics of the optimization algorithm and the subgraph scale to obtain multiple subgraphs.

[0022] In another possible implementation, the characteristics of the computation graph include the control logic, the computation logic, and the scale of the computation graph.

[0023] In another possible implementation, the multiple subgraphs include a first subgraph and a second subgraph. When the optimization module determines the scheduling mode of the multiple subgraphs, it is specifically used to: determine the scheduling mode of the second subgraph according to the resource usage status indicated by the optimization result of the first subgraph, the first subgraph is a subgraph that runs earlier than the second subgraph, and the resource usage status includes at least one of storage resource usage status, computing resource usage status, network resource usage status and energy consumption status.

[0024] In another possible implementation, when the optimization module determines the scheduling mode of the second subgraph according to the resource usage status indicated by the optimization result of the first subgraph, it is specifically used to: determine the scheduling mode of the second subgraph according to the resource usage status indicated by the optimization result of the first subgraph by at least two optimization algorithms.

[0025] In another possible implementation, the at least two optimization algorithms include a greedy strategy, a heuristic search, and a global optimization.

[0026] In another possible implementation, the multiple subgraphs include a first subgraph and a second subgraph. When the execution module runs the subgraph according to the scheduling mode of each subgraph in the multiple subgraphs, it is specifically used to: run the first subgraph according to the scheduling mode of the first subgraph; use the output data of the first subgraph as the input data of the second subgraph, and run the second subgraph according to the scheduling mode of the second subgraph.

[0027] In a third aspect, a computer system is provided, which includes a memory and multiple processors of different types, the memory being used to store a set of computer instructions; when the processor executes the set of computer instructions, the processor is used to execute the operating steps of the method in the first aspect or any possible implementation of the first aspect to implement computational graph optimization.

[0028] In a fourth aspect, a computer-readable storage medium is provided, comprising: computer software instructions; when the computer software instructions are executed in a processor, the processor executes the operating steps of the method described in the first aspect or any possible implementation of the first aspect.

[0029] In a fifth aspect, a computer program product is provided. When the computer program product is run on a computer, the computer is caused to execute the operation steps of the method described in the first aspect or any possible implementation manner of the first aspect.

[0030] The technical effects brought about by any design method in the second to fifth aspects can refer to the technical effects brought about by the first aspect or different design methods in the first aspect, and will not be repeated here.

[0031] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of a calculation graph provided for this application;

[0033] Figure 2 A schematic diagram of the effect of the execution order of a computation graph on peak memory provided in this application;

[0034] Figure 3 A schematic diagram of the effect of the execution order of a computation graph on the running time provided in this application;

[0035] Figure 4 A schematic diagram of heterogeneous computing provided for this application;

[0036] Figure 5 A schematic diagram of a system architecture provided for this application;

[0037] Figure 6 A flowchart of a computational graph optimization method provided in this application;

[0038] Figure 7 A schematic diagram of a computational graph partitioning provided for this application;

[0039] Figure 8 A schematic diagram of a computational graph partitioning provided for this application;

[0040] Fig. 9 A schematic diagram of the structure of a computational graph optimization device provided in this application;

[0041] Fig.10 A schematic diagram of the structure of a data processing system provided for this application;

[0042] Fig.11 A schematic diagram of the structure of a computer device provided in this application. DETAILED DESCRIPTION

[0043] To facilitate understanding, the main terms involved in this application are first explained.

[0044] Computation Graph: In machine learning and deep learning, neural network models can be represented by computational graphs, which include nodes and edges. Nodes represent variables (such as scalars, vectors, or tensors) or correspond to an operation, and edges represent operations on corresponding data or the flow direction of corresponding data, such as functions.

[0045] For example, Figure 1 As shown in (a), calculate Figure 1 Including node x, node y and edge f, calculate Figure 1Represents the function y=f(x); Figure 1 As shown in (b), calculate Figure 2 Including nodes x, y, z and edge g, calculate Figure 2 Represents the function z=g(x,y).

[0046] Subgraph: A part of a computational graph, including some nodes and edges between nodes, which can also be regarded as an independent computational graph.

[0047] Operator: In a computational graph, an operator refers to an entity that performs a specific computation. For example, operators include addition, multiplication, convolution, etc.

[0048] The execution order of a computation graph refers to the order in which operators in the computation graph are executed. In some embodiments, operators in the computation graph can be executed after the input data meets the conditions. Due to the characteristics of the computation graph, multiple operators sometimes need to meet the execution conditions at the same time before multiple operators can be executed. The execution order of operators has a significant impact on the peak memory, intermediate state, and running time during the execution of the computation graph.

[0049] Peak memory: The maximum storage capacity required during the computation process. After optimizing the execution order of operators in the computation graph, the memory requirements of the computation graph can be reduced. Optimizing peak memory can improve the utilization of limited memory resources.

[0050] For example, Figure 2 A schematic diagram of the effect of the execution order of a computational graph on peak memory provided by this application. Figure 2 As shown in (a) of FIG. , the structure of the three-branch computation graph. The computation graph includes nodes A to H. Figure 2 As shown in (b) in Figure 1, the computation graph is run according to the execution order of breadth-first search (BFS), that is, the computation graph is run in the order of nodes A to H, and the peak memory is 1500MB. The computation graph is run according to the optimal execution order, that is, the computation graph is run in the order of nodes A, C, F, B, D, G, E to H, and the peak memory is 1200MB. Therefore, the peak memory of the optimal execution order is smaller than the peak memory of the BFS execution order.

[0051] Runtime: The length of time it takes for a program to execute from start to finish. Optimizing runtime can improve program execution efficiency.

[0052] For example, Figure 3 As shown in FIG. 1 , a schematic diagram of the effect of the execution order of a computation graph on the running time provided by the present application is shown. Figure 3The structure of the computational graph shown in (a) is shown in Figure 3 As shown in (b) of the figure, the operators in the computation graph are executed in the order of AIBCDGEFH, and the total time required is 2700 microseconds (μs). Figure 3 As shown in (c) in the figure, the operators in the computation graph are executed in the order of BCEFAIDGH, and the total time required is 2100μs.

[0053] Heterogeneous computing refers to the use of different types of computing resources (such as neural network processing unit (NPU), graphic processing unit (GPU), central processing unit (CPU), etc.) in the system to complete computing tasks.

[0054] For example, Figure 4 As shown, it is a schematic diagram of heterogeneous computing provided by the present application. Program segments 1 to 9 are executed by three types of processors: CPU, NPU and GPU. Among them, the CPU executes program segments 1, 2, 5, 6 and 9. The NPU executes program segments 7 and 8. The GPU executes program segments 3 and 4.

[0055] Multi-objective optimization: In multi-objective optimization, one tries to find a balance between multiple, often conflicting, objectives, such as seeking a balance between minimizing running time and minimizing peak memory usage.

[0056] Computational graph optimization methods can be mainly divided into three categories: greedy strategy, heuristic search and global optimization.

[0057] Greedy algorithms: They are used to prioritize the operator that can be completed first, and then select the next operator that can be completed first. Greedy algorithms are mainly used to find local optimal solutions. They are fast to execute, but cannot ensure the global optimal solution. As a result, the optimization results on large-scale computational graphs are often far less than expected.

[0058] Heuristic search: Uses additional factors, such as computational and communication costs, to guide the search process. Heuristic search has a wider search range than greedy strategies, so it can find better solutions. However, heuristic search may still fail to find the global optimal solution. The efficiency of heuristic search is also subject to certain limitations. For computational graphs with higher complexity, heuristic search takes a long time to search.

[0059] Global optimization: attempts to find the global optimal solution. Common global optimization algorithms include evolutionary algorithms, simulated annealing, particle swarm optimization, etc. Global optimization can find the global optimal solution on a small-scale computational graph, and the optimization effect of global optimization is better than that of greedy strategy and heuristic search. However, when the scale of the computational graph is large (such as more than 1500 nodes), the efficiency of global optimization is low, the search time increases significantly, and it may not be possible to find an effective solution.

[0060] Evolutionary algorithm: is a class of optimization algorithms that simulate natural selection and genetics. This algorithm usually uses a set of candidate solutions (called a population) to search the solution space and improves the population by iteratively applying operations such as mutation and crossover.

[0061] In general, large-scale neural network models mean more parameters and greater computational complexity, which leads to more training resources and longer training and running times. Therefore, if the peak memory and running time required for neural network models can be reduced by optimizing the computational graph, the resource consumption and running time required for training and running large-scale computational graphs can be effectively reduced. However, the above optimization methods all have certain limitations. For the optimization of large-scale computational graphs, there is a lot of room for improvement in how to balance the optimization efficiency and optimization results, as well as the running time and resource usage of the computational graph.

[0062] In order to quickly find an optimization method for a large-scale computational graph and ensure the quality of the optimization method, the present application provides a computational graph optimization method, which obtains a computational graph corresponding to a neural network model, and before running the computational graph, divides the computational graph into multiple subgraphs according to the characteristics of the computational graph, and determines the scheduling mode of each subgraph, that is, the execution order of operators in the subgraph and at least one computing unit used to run the subgraph, and for each of the multiple subgraphs, runs the subgraph according to the scheduling mode of the subgraph.

[0063] Only searching for the scheduling mode of the entire computational graph based on one optimization algorithm leads to long optimization time and low optimization efficiency of the computational graph, long running time of the computational graph, and more resource usage. The computational graph optimization method provided by the present application divides a large-scale computational graph into small-scale subgraphs based on the characteristics of the computational graph. Since the subgraphs are small in size, each subgraph is optimized separately, and the scheduling mode of each subgraph is found as quickly as possible, so that the optimization efficiency for the subgraph is high and the optimization time is short. Therefore, for large-scale computational graphs, it is possible to effectively reduce the optimization complexity, improve the optimization efficiency, shorten the running time of the computational graph, and reduce resource usage.

[0064] The implementation of the computational graph optimization method provided in this application is described in detail below with reference to the accompanying drawings.

[0065] Figure 5 A schematic diagram of a system architecture provided for this application. Figure 5 As shown, the system 500 includes an execution device 510 , a training device 520 , a database 530 , a terminal device 540 , a data storage system 550 and a data acquisition device 560 .

[0066] The execution device 510 can be a terminal, such as a mobile phone terminal, a tablet computer, a laptop computer, a virtual reality (VR), an augmented reality (AR) device, a mixed reality (MR) device, an extended reality (ER) device, a camera or a vehicle-mounted terminal, etc., and can also be an edge device (for example, a box carrying a chip with processing capabilities), etc.

[0067] The training device 520 may be a server or a cloud device, etc. The training device 520 has strong computing capabilities and can run a neural network model and perform calculations such as training the neural network model.

[0068] As a possible embodiment, the execution device 510 and the training device 520 are different processors deployed on different physical devices (such as servers or servers in a cluster). For example, the execution device 510 can be a neural network processing unit (NPU), a graphic processing unit (GPU), a central processing unit (CPU), other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The training device 520 can be a GPU, an NPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0069] The data acquisition device 560 is used to collect training data and store the training data in the database 530. The training data can be data in at least one form of image, voice and text. For example, the training data includes a training image and a target in the training image.

[0070] The training device 520 is used to train the neural network model using the training data until the loss function in the neural network model converges, and the training of the neural network model is completed when the loss function value is less than a specific threshold, so that the neural network model reaches a certain accuracy. Alternatively, all the training data in the database 530 are used for training, and the training of the neural network model is completed, so that the trained neural network model has functions such as recognition or classification. Furthermore, the training device 520 configures the trained neural network model 501 to the execution device 510. The execution device 510 is used to realize the functions of the neural network model 501 by processing application data according to the trained neural network model 501, for example, realizing recognition functions such as recognizing images, languages ​​or texts.

[0071] In some embodiments, the training device 520 can configure the trained neural network model 501 to multiple execution devices 510. Each execution device 510 uses the trained neural network model 501 to implement functions such as recognition or classification.

[0072] For example, in the autonomous driving scenario, when the autonomous driving car is driving along the predetermined route, the neural network model is mainly used to identify road signs, driving reference objects and obstacles on the road in the environment to ensure that the autonomous driving car drives safely and accurately. Road signs can include graphic road signs or text road signs. Driving reference objects can be buildings or plants. Obstacles on the road can include dynamic objects (such as animals) or stationary objects (such as stationary vehicles).

[0073] For example, in monitoring scenarios, neural network models are used mainly to identify targets (such as cars and users) in environments such as intersections or parks.

[0074] For example, in natural language processing scenarios, neural network models are used mainly to recognize speech or text.

[0075] In order to improve the accuracy of the neural network model in processing data, the training device 520 can also perform iterative training on the neural network model based on the training data maintained by the database 530 and the application data provided by the execution device 510. It can be understood that iterative training refers to any training after the first training of the neural network model.

[0076] The higher the accuracy of the neural network model in processing data, the longer the neural network model takes to process data, and the larger the memory capacity occupied by the neural network model.

[0077] In addition, deep learning is widely used in the field of artificial intelligence, and deep learning pre-training models have experienced explosive growth, with model parameters and training data volumes increasing exponentially. Large-scale neural network models require more parameters and more computation, which in turn requires more training resources and longer training and running times.

[0078] The computational graph optimization method provided in the present application is that the training device 520 or the execution device 510 converts a large-scale neural network model into a computational graph, divides the computational graph into multiple subgraphs according to the characteristics of the computational graph, optimizes each subgraph separately, and finds the scheduling mode of each subgraph as quickly as possible, thereby optimizing the execution order of the computational graph and the computing units used, which can not only quickly find the optimization method for large-scale computational graphs, but also ensure the quality of the optimization method.

[0079] For example, when training the neural network model 501, the training device 520 can first optimize the calculation graph of the neural network model 501, and then execute the training process of the neural network model 501 according to the optimized calculation graph.

[0080] For another example, after the training device 520 completes training of the neural network model, it deploys the trained neural network model 501 to the execution device 510. Before the execution device 510 runs the trained neural network model 501, it can first optimize the computational graph of the trained neural network model 501, and then execute the trained neural network model 501 according to the optimized computational graph.

[0081] It should be noted that, in actual applications, the training data maintained in the database 530 may not all come from the data acquisition device 560, but may also be received from other devices. In addition, the training device 520 may not train the neural network model entirely based on the training data maintained by the database 530, but may also obtain training data from the cloud or other places to train the neural network model. The above description should not be used as a limitation on the embodiments of the present application.

[0082] Furthermore, according to the functions performed by the execution device 510, the execution device 510 can be further divided into: Figure 5 In the illustrated architecture, as shown in the figure, the execution device 510 is configured with a computing module 511 , an I / O interface 512 and a pre-processing module 513 .

[0083] The I / O interface 512 is used to exchange data with external devices. A user can input data to the I / O interface 512 through a terminal device 540. The input data may include images or videos. In addition, the input data may also come from a database 530.

[0084] The preprocessing module 513 is used to perform preprocessing according to the input data received by the I / O interface 512 .

[0085] When the execution device 510 preprocesses the input data, or when the computing module 511 of the execution device 510 performs calculations and other related processing, the execution device 510 can call the data, code, etc. in the data storage system 550 for corresponding processing, and can also store the data and instructions obtained from the corresponding processing in the data storage system 550.

[0086] For example, the optimized neural network model stored in the execution device 510 can be applied to the execution device 510. After the execution device 510 acquires the application data, the calculation module 511 inputs the application data into the optimized neural network model to obtain a processing result.

[0087] Finally, the I / O interface 512 returns the processing result to the terminal device 540, thereby providing it to the user so that the user can view the processing result.

[0088] exist Figure 5In the case shown, the user can manually give input data, and the manual giving can be operated through the interface provided by the I / O interface 512. In another case, the terminal device 540 can automatically send input data to the I / O interface 512. If the terminal device 540 is required to automatically send input data and needs to obtain the user's authorization, the user can set the corresponding authority in the terminal device 540. The user can view the processing results output by the execution device 510 at the terminal device 540, and the specific presentation form can be a specific method such as display, sound, action, etc. The terminal device 540 can also be used as a data acquisition terminal to collect the input data of the input I / O interface 512 and the processing results of the output I / O interface 512 as new sample data, and store them in the database 530. Of course, it is also possible not to collect through the terminal device 540, but the I / O interface 512 stores the input data of the input I / O interface 512 and the processing results of the output I / O interface 512 as new sample data in the database 530.

[0089] Figure 5 This is only a schematic diagram of a system architecture provided by an embodiment of the present application. Figure 5 The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. Figure 5 In the embodiment, the data storage system 550 is an external memory relative to the execution device 510. In other cases, the data storage system 550 may also be placed in the execution device 510.

[0090] It should be noted that the computational graph optimization function provided in this application can be implemented by a software system, by a hardware device, or by a combination of a software system and a hardware device.

[0091] In one possible implementation, the cloud service provider abstracts the computational graph optimization function into a cloud service and deploys the cloud service in a cloud data center. Users can consult and purchase this cloud service through the cloud service platform. After purchasing this cloud service, users can submit a computational graph optimization request to the cloud data center through a terminal device, and the cloud data center runs a computational graph optimization module to implement the computational graph optimization function provided by this application.

[0092] In another possible implementation, the computational graph optimization module can be packaged as a software package by a software provider. The user purchases the software package, and the user deploys the software package on his own server, or the user deploys the software package on a cloud server. For example, the computational graph optimization module is deployed by the tenant in the computing resources (such as a virtual machine) of the cloud data center rented by the tenant. The tenant purchases the computing resource cloud service provided by the cloud service provider through the cloud service platform, and runs the computational graph optimization module in the purchased computing resources, so that the computational graph optimization module performs the computational graph optimization function provided by this application. Optionally, the computational graph optimization module can also encrypt the data uploaded by the user and the file path of the data, and avoid direct contact with the data uploaded by the user without hindering the implementation effect, so as to ensure information security.

[0093] Next, the calculation graph optimization process is described in detail with reference to the accompanying drawings.

[0094] Figure 6 A flowchart of a computational graph optimization method provided in this application. Figure 6 As shown, the method includes the following steps.

[0095] Step 610: Obtain a computational graph corresponding to the neural network model.

[0096] The computational graph described in the embodiments of the present application may refer to a computational graph corresponding to a neural network model. The source code of the neural network model may be converted into a computational graph, and the method of converting the code into the computational graph is not limited, and reference may be made to traditional methods.

[0097] Optionally, the source code of other applications can also be converted into a calculation graph. Alternatively, the source code of a functional module in an application is converted into a calculation graph, or any source code that can achieve a certain function is converted into a calculation graph, or the source code of an application that needs to be optimized for performance is converted into a calculation graph, without limitation. Here, the application that needs to be optimized for performance can be an application that needs to improve operating efficiency, without limitation. Among them, the application can be a high performance computing (HPC) application, a cloud computing application, an artificial intelligence (AI) application, etc., without limitation.

[0098] Step 620: partition the computation graph according to the partitioning strategy to obtain multiple subgraphs.

[0099] The computation graph is divided into multiple subgraphs, so as to find the scheduling mode of each subgraph in the multiple subgraphs, that is, the execution order of the operators in the subgraph and at least one computing unit used to run the subgraph, thereby reducing the optimization complexity, improving the optimization efficiency, shortening the optimization time, shortening the running time of the computation graph and reducing resource usage. Among them, the sizes of multiple subgraphs can be the same or different. Multiple subgraphs can be connected to restore the computation graph, which conforms to the connection relationship of the computation graph.

[0100] In a first possible partitioning method, the computation graph is partitioned according to the characteristics of the computation graph to obtain multiple subgraphs. The characteristics of the computation graph include the control logic, the computation logic, and the scale of the computation graph. The computation graph is partitioned according to at least one of the control logic, the computation logic, and the scale of the computation graph to obtain multiple subgraphs.

[0101] Among them, the control logic may refer to the program execution logic structure composed of program statements contained in the program. For example, the control logic includes at least one of a sequential structure, a branch structure, or a loop structure. For example, the calculation graph is divided according to the structural characteristics of the calculation graph to obtain multiple subgraphs. The structural characteristics of the calculation graph may include the program execution logic structure indicated by the control logic of the calculation graph.

[0102] Understandably, a subgraph can be a program segment including input and output, and a computing process. A subgraph can include one or more statements. A subgraph can be part of a sequential structure, a branch structure, or a loop structure. For example, a program is a code containing a branch structure, the first subgraph includes a program segment of an If statement, and the second subgraph includes a program segment of an Else statement.

[0103] The computational logic may refer to the execution logic of the operators in the function. For example, the program contains computational features such as matrix computation, vector computation, scalar computation, computational memory access ratio, and computational time. The computational graph is divided according to the computational features of the computational graph to obtain multiple subgraphs. The computational graph is divided into similar or identical computations, and a subgraph may include operators of similar or identical computations, thereby making full use of system resources to save energy, such as computing resources, storage resources, or network resources.

[0104] The scale of a computational graph includes the number of edges and operators of the computational graph. The computational graph is divided into multiple subgraphs according to the scale of the computational graph. The number of subgraphs is determined according to the scale of the computational graph. If the scale of the computational graph is large, the number of subgraphs obtained by dividing the computational graph can be increased.

[0105] In a second possible partitioning method, the computational graph is partitioned according to the characteristics of the computational graph and the characteristics of the optimization algorithm to obtain multiple subgraphs.

[0106] From the above description of the optimization method of the computation graph, it can be seen that each optimization algorithm has different advantages. It is also possible to divide the computation graph into subgraphs that meet the characteristics of the optimization algorithm by combining the characteristics of the computation graph and the characteristics of the optimization algorithm. Thus, the subgraphs that meet the structure of the computation graph and are compatible with the optimization algorithm are divided so that when the subgraph is run, the running time of the subgraph is shortened and the resource usage is reduced, thereby shortening the running time of the computation graph and reducing the resource usage.

[0107] For example, the computation graph is divided according to the scale suitable for the optimization algorithm to solve. For example, a large-scale computation graph is divided into 20 subgraphs, each of which contains 300 nodes. Optimizing the subgraph containing 300 nodes based on the global optimization algorithm can not only quickly find the optimization method of the subgraph, but also ensure the quality of the optimization method.

[0108] Optionally, the computation graph is divided into multiple subgraphs by combining features of multiple optimization algorithms. The multiple subgraphs include subgraphs of affinity optimization algorithms. The sizes of subgraphs of different affinity optimization algorithms may be different. For example, the computation graph may be divided into multiple subgraphs according to greedy strategy, heuristic search, and global optimization. The multiple subgraphs include subgraphs of affinity greedy strategy, subgraphs of heuristic search, and subgraphs of global optimization.

[0109] When dividing the computational graph according to the characteristics of the optimization algorithm, the subgraph must also conform to the characteristics of the computational graph to ensure that the subgraph is a program segment that includes input and output and is a computational process. Avoid dividing the subgraph so that it does not conform to the control logic and computational logic of the computational graph.

[0110] In a third possible partitioning method, the computation graph is partitioned according to the characteristics of the computation graph, the characteristics of the optimization algorithm, and the subgraph size to obtain multiple subgraphs. The subgraph size may be a preset subgraph size. For example, the computation graph is partitioned according to the characteristics of the computation graph and the subgraph size to obtain multiple target subgraphs. If the sizes of the multiple target subgraphs meet the characteristics of the optimization algorithm, there is no need to adjust the sizes of the target subgraphs. If the sizes of the multiple target subgraphs do not meet the characteristics of the optimization algorithm, the sizes of the target subgraphs may be adjusted.

[0111] For example, Figure 7 A schematic diagram of a computation graph partitioning provided in this application. The computation graph includes nodes A to H. Figure 7 As shown in (a), node A can be divided into sub Figure 1 , divide nodes B, C, and D into sub Figure 2 , divide nodes E, F, and G into sub Figure 3 , divide node H into sub Figure 4 .like Figure 7 As shown in (b), nodes A, B, C, and D can be divided into sub Figure 1, divide nodes E, F, G, and H into sub Figure 2 .like Figure 7 As shown in (c), nodes B and E can be divided into sub Figure 1 , divide nodes A, C, F, and H into sub Figure 2 , divide nodes D and G into sub Figure 3 .

[0112] Step 630: Determine the scheduling mode of each subgraph.

[0113] The scheduling mode of each subgraph is determined according to the optimization algorithm. The scheduling mode may refer to the execution order of the operators in the subgraph and at least one computing unit used to run the subgraph. The same computing unit or multiple different types of computing units may be used to run a subgraph. Using multiple different types of computing units to run a subgraph can improve the running efficiency of the subgraph and shorten the running time of the subgraph. This is the heterogeneous computing described in the above embodiments.

[0114] You can first set the optimization goal and find the scheduling mode of the subgraph that meets the optimization goal. For example, minimize the running time and minimize the peak memory as the optimization goal, and find the scheduling mode of the subgraph that meets the optimization goal. The output of the optimization algorithm can indicate the scheduling mode of the subgraph.

[0115] In some embodiments, the scheduling mode of the current subgraph is determined according to the resource usage state indicated by the optimization result of the previous subgraph. And so on, until all subgraphs are optimized. For example, the multiple subgraphs include a first subgraph and a second subgraph, and the scheduling mode of the second subgraph is determined according to the resource usage state indicated by the optimization result of the first subgraph. The first subgraph is a subgraph that runs earlier than the second subgraph. The resource usage state includes at least one of a storage resource usage state, a computing resource usage state, a network resource usage state, and an energy consumption state.

[0116] The resource usage status indicated by the subgraph optimization result may refer to the estimated resource usage status of the system after the subgraph is run. For example, the estimated resource occupancy, remaining resource status, resource occupancy duration, energy consumption, etc. of the system after the subgraph is run. For example, the occupancy of computing resources, remaining computing resources, and occupancy duration of computing resources.

[0117] Therefore, when seeking a scheduling mode for a subgraph that meets the optimization goal, the usage status of resources in the system is referred to in order to make full use of system resources, reduce resource occupation, and improve resource utilization.

[0118] In other embodiments, the scheduling mode of the subgraph is determined according to the optimization results of the subgraph optimization by at least two optimization algorithms. The optimization results of the subgraph optimization are obtained according to each optimization algorithm, and the optimization results of each optimization algorithm are compared, and the optimization results that meet the optimization objectives and expectations are used as the scheduling mode of the subgraph. The at least two optimization algorithms include greedy strategy, heuristic search and global optimization.

[0119] For example, the scheduling mode of the second subgraph is determined according to the resource usage status indicated by the optimization results of the first subgraph by at least two optimization algorithms.

[0120] According to the execution order of operators in the subgraph indicated by the optimization results and at least one computing unit used to run the subgraph, the running time of the computational graph can be effectively shortened and resource usage can be reduced, such as reducing storage resources, computing resources, and energy consumption.

[0121] In other embodiments, each subgraph is optimized according to the serial execution order of the subgraphs. For example, the scheduling mode of the (i+1)th subgraph is determined according to the resource usage state indicated by the optimization result of the i-th subgraph.

[0122] Optionally, during the optimization process, the search process of the solution space can be adjusted according to the strategy of "simulated annealing" to avoid falling into a local optimal solution.

[0123] Finally, the optimization results of all subgraphs are combined to obtain the optimized execution order of the entire computational graph and the computational unit that executes each operator. This execution order not only optimizes the resource consumption of each subgraph, but also optimizes the resource consumption of the entire computational graph.

[0124] It should be noted that each optimization target (such as peak memory, running time, energy consumption) can be optimized separately, or multiple optimization targets can be optimized together. For example, peak memory and running time can be optimized at the same time to find the best balance between the two.

[0125] Step 640: Run the subgraph according to the scheduling mode of each subgraph in the multiple subgraphs.

[0126] The computation graph is divided into multiple subgraphs according to the partitioning strategy, and for each of the multiple subgraphs, the subgraph is run according to the scheduling mode of the subgraph. In some embodiments, the multiple subgraphs are run in series according to the execution order of the multiple subgraphs constituting the computation graph. In other embodiments, the execution order of the multiple subgraphs constituting the computation graph can also be ensured, and the multiple subgraphs can be run in parallel.

[0127] For example, multiple subgraphs are put into a queue according to the execution order of the computation graph composed of the multiple subgraphs, the subgraphs are read from the queue, and the scheduling mode of the subgraphs is determined until the queue is empty. Multiple subgraphs are serially run according to the execution order of the computation graph composed of the multiple subgraphs. The subgraphs are run according to the scheduling mode of the subgraphs.

[0128] The operation result of the previous subgraph is used as the input data of the subsequent subgraph, and the current subgraph is operated according to the scheduling mode of the current subgraph. For example, the multiple subgraphs include a first subgraph and a second subgraph, and the first subgraph is operated according to the scheduling mode of the first subgraph; the output data of the first subgraph is used as the input data of the second subgraph, and the second subgraph is operated according to the scheduling mode of the second subgraph.

[0129] For example, Figure 8 As shown, the computational graph is divided into three subgraphs. Figure 1 The result of running Figure 2 The input data, Figure 2 The result of running Figure 3 The input data, Figure 3 The running result is the running result of the computation graph. When running each subgraph, the subgraph is executed according to the scheduling mode of the subgraph, shortening the running time of the computation graph and reducing resource usage.

[0130] Therefore, by decomposing large-scale computational graphs into smaller and easier-to-optimize subgraphs, the optimization complexity is effectively reduced, and various optimization algorithms can be used to achieve excellent performance on large-scale computational graphs, thereby shortening the running time of the computational graph and reducing resource usage.

[0131] It is understandable that in order to implement the functions in the above embodiments, the computer device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0132] Combined with the above Figures 1 to 8 , describes in detail the computational graph optimization method provided by this application, and will be combined with Fig. 9 , describing the apparatus provided according to the present application. These apparatuses can be used to implement the functions of the computer device in the above method embodiment, and thus can also achieve the beneficial effects possessed by the above method embodiment. In this embodiment, the apparatus can be as follows Figure 5 The execution device or training device shown may also be a module (such as a chip) applied to a computer device.

[0133] like Fig. 9 As shown, the computation graph optimization device 900 includes a communication module 910, an optimization module 920, an execution module 930 and a storage module 940. The computation graph optimization device 900 is used to implement the above Figure 6 The functions of the computer device in the method embodiment shown in FIG.

[0134] The communication module 910 is used to obtain the computational graph corresponding to the neural network model. For example, the communication module 910 is used to execute Figure 6 Step 610.

[0135] The optimization module 920 is used to divide the computation graph into multiple subgraphs according to a partitioning strategy, where the partitioning strategy indicates a method of partitioning the computation graph according to the features of the computation graph. Figure 6 Step 620.

[0136] The optimization module 920 is further used to determine a scheduling mode for the plurality of subgraphs, where the scheduling mode is used to indicate the execution order of the operators in the subgraphs and at least one computing unit used to run the subgraphs. Figure 6 Step 630.

[0137] The optimization module 920 is specifically used to divide the computation graph into multiple subgraphs according to the characteristics of the computation graph and the characteristics of the optimization algorithm.

[0138] The optimization module 920 is further used to divide the computation graph into multiple subgraphs according to the characteristics of the computation graph, the characteristics of the optimization algorithm and the subgraph scale.

[0139] The execution module 930 is used to run the subgraph according to the scheduling mode of each subgraph in the multiple subgraphs. For example, the execution module 930 is used to execute Figure 6 Step 640.

[0140] The storage module 940 is used to store the computation graph, scheduling mode, optimization algorithm, and the running results of the subgraph, so as to optimize the computation graph and execute multiple subgraphs.

[0141] It should be understood that the computational graph optimization device 900 of the embodiment of the present application can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), and the PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. It can also be implemented by software. Figure 6 The computation graph optimization method shown in the figure and its various modules may also be software modules, and the computation graph optimization device 900 and its various modules may also be software modules.

[0142] The computation graph optimization device 900 according to the embodiment of the present application may correspond to executing the method described in the embodiment of the present application, and the above and other operations and / or functions of each unit in the computation graph optimization device 900 are respectively to implement Figure 6 For the sake of brevity, the corresponding processes of each method in are not repeated here.

[0143] Fig.10 This is a schematic diagram of the architecture of a data processing system provided in this application. Fig.10 As shown, data processing system 1000 includes client 1010 , computing cluster 1020 , and storage cluster 1030 .

[0144] The storage cluster 1030 includes at least two storage nodes 1031. A storage node 1031 includes one or more controllers, a network card and multiple hard disks. The hard disk is used to store data. The hard disk can be a magnetic disk or other types of storage media, such as a solid-state hard disk or a shingled magnetic recording hard disk. The network card is used to communicate with the computing node 1021 included in the computing cluster 1020. The controller is used to write data to the hard disk or read data from the hard disk according to the read / write data request sent by the computing node 1021. In the process of reading and writing data, the controller needs to convert the address carried in the read / write data request into an address that the hard disk can recognize.

[0145] The computing cluster 1020 includes at least two computing nodes 1021, and the computing nodes 1021 can communicate with each other. The computing node 1021 is a computing device, such as a server, a desktop computer, or a controller of a storage array.

[0146] The computing cluster 1020 includes a heterogeneous computing architecture to provide high-performance computing. For example, the computing node 1021 may include a central processing unit (CPU), a graphics processing unit (GPU), a data processing unit (DPU), a neural processing unit (NPU), and an embedded neural-network processing unit (NPU) and other computing units with computing capabilities to provide high-performance computing.

[0147] The client 1010 communicates with the computing cluster 1020 and the storage cluster 1030 through the network 1040. For example, the client 1010 sends a request to the computing cluster 1020 through the network 1040, requesting the computing cluster 1020 to optimize the computing graph. The network 1040 may refer to an internal network of an enterprise (such as a local area network (LAN)) or the Internet. The client 1010 refers to a computer connected to the network 1040, which may also be called a workstation. Different clients can share resources on the network (such as computing resources and storage resources).

[0148] In some embodiments, the computing cluster 1020 includes a control node 1022. The control node and the computing node can be independent physical devices, and the control node can also be referred to as a control device or a named node. The computing node can be referred to as a computing device or a data node. The control node 1022 is used to manage and allocate tasks, and multiple computing nodes execute multiple tasks in parallel to increase the data processing rate. In the present application, the control node 1022 is used to divide the computing graph into multiple subgraphs according to the characteristics of the computing graph, determine the scheduling mode of each subgraph, that is, the execution order of the operators in the subgraph and the at least one computing unit used to run the subgraph, and for each subgraph in the multiple subgraphs, instruct different computing nodes 1021 to run the subgraph according to the scheduling mode of the subgraph.

[0149] In an embodiment of the present application, the storage cluster 1030 stores computation graphs, subgraphs, scheduling modes, subgraph operation results, and the like.

[0150] The client 1010 is installed with a client program 1011. The client 1010 runs the client program 1011 to display a user interface (UI). The user 1050 operates the user interface to submit a request. For example, the user 1050 operates the user interface to submit multiple requests. After the computing node 1021 obtains the request, it loads the computation graph from the storage cluster 1030, divides the computation graph into multiple subgraphs, and determines the scheduling mode of each subgraph. In some embodiments, after the control node 1022 obtains the request, it loads the computation graph from the storage cluster 1030, divides the computation graph into multiple subgraphs, and determines the scheduling mode of each subgraph.

[0151] In other embodiments, the computing node 1021 runs the subgraph according to the scheduling mode of the subgraph based on a heterogeneous computing architecture.

[0152] Optionally, the system administrator 1060 can call the application platform interface (API) 1012 or the command-line interface (CLI) interface 1013 through the client 1010 to configure system information, such as the retrieval strategy configured for the computing node provided in the embodiment of the present application.

[0153] Fig.10 This is only a schematic diagram. The embodiments of the present application do not limit the device connection mode and the number of devices in the data processing system. For example, the data processing system may include multiple clients. A client may be connected to multiple computing nodes. Different clients establish connections with different computing nodes.

[0154] Fig.11 1 is a schematic diagram of the structure of a computer device 1100 provided in this application. Fig.11 As shown, the computer device 1100 includes a processor 1110, a bus 1120, a memory 1130, a communication interface 1140, a memory 1150 (also referred to as a main memory unit), and a processor 1160. The processor 1110, the processor 1160, the memory 1130, the memory 1150, and the communication interface 1140 are connected via the bus 1120.

[0155] It should be understood that in this embodiment, the processor 1110 may be a CPU, and the processor 1110 may also be other general-purpose processors, digital signal processors (DSP), ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0156] The computer device 1100 may also include a graphics processing unit (GPU), a neural network processing unit (NPU), a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the present application. For example, the processor 1160 may be a GPU or an NPU.

[0157] The communication interface 1140 is used to implement the communication between the computer device 1100 and an external device or device. Figure 6 When the functions shown are shown, the communication interface 1140 is used for the computation graph of the code to be optimized, so that the processor 1110 or the processor 1160 can perform the computation graph optimization. Since the processor 1110 and the processor 1160 execute multiple subgraphs.

[0158] The bus 1120 may include a path for transmitting information between the above components (such as the processor 1110, the memory 1150, and the storage 1130). In addition to the data bus, the bus 1120 may also include a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as bus 1120 in the figure. The bus 1120 may be a Peripheral Component Interconnect Express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a computer express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus 1120 can be divided into an address bus, a data bus, a control bus, etc.

[0159] As an example, the computer device 1100 may include multiple processors. The processor may be a multi-core (multi-CPU) processor. The processor here may refer to one or more devices, circuits, and / or computing units for processing data (such as computer program instructions).

[0160] It is worth mentioning that Fig.11In the example, a computer device 1100 including one processor 1110 and one memory 1130 is taken as an example. Here, the processor 1110 and the memory 1130 are respectively used to indicate a type of device or equipment. In a specific embodiment, the number of each type of device or equipment can be determined according to business requirements.

[0161] The memory 1150 may be a volatile memory pool or a non-volatile memory pool, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DR RAM). Memory 1150 is used to store computation graphs, scheduling modes, optimization algorithms, subgraph operation results, etc.

[0162] The memory 1130 may correspond to a storage medium used to store information such as calculation graphs and optimization algorithms in the above method embodiments, for example, a disk such as a mechanical hard disk or a solid-state hard disk.

[0163] The computer device 1100 may be a general-purpose device or a dedicated device. For example, the computer device 1100 may be an edge device (e.g., a box carrying a chip with processing capabilities), etc. Alternatively, the computer device 1100 may also be a server or other device with computing capabilities.

[0164] It should be understood that the computer device 1100 according to this embodiment may correspond to the computational graph optimization device 900 in this embodiment, and may correspond to executing Figure 6The corresponding subject in any method, and the above and other operations and / or functions of each module in the computation graph optimization device 900 are respectively to implement Figure 6 For the sake of brevity, the corresponding processes of each method in are not repeated here.

[0165] The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a computing device. Of course, the processor and the storage medium can also exist in a computing device as discrete components.

[0166] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD). The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A computational graph optimization method, characterized in that: include: Get the computational graph corresponding to the neural network model; Divide the computation graph according to a partitioning strategy to obtain a plurality of subgraphs, wherein the partitioning strategy is used to indicate a method of partitioning the computation graph according to features of the computation graph; Determining a scheduling mode for the plurality of subgraphs, the scheduling mode being used to indicate an execution order of operators in the subgraphs and at least one computing unit used to run the subgraphs; The subgraphs are executed according to a scheduling mode of each subgraph among the plurality of subgraphs.

2. The method according to claim 1, characterized in that The computation graph is divided according to the partitioning strategy to obtain multiple subgraphs, including: The computation graph is divided according to the characteristics of the computation graph and the characteristics of the optimization algorithm to obtain the multiple subgraphs.

3. The method according to claim 2, characterized in that The computation graph is divided according to the partitioning strategy to obtain multiple subgraphs, including: The computation graph is divided according to the characteristics of the computation graph, the characteristics of the optimization algorithm and the subgraph scale to obtain the multiple subgraphs.

4. The method according to any one of claims 1 to 3, characterized in that The characteristics of the computation graph include the control logic, computation logic and scale of the computation graph.

5. The method according to any one of claims 1 to 4, characterized in that The multiple subgraphs include a first subgraph and a second subgraph, and determining a scheduling mode of the multiple subgraphs includes: The scheduling mode of the second subgraph is determined according to the resource usage status indicated by the optimization result of the first subgraph, the first subgraph is a subgraph that runs before the second subgraph, and the resource usage status includes at least one of storage resource usage status, computing resource usage status, network resource usage status and energy consumption status.

6. The method according to claim 5, characterized in that Determining a scheduling mode of the second subgraph according to a resource usage state indicated by an optimization result of the first subgraph includes: The scheduling mode of the second subgraph is determined according to the resource usage status indicated by the optimization results of the first subgraph by at least two optimization algorithms.

7. The method according to claim 6, characterized in that The at least two optimization algorithms include a greedy strategy, a heuristic search, and a global optimization.

8. The method according to any one of claims 1 to 7, characterized in that The multiple subgraphs include a first subgraph and a second subgraph, and running the subgraphs according to a scheduling mode of each subgraph in the multiple subgraphs includes: running the first subgraph according to the scheduling mode of the first subgraph; The output data of the first subgraph is used as the input data of the second subgraph, and the second subgraph is run according to the scheduling mode of the second subgraph.

9. A computational graph optimization device, characterized in that: include: A communication module, used to obtain the computational graph corresponding to the neural network model; An optimization module, used for dividing the computation graph into a plurality of subgraphs according to a partitioning strategy, wherein the partitioning strategy is used for indicating a method for partitioning the computation graph according to features of the computation graph; The optimization module is further used to determine a scheduling mode for the multiple subgraphs, where the scheduling mode is used to indicate the execution order of operators in the subgraphs and at least one computing unit used to run the subgraphs; An execution module is used to run each subgraph in the multiple subgraphs according to the scheduling mode of the subgraph.

10. A data processing system, characterized in that: The data processing system includes a storage node and multiple computing nodes of different types, and the storage node is used to store a set of computer instructions; when the computing node executes the set of computer instructions, the operating steps of the method described in any one of claims 1 to 8 are performed to achieve calculation graph optimization.

11. A computer-readable storage medium, characterized in that: The method comprises computer software instructions; when the computer software instructions are executed in a processor, the operation steps of the method described in any one of claims 1 to 8 are executed.

12. A computer program product, characterized in that When the computer program product is executed on a computer, the computer is enabled to execute the operation steps of the method according to any one of claims 1 to 8.

Images