Calculation graph processing method and device

Abstract

The embodiment of the invention discloses a computational graph processing method and device, and the method comprises the steps: carrying out the merging operation of at least two first sub-graphs in a computational graph through the traversal of the computational graph, and obtaining a second sub-graph; and then binding segmentation information for the second sub-graph according to the segmentation information of the first sub-graph. Candidate segmentation information of the second subgraph is determined according to the number of the available computing resources, the candidate segmentation information comprises at least one segmentation axis and a candidate segmentation coefficient of the segmentation axis, and a segmentation mode corresponding to the candidate segmentation coefficient can use the number of the available computing resources to the maximum extent. And in response to the condition that the candidate segmentation coefficient of at least one segmentation axis in the segmentation information of the second sub-graph is a factor of the maximum segmentation coefficient, determining that the merging operation is valid. Therefore, the operators can be quickly grouped, operator grouping and computing resource allocation are uniformly considered, the repeated computing process in the sub-graph merging process is reduced, the compilation process of the computing graph is accelerated, and the computing resource utilization rate is increased.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for processing a computational graph. Background Art

[0002] Artificial intelligence has been developing continuously in recent years, making great achievements in many fields and affecting all aspects of life, such as face recognition, autonomous driving, safety protection, gaming and entertainment, voice technology, medical treatment, etc. Artificial intelligence and its related industries have become a new frontier of current technological competition, profoundly affecting the technological progress of the new generation.

[0003] With the continuous development of artificial intelligence technology, the corresponding amount of computing operations and memory access has also increased rapidly, far exceeding the processing capabilities of hardware. At present, the expression of artificial intelligence technology is often presented in the form of computational graphs in processing. The impact of the execution efficiency of computational graphs on hardware processing capabilities cannot be underestimated. How to optimize the execution efficiency of computational graphs on various hardware, the scheduling problem is one of the most critical issues. Good computational graph scheduling can greatly optimize the performance of hardware.

[0004] There are mainly the following methods in the existing automatic scheduling technical solutions: one is to specify a part or the whole of the calculation graph to be calculated on the hardware accelerator completely by the general default scheduling method. This scheduling method has a coarse granularity and cannot finely control the allocation of computing and storage resources during the calculation process, and cannot give full play to the ultimate performance of a specific chip architecture; the other is to first group by operator characteristics and local characteristics of the calculation graph, and then allocate computing resources to each group separately. The two processes are carried out separately and independently, and the impact of each other is not considered. This method does not take into account the overall situation. The relevant information of each group will be repeatedly calculated during the resource allocation process, resulting in more redundant calculations, which causes a lot of waste when compiling the calculation graph; at the same time, the computing resources are considered separately when each group is split, which may make it impossible to find a way to fully utilize the computing resources in the computing resource allocation derivation stage. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a method and device for processing a computational graph, so as to consider the two processes of operator grouping and computing resource allocation in a unified manner, thereby reducing the repeated calculation process in the process of merging computational subgraphs, speeding up the compilation process of the computational graph, and making full use of computing resources, thereby improving the utilization rate of computing resources.

[0006] In a first aspect, a method for processing a computational graph is provided, the method comprising:

[0007] Traversing a computation graph, merging at least two first subgraphs in the computation graph to obtain a second subgraph, wherein the first subgraph includes at least one operator, and the first subgraph is bound with segmentation information, the segmentation information includes an axis correspondence, at least one segmentation axis, and a maximum segmentation coefficient of the at least one segmentation axis, and the axis correspondence is a relationship between input data and output data of the subgraph;

[0008] Binding segmentation information to the second subgraph according to the segmentation information of the first subgraph;

[0009] Determining candidate segmentation information of the second subgraph according to the number of available computing resources, the candidate segmentation information comprising: at least one segmentation axis and candidate segmentation coefficients of the at least one segmentation axis, wherein the segmentation method corresponding to the candidate segmentation coefficient can maximize the use of the number of available computing resources;

[0010] In response to a candidate segmentation coefficient of at least one segmentation axis in the segmentation information of the second subgraph being a factor of the maximum segmentation coefficient, it is determined that the merging operation is valid.

[0011] In a second aspect, a computation graph processing device is provided, the device comprising:

[0012] A merging module, configured to traverse a computation graph and merge at least two first subgraphs in the computation graph to obtain a second subgraph, wherein the first subgraph includes at least one operator, and the first subgraph is bound with segmentation information, and the segmentation information includes an axis correspondence, at least one segmentation axis, and a maximum segmentation coefficient of the at least one segmentation axis, and the axis correspondence is a relationship between input data and output data of the subgraph;

[0013] A binding module, configured to bind segmentation information to the second subgraph according to the segmentation information of the first subgraph;

[0014] A first determining module is configured to determine candidate segmentation information of the second subgraph according to the number of available computing resources, wherein the candidate segmentation information includes: at least one segmentation axis and candidate segmentation coefficients of the at least one segmentation axis, wherein the segmentation method corresponding to the candidate segmentation coefficient can maximize the use of the number of available computing resources;

[0015] The second determining module is configured to determine that the merging operation is valid in response to a candidate segmentation coefficient of at least one segmentation axis in the segmentation information of the second subgraph being a factor of the maximum segmentation coefficient.

[0016] According to a third aspect, an electronic device is provided, the device comprising:

[0017] a memory for storing one or more computer program instructions;

[0018] A processor, wherein the one or more computer program instructions are executed by the processor to implement the method as in the first aspect.

[0019] According to a fourth aspect, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method according to the first aspect is implemented.

[0020] An embodiment of the present invention traverses a computational graph and merges at least two first subgraphs in the computational graph to obtain a second subgraph, wherein the first subgraph includes at least one operator, and the first subgraph is bound with segmentation information, and the segmentation information includes axis correspondence, at least one segmentation axis, and a maximum segmentation coefficient of the at least one segmentation axis, and the axis correspondence is a relationship between input data and output data of the subgraph; according to the segmentation information of the first subgraph, the segmentation information is bound to the second subgraph; candidate segmentation information of the second subgraph is determined according to the number of available computing resources, and the candidate segmentation information includes: at least one segmentation axis and candidate segmentation coefficients of the at least one segmentation axis, wherein the segmentation method corresponding to the candidate segmentation coefficient can maximize the use of the number of available computing resources; in response to the candidate segmentation coefficient of at least one segmentation axis in the segmentation information of the second subgraph being a factor of the maximum segmentation coefficient, the merging operation is determined to be valid. In this way, operators can be grouped quickly, and by considering operator grouping and computing resource allocation in a unified manner, the repeated calculation process in the process of merging computing subgraphs is reduced, the compilation process of the computing graph is accelerated, and the computing resource utilization rate is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0022] Figure 1 A flowchart of a method for processing a computational graph according to an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of derivation of axis correspondence of a single operator according to an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of derivation of axis correspondence of a fusion operator according to an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of initializing the calculation graph according to an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of the use of computing graph grouping in an embodiment of the present invention;

[0027] Figure 6 A schematic diagram of a computing graph processing device according to an embodiment of the present invention;

[0028] Figure 7 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The present application is described below based on embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, some specific details are described in detail. It is possible for those skilled in the art to fully understand the present application without the description of these details. In order to avoid confusing the essence of the present application, known methods, processes, flows, components and circuits are not described in detail.

[0030] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0031] Unless the context clearly requires otherwise, the words "include", "comprising" and similar words throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, the meaning is "including but not limited to".

[0032] In the description of this application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.

[0033] Before introducing the specific embodiments of the present invention in detail, the application scenario of the present invention is first described. The present invention can be applied to the grouping process of the calculation graph. Among them, the calculation graph, also known as the data flow graph, can often be regarded as a directed acyclic graph (DAG). The emergence of the calculation graph is to solve many problems encountered in the process of artificial intelligence systematization, such as how to simplify, merge, and convert neural networks, how to plan so that the computing unit can be executed at high speed on the accelerator, and how to pre-allocate and manage memory. To solve the above problems, it is necessary to provide a unified description for various neural networks, such as convolutional neural networks or deep neural networks. Specifically, the common operations in various neural network models are encapsulated as operators through the artificial intelligence programming framework (Artificial Intelligence Framework Problem) for programmers to directly call, such as convolution, pooling, etc. In the above programming framework, a calculation graph is usually used to describe the calculation process, tensors are used to represent various data in the calculation graph, and operators are used to represent various operations. The use of the calculation graph allows the artificial intelligence programming framework to represent the global information defined by various neural network models before execution, and improves the calculation efficiency by simplifying the calculation graph.

[0034] When deploying an artificial intelligence system on a hardware platform, the available computing resources of the hardware platform must be considered. Currently, mainstream hardware accelerators usually have a multi-level storage architecture, including local memory, which has the fastest data exchange speed with computing resources (or called acceleration cores), shared memory, which has the second fastest speed, and global memory, which has the slowest speed. Each acceleration core has its own local memory. It can be seen that if all the calculated data is stored in the local memory, the execution efficiency of the corresponding calculation graph is the fastest. However, the amount of data corresponding to the tensor of the calculation graph and the intermediate calculation results generated by each operator is often large, and the required space storage resources are also large, which makes it difficult to put it completely on the local memory. At the same time, if the data is placed on storage at different levels, it will cause a large amount of data transfer between different levels, thereby increasing the performance overhead caused by many unnecessary data exchanges.

[0035] To solve the above technical problems, the embodiments of the present invention provide a method and device for processing a computational graph, so that in the process of grouping operators in the computational graph, the amount of available computing resources corresponding to the computational graph is considered, so that the tensors corresponding to the grouped computational subgraphs can be segmented, and the segmented tensors can be completely placed on the local memory to reduce the performance overhead caused by data exchange. The following is a detailed description in conjunction with specific embodiments.

[0036] Figure 1 FIG. 1 is a flowchart of a method for processing a computational graph according to an embodiment of the present invention. Figure 1 As shown, the processing method of the computational graph includes the following steps:

[0037] In step S101, the computation graph is traversed, and at least two first subgraphs in the computation graph are merged to obtain a second subgraph, wherein the first subgraph includes at least one operator, and the first subgraph is bound with segmentation information, and the segmentation information includes an axis correspondence, at least one segmentation axis, and a maximum segmentation coefficient of the at least one segmentation axis, and the axis correspondence is a relationship between input data and output data of the subgraph.

[0038] Among them, calculation Figure 1 It generally includes multiple operators and directed edges connecting adjacent operators, recording the computational structure of the corresponding neural network model.

[0039] Optionally, after determining the computational graph to be processed, initialize the computational graph. Specifically, generate a corresponding computational subgraph for each operator in the computational graph, and then obtain the segmentation axis of each operator, which is used to divide a larger tensor into several smaller tensors. And calculate the maximum segmentation coefficient of each segmentation axis according to the output data shape of each operator. And, traverse the computational graph in reverse to deduce and determine the axis correspondence of each operator. According to the segmentation axis of each operator, the maximum segmentation coefficient of the segmentation axis, and the axis correspondence obtained, bind segmentation information to each generated computational subgraph. Each computational subgraph generated in the initial process includes an operator, and the segmentation information bound to each computational subgraph is the segmentation axis of each operator, the maximum segmentation coefficient of the segmentation axis, and the axis correspondence.

[0040] In a general sense, operators can include single operators or fused operators. A single operator means that the generalized operator contains only a single operator, while a fused operator is composed of multiple single operators. When deriving the axis correspondence, the axis correspondence of a single operator is relatively easy to obtain, while the axis correspondence of a fused operator requires traversing each operator in the fused operator in reverse order, and deriving the axis correspondence between the output tensor of the generalized operator and its input tensor through the intermediate calculation results of each operator. In other words, the derivation of the axis correspondence of the fused operator requires dividing the fused operator into multiple single operators, and on the basis of the derivation of the axis correspondence of the single operator, traverse each operator in the fused operator in reverse order, and determine the data flow between the operators through the intermediate settlement results between the operators.

[0041] Figure 2 FIG. 1 is a schematic diagram of the derivation of the axis correspondence relationship of a single operator in an embodiment of the present invention. Figure 2As shown, assuming that the operator is a two-dimensional matrix multiplication operator, determine the corresponding relationship between the output tensor and the input tensor. The input tensor of the two-dimensional matrix multiplication operator includes a left input tensor 21 and a right input tensor 22. The segmentation dimension of the left input tensor 21 of the two-dimensional matrix multiplication operator is [m, k], and the segmentation dimension of the right input tensor 22 is [k, n]. According to the operation rules of the two-dimensional matrix multiplication operator, it is easy to know that the segmentation dimension of the output tensor 23 is [m, n], where m is the number of matrix rows of the left input tensor 21 and the output tensor 23, n is the number of matrix columns of the right input tensor 22 and the output tensor 23, and k is the number of matrix columns of the left input tensor 21 and the number of matrix rows of the right input tensor 22. Then the segmentation dimension of the target operator can be any of the above dimensions, and each dimension corresponds to a segmentation axis.

[0042] Figure 3 FIG. 1 is a schematic diagram of the derivation of the axis correspondence relationship of the fusion operator in an embodiment of the present invention. Figure 3 As shown, the fusion operator includes an addition operator and a matrix multiplication operator. The solid arrows in the figure indicate the data flow in the calculation diagram, and the dotted arrows indicate the derivation direction of the axis correspondence. The segmentation dimensions corresponding to the input tensors 331, 332, and 320 of the fusion operator are [a, k], [b, k], and [k, n], respectively, and the segmentation dimension corresponding to the output tensor 310 is [m, n]. Each segmentation dimension corresponds to a segmentation axis. It is difficult to directly derive the axis correspondence from the segmentation axis corresponding to the input tensor and the segmentation axis corresponding to the output tensor. Therefore, it is necessary to traverse each operator in the fusion operator in a reverse traversal manner, and derive the axis correspondence between the output tensor and the input tensor of the fusion operator through the intermediate calculation results.

[0043] Specifically, according to the opposite direction of the input tensor flow direction, the first operator to be analyzed is determined to be a two-dimensional matrix multiplication operator. For this operator, according to the segmentation dimension [m, n] corresponding to its output tensor 310 and the segmentation dimension [k, n] corresponding to a known input tensor 320, it is easy to derive that the segmentation dimension corresponding to another input tensor 330 of the two-dimensional matrix operator should be [m, k]. That is to say, the segmentation dimension of the intermediate calculation result corresponding to the second-to-last operator is [m, k]. The second-to-last operator is an addition operator. For this single operator, it is easy to derive that the relationship between the segmentation dimension [a, k] corresponding to its input tensor 331 and the segmentation dimension [b, k] corresponding to the input tensor 332 and the segmentation dimension [m, k] corresponding to the intermediate calculation result 330 corresponding to the addition operator is m=a+b. Therefore, it can be obtained that the axis correspondence between the segmentation dimensions [a, k], [b, k], [k, n] corresponding to the fusion operator input tensors 331, 332, 320 and the segmentation dimension [m, n] corresponding to the output tensor 310. The axis correspondence between various operators can be determined according to the above method. Then, for each operator, the segmentation axis of the operator is determined according to its axis correspondence.

[0044] According to the above method, the segmentation axis of each operator in the computation graph can be determined, and then when traversing the computation graph and generating a corresponding computation subgraph for each operator in the computation graph, the segmentation axis of the operator is determined as the segmentation axis of the corresponding computation subgraph.

[0045] Figure 4 Schematic diagram of initializing the calculation graph according to an embodiment of the present invention. Figure 4 As shown, the computation graph 40 is traversed to generate a corresponding computation subgraph for each operator in the computation graph 40. The operator is represented by group, and the computation graph 30 includes 7 operators, namely group1, group2, group3...group7. The computation subgraph is initialized, that is, a separate computation subgraph is generated for each operator, and the operators group1, group2, group3...group7 correspond to computation subgraphs 41, 42, 43...47. Then, the segmentation information is bound to each computation subgraph.

[0046] Among them, the first subgraph is a subgraph to be merged, and the subgraph to be merged refers to a subgraph that can still be merged. Specifically, the first subgraph can be a computational subgraph obtained by initializing the computational graph, and the first subgraph includes an operator. The first subgraph can also be a computational subgraph obtained after a merge operation in the computational graph, and the first subgraph may include an operator (the operator has not been merged with other operators or the merge attempt failed), or it may include multiple merged operators.

[0047] Optionally, before merging the at least two first subgraphs in the computation graph, the method further includes obtaining the at least two first subgraphs in the computation graph, wherein the rule for obtaining the at least two first subgraphs to be merged can be set according to business requirements.

[0048] In one possible implementation, any two adjacent sequential subgraphs in the computation graph (for example, computation subgraph 41 and computation subgraph 42) are determined as two first subgraphs to be merged, wherein there is a linear data dependency between the two sequential subgraphs, both sequential subgraphs have a single output, and the downstream subgraphs in the two sequential subgraphs have a single input, and the input of the downstream subgraphs in the two sequential subgraphs is the output of the upstream subgraph.

[0049] In one possible implementation, a plurality of branch subgraphs (e.g., computation subgraphs 43-46) in the computation graph that may constitute a minimum branch subgraph are determined as at least two first subgraphs to be merged, wherein the minimum branch subgraph is a single input and a single output, that is, the minimum branch subgraph has an input subgraph (e.g., computation subgraph 43) and an output subgraph (e.g., computation subgraph 46), the data in the minimum branch subgraph comes from the input subgraph, and the data converges at the output subgraph.

[0050] In step S102, segmentation information is bound to the second subgraph according to the segmentation information of the first subgraph.

[0051] In the process of merging at least two first subgraphs into a second subgraph, it is necessary to further judge the segmentation information of the second subgraph in step S104, so the segmentation information of the second subgraph is calculated in step S102, and the segmentation information is bound to the second subgraph. After the segmentation information is bound to the second subgraph, if the merged subgraph (i.e., the second subgraph) is merged again, the corresponding segmentation information can be directly obtained according to the binding information of the second subgraph, and there is no need to calculate the segmentation information of the second subgraph again, which greatly reduces the redundant calculations in the process of calculating graph compilation, thereby optimizing the graph compilation process.

[0052] Specifically, the axis correspondence and at least one segmentation axis of the second subgraph are determined according to the axis correspondence between two adjacent first subgraphs in the second subgraph. According to the axis correspondence between all first subgraphs, it can be determined that there is a common segmentation axis (i.e., the segmentation axis of the second subgraph) between the at least two first subgraphs, that is, each first subgraph can be segmented on the common segmentation axis.

[0053] Specifically, by traversing the axis correspondence between the first subgraphs in reverse order, a common segmentation axis between at least two first subgraphs is determined, and then the common segmentation axis is determined as the segmentation axis of the second subgraph. Figure 4 The calculation subgraph 45, the calculation subgraph 46 and the calculation subgraph 47 (i.e., the first subgraph) are merged, the segmentation axes of the calculation subgraph 45 are the 0-axis and the 1-axis, the segmentation axes of the calculation subgraph 46 are the 0-axis and the 1-axis, and the segmentation axes of the calculation subgraph 47 are the 0-axis and the 4-axis. If the 0-axis of the calculation subgraph 47 corresponds to the 1-axis of the calculation subgraph 46, the 0-axis of the calculation subgraph 47 corresponds to the 0-axis of the calculation subgraph 45, and the 1-axis of the calculation subgraph 46 corresponds to the 0-axis of the calculation subgraph 45, then the 0-axis of the calculation subgraph 45, the 1-axis of the calculation subgraph 46, and the 0-axis of the calculation subgraph 47 are the common segmentation axes of the three first subgraphs. Therefore, the segmentation axes of the second subgraph obtained after the merger of the calculation subgraph 45, the calculation subgraph 46, and the calculation subgraph 47 are the 0-axis of the calculation subgraph 45, the 1-axis of the calculation subgraph 46, and the 0-axis of the calculation subgraph 47. Furthermore, since the 0-axis of calculation subgraph 45, the 1-axis of calculation subgraph 46, and the 0-axis of calculation subgraph 47 are corresponding axes, that is, the 0-axis of calculation subgraph 45 and the 1-axis of calculation subgraph 46 can be determined by the 0-axis of calculation subgraph 47, the 0-axis of calculation subgraph 47 can be defined as the dividing axis of the second subgraph.

[0054] According to each of the segmentation axes of the first subgraphs and the corresponding maximum segmentation coefficients, the maximum segmentation coefficient of each segmentation axis of the second subgraph is determined. Among them, the common segmentation axis of all the first subgraphs is the segmentation axis of the second subgraph, and the greatest common factor of the maximum segmentation coefficients of the common segmentation axes of all the first subgraphs is the maximum segmentation coefficient of the segmentation axis of the second subgraph. Therefore, the method for determining the maximum segmentation coefficient of each segmentation axis of the second subgraph is to determine the greatest common factor of the maximum segmentation coefficients of the common segmentation axes of all the first subgraphs. In response to the greatest common factor being greater than one, the greatest common factor is determined as the maximum segmentation coefficient of the corresponding segmentation axis in the second subgraph. For example, for the above example, the 0-axis of calculation subgraph 45, the 1-axis of calculation subgraph 46, and the 0-axis of calculation subgraph 47 are the common dividing axes of the three first subgraphs, and the maximum dividing coefficient of the 0-axis of calculation subgraph 45 is 4, the maximum dividing coefficient of the 1-axis of calculation subgraph 46 is 4, and the maximum dividing coefficient of the 0-axis of calculation subgraph 47 is 6, where the greatest common divisor of 4, 4, and 6 is 2, and the maximum dividing coefficient of the dividing axis of the second subgraph is 2.

[0055] In response to the greatest common divisor being not greater than one, it means that the second subgraph does not have a way to maximize the use of computing resources and to perform whole-cutting on the second subgraph, and the second subgraph cannot be merged, that is, the merge operation in step S101 is invalid. For example, for the above example, the maximum split coefficient of the 0-axis of the calculation subgraph 45 is 4, the maximum split coefficient of the 1-axis of the calculation subgraph 46 is 3, and the maximum split coefficient of the 0-axis of the calculation subgraph 47 is 6. The greatest common divisor of 4, 3, and 6 is 1. This means that the input tensor in the merged second subgraph can only be cut into 1 point, that is, the input tensor cannot be split, and the merge operation is meaningless. Therefore, if the greatest common divisor is not greater than one, it means that at least two first subgraphs cannot be merged into the second subgraph. If it is greater than one, the candidate split information of the second subgraph can be further determined through step S103.

[0056] After calculating the segmentation information of the second sub-graph, the segmentation information of the second sub-graph is bound to the second sub-graph. Specifically, the axis correspondence of the second sub-graph is bound to the second sub-graph. And, the segmentation axis in the second sub-graph and the maximum segmentation coefficient of the segmentation axis are directly assigned to the second sub-graph. For example, for the above example, it is determined that the segmentation axis of the second sub-graph is the 0-axis of the calculation sub-graph 47, and the maximum segmentation coefficient of the segmentation axis of the second sub-graph is 2, so the 0-axis of the calculation sub-graph 47 and the maximum segmentation coefficient of 2 are directly assigned to the merged second sub-graph.

[0057] In step S103, candidate segmentation information of the second subgraph is determined according to the number of available computing resources, the candidate segmentation information including: at least one segmentation axis and candidate segmentation coefficients of the at least one segmentation axis, wherein the segmentation method corresponding to the candidate segmentation coefficient can maximize the use of the number of available computing resources.

[0058] Specifically, the calculation peak value of the second subgraph is obtained, and the candidate segmentation information of the second subgraph is determined according to the calculation peak value and the number of available computing resources. The calculation peak value of the second subgraph is the maximum value of all surviving data when calculating a node during the calculation process of the second subgraph. The minimum segmentation coefficient of the second subgraph is determined according to the calculation peak value of the second subgraph and the number of available computing resources. Then, each segmentation axis of the second subgraph and the minimum segmentation coefficient are respectively determined as each candidate information, that is, each candidate information includes a segmentation axis and the minimum segmentation coefficient corresponding to the segmentation axis.

[0059] In step S104, in response to the candidate slicing coefficient of at least one slicing axis in the slicing information of the second subgraph being a factor of the maximum slicing coefficient, it is determined that the merging operation is effective. The candidate slicing coefficient indicates that the available computing resources are used to the maximum extent, and the candidate slicing coefficient is a factor of the maximum slicing coefficient, indicating that the corresponding slicing axis can be fully sliced. Therefore, if the candidate slicing coefficient of at least one slicing axis of the second subgraph is a factor of the maximum slicing coefficient, it can be indicated that the available computing resources can be used to the maximum extent when calculating the second subgraph, and the full slicing can be achieved.

[0060] Through step S102, it is calculated that there may be multiple segmentation axes of the second subgraph, and each segmentation axis corresponds to a maximum segmentation coefficient and a candidate segmentation coefficient.

[0061] If a candidate split coefficient of a split axis in the second subgraph is a factor of the corresponding maximum split coefficient, it means that the second subgraph can be merged, that is, the merging operation in step S101 is valid; and the split axis in the second subgraph whose candidate split coefficient is not a factor of the corresponding maximum split coefficient is deleted from the split information bound to the second subgraph.

[0062] If the candidate split coefficients of all split axes in the second subgraph are not factors of the corresponding maximum split coefficient, it means that there is no way to maximize the use of computing resources and to perform a full split on the second subgraph, and the second subgraph cannot be merged, that is, the merge operation in step S101 is invalid. Specifically, the merge operation in step S101 is a temporary "hypothetical merge". If it is determined through step S104 that the merge operation in step S101 is invalid, the generated split information and candidate split information of the second subgraph can be deleted. After determining that the merge operation is invalid, step S101 can be executed again to determine the first subgraph to be merged in the calculation graph.

[0063] In a possible implementation, in response to the candidate segmentation coefficients of at least two segmentation axes in the segmentation information of the second subgraph being factors of the corresponding maximum segmentation coefficient, an optimal core binding strategy is screened out according to a preset screening condition.

[0064] Optionally, the optimal core binding strategy of the second subgraph can be screened out according to the number of computing resources used by the segmentation methods corresponding to the candidate segmentation coefficients of the at least two segmentation axes. Among them, the number of computing resources used by the segmentation methods corresponding to the candidate segmentation coefficients of the segmentation axes in the optimal core binding strategy of the second subgraph is the largest. Through this process, the computational graph can maximize the utilization of computing resources during the actual operation process, thereby optimizing the computational graph operation process.

[0065] Optionally, the optimal core binding strategy for the second subgraph may be selected according to the priority of the preset split axis or the shape of the input tensor. The setting method of the priority is not limited here.

[0066] Finally, in response to the merge operation being effective, the second subgraph is re-determined as the first subgraph to be merged, and the above steps S101-S104 are repeated to continue attempting to merge. In response to the merge operation being invalid, the first subgraph to be merged is re-determined. In response to the first subgraph to be merged not existing, the scheduling information of the computation graph is determined according to the candidate segmentation information of each of the merged subgraphs, and the computation graph is scheduled for execution based on the scheduling information.

[0067] Figure 5 A schematic diagram of the grouping of computational graphs used in an embodiment of the present invention is shown in FIG. Figure 5As shown, after the computation graph is grouped by the above method, two computation subgraphs 510 and 520 are obtained. When executing the computation graph, the storage space corresponding to the input tensor is larger than the available computing resources allocated to the computation graph by the local memory. Therefore, the input tensor needs to be segmented according to the segmentation information of the computation subgraph 510. The segmentation information of each group after the grouping in the computation graph is determined by the above method. The optimal core binding strategy of the computation subgraph 510 is to divide it into three parts along the corresponding segmentation axis so that the number of computing resources used by the computation subgraph 510 is the largest. Therefore, the computation subgraph 510 divides the input tensor into three parts, and each tensor after segmentation is smaller than the available computing resources. The available computing resources process one tensor at a time, and the corresponding processing processes correspond to the computation subgraphs 511, 512 and 513 respectively. The processing results corresponding to each tensor are recorded during the processing process. After the three tensors are processed, the processing results corresponding to the three tensors are connected together by a character connection operator (concat) 514 as the output of the computation subgraph 510. Finally, the output of computation subgraph 510 is processed as the input of computation subgraph 520 .

[0068] The method of the embodiment of the present invention traverses a computational graph and merges at least two first subgraphs in the computational graph to obtain a second subgraph, wherein the first subgraph includes at least one operator, and the first subgraph is bound with segmentation information, and the segmentation information includes axis correspondence, at least one segmentation axis, and the maximum segmentation coefficient of the at least one segmentation axis, and the axis correspondence is the relationship between the input data and output data of the subgraph; according to the segmentation information of the first subgraph, the segmentation information is bound to the second subgraph; according to the number of available computing resources, the candidate segmentation information of the second subgraph is determined, and the candidate segmentation information includes: at least one segmentation axis and candidate segmentation coefficients of the at least one segmentation axis, wherein the segmentation method corresponding to the candidate segmentation coefficient can maximize the use of the number of available computing resources; in response to the candidate segmentation coefficient of at least one segmentation axis in the segmentation information of the second subgraph being a factor of the maximum segmentation coefficient, the merging operation is determined to be valid. In this way, operators can be grouped quickly, and by considering operator grouping and computing resource allocation in a unified manner, the repeated calculation process in the process of merging computing subgraphs is reduced, the compilation process of the computing graph is accelerated, and the computing resource utilization rate is improved.

[0069] Figure 6 FIG. 1 is a schematic diagram of a computing graph processing device according to an embodiment of the present invention. Figure 6 As shown, the processing device of the calculation graph includes:

[0070] A merging module 601 is used to traverse the computation graph and merge at least two first subgraphs in the computation graph to obtain a second subgraph, wherein the first subgraph includes at least one operator, and the first subgraph is bound with segmentation information, and the segmentation information includes an axis correspondence, at least one segmentation axis, and a maximum segmentation coefficient of the at least one segmentation axis, and the axis correspondence is a relationship between input data and output data of the subgraph;

[0071] A binding module 602, configured to bind segmentation information to the second subgraph according to the segmentation information of the first subgraph;

[0072] A first determining module 603 is configured to determine candidate segmentation information of the second subgraph according to the number of available computing resources, wherein the candidate segmentation information includes: at least one segmentation axis and candidate segmentation coefficients of the at least one segmentation axis, wherein the segmentation method corresponding to the candidate segmentation coefficient can maximize the use of the number of available computing resources;

[0073] The second determining module 604 is configured to determine that the merging operation is valid in response to a candidate slicing coefficient of at least one slicing axis in the slicing information of the second subgraph being a factor of the maximum slicing coefficient.

[0074] The device of the embodiment of the present invention traverses the computational graph and merges at least two first subgraphs in the computational graph to obtain a second subgraph, wherein the first subgraph includes at least one operator, and the first subgraph is bound with segmentation information, and the segmentation information includes axis correspondence, at least one segmentation axis and the maximum segmentation coefficient of the at least one segmentation axis, and the axis correspondence is the relationship between the input data and output data of the subgraph; according to the segmentation information of the first subgraph, the segmentation information is bound to the second subgraph; according to the number of available computing resources, the candidate segmentation information of the second subgraph is determined, and the candidate segmentation information includes: at least one segmentation axis and candidate segmentation coefficients of the at least one segmentation axis, wherein the segmentation method corresponding to the candidate segmentation coefficient can maximize the use of the number of available computing resources; in response to the candidate segmentation coefficient of at least one segmentation axis in the segmentation information of the second subgraph being a factor of the maximum segmentation coefficient, the merging operation is determined to be valid. In this way, operators can be grouped quickly, and by considering operator grouping and computing resource allocation in a unified manner, the repeated calculation process in the process of merging computing subgraphs is reduced, the compilation process of the computing graph is accelerated, and the computing resource utilization rate is improved.

[0075] Figure 7 Schematic diagram of an electronic device according to an embodiment of the present invention. Figure 7 As shown, Figure 7The electronic device shown is a general address query device, which includes a general computer hardware structure, which at least includes a processor 701 and a memory 702. The processor 701 and the memory 702 are connected via a bus 703. The memory 702 is suitable for storing instructions or programs executable by the processor 701. The processor 701 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 701 executes the instructions stored in the memory 702, thereby executing the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 703 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 704 and the display device and the input / output (I / O) device 705. The input / output (I / O) device 705 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 705 is connected to the system via an input / output (I / O) controller 706.

[0076] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices (equipment) or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] The present application is described with reference to flowcharts of methods, apparatuses (devices) and computer program products according to embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.

[0078] These computer program instructions may be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.

[0079] These computer program instructions may also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.

[0080] The electronic device of an embodiment of the present invention traverses a computational graph and merges at least two first subgraphs in the computational graph to obtain a second subgraph, wherein the first subgraph includes at least one operator, and the first subgraph is bound with segmentation information, and the segmentation information includes axis correspondence, at least one segmentation axis, and a maximum segmentation coefficient of the at least one segmentation axis, and the axis correspondence is a relationship between input data and output data of the subgraph; according to the segmentation information of the first subgraph, the segmentation information is bound to the second subgraph; candidate segmentation information of the second subgraph is determined according to the number of available computing resources, and the candidate segmentation information includes: at least one segmentation axis and candidate segmentation coefficients of the at least one segmentation axis, wherein the segmentation method corresponding to the candidate segmentation coefficient can maximize the use of the number of available computing resources; in response to the candidate segmentation coefficient of at least one segmentation axis in the segmentation information of the second subgraph being a factor of the maximum segmentation coefficient, the merging operation is determined to be valid. In this way, operators can be grouped quickly, and by considering operator grouping and computing resource allocation in a unified manner, the repeated calculation process in the process of merging computing subgraphs is reduced, the compilation process of the computing graph is accelerated, and the computing resource utilization rate is improved.

[0081] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above method embodiments.

[0082] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by specifying relevant hardware through a program, and the program is stored in a storage medium, including several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0083] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for processing a computational graph, characterized in that: The method comprises: Traversing a computation graph, merging at least two first subgraphs in the computation graph to obtain a second subgraph, wherein the first subgraph includes at least one operator, and the first subgraph is bound with segmentation information, the segmentation information includes an axis correspondence, at least one segmentation axis, and a maximum segmentation coefficient of the at least one segmentation axis, and the axis correspondence is a relationship between input data and output data of the subgraph; Binding segmentation information to the second subgraph according to the segmentation information of the first subgraph; Determining candidate segmentation information of the second subgraph according to the number of available computing resources, the candidate segmentation information comprising: at least one segmentation axis and candidate segmentation coefficients of the at least one segmentation axis, wherein the segmentation method corresponding to the candidate segmentation coefficient can maximize the use of the number of available computing resources; In response to a candidate segmentation coefficient of at least one segmentation axis in the segmentation information of the second subgraph being a factor of the maximum segmentation coefficient, it is determined that the merging operation is valid.

2. The method according to claim 1, characterized in that The method for determining the at least two first subgraphs comprises: Any adjacent sequential subgraphs in the computation graph are determined as at least two first subgraphs to be merged.

3. The method according to claim 1, characterized in that The method for determining the at least two first subgraphs comprises: A plurality of branch subgraphs including a minimum branch subgraph in the computation graph are determined as at least two first subgraphs to be merged.

4. The method according to any one of claims 1 to 3, characterized in that The step of binding the segmentation information to the second subgraph according to the segmentation information of the first subgraph includes: Determine an axis correspondence relationship and at least one segmentation axis of the second subgraph according to an axis correspondence relationship between at least two adjacent first subgraphs in the second subgraph; According to each segmentation axis of the first subgraph and the corresponding maximum segmentation coefficient, a maximum segmentation coefficient of each segmentation axis of the second subgraph is determined.

5. The method according to claim 4, characterized in that The determining, according to the axis correspondence between at least two adjacent first subgraphs in the second subgraph, the axis correspondence relationship and at least one segmentation axis of the second subgraph comprises: Determine a common segmentation axis between the at least two first subgraphs by traversing the axis correspondence relationship between the at least two first subgraphs in reverse order; The common segmentation axis is determined as the segmentation axis of the second subgraph.

6. The method according to claim 5, characterized in that The determining, according to each of the segmentation axes of the first subgraphs and the corresponding maximum segmentation coefficients, the maximum segmentation coefficient of each segmentation axis of the second subgraph comprises: Determine the greatest common divisor of the maximum segmentation coefficients of the common segmentation axes of all the first subgraphs; In response to the greatest common factor being greater than one, the greatest common factor is determined as a maximum segmentation coefficient of a corresponding segmentation axis in the second subgraph.

7. The method according to claim 2, characterized in that The method further comprises: In response to the merging operation being valid, determining the second subgraph as the first subgraph; In response to the merging operation being invalid, re-determining a first subgraph to be merged; In response to the absence of the first subgraph to be merged, determining the scheduling information of the computation graph according to the candidate segmentation information of each merged subgraph; The computation graph is scheduled for execution based on the scheduling information.

8. A computing graph processing device, characterized in that: The device comprises: A merging module, configured to traverse a computation graph and merge at least two first subgraphs in the computation graph to obtain a second subgraph, wherein the first subgraph includes at least one operator, and the first subgraph is bound with segmentation information, and the segmentation information includes an axis correspondence, at least one segmentation axis, and a maximum segmentation coefficient of the at least one segmentation axis, and the axis correspondence is a relationship between input data and output data of the subgraph; A binding module, configured to bind segmentation information to the second subgraph according to the segmentation information of the first subgraph; A first determining module is configured to determine candidate segmentation information of the second subgraph according to the number of available computing resources, wherein the candidate segmentation information includes: at least one segmentation axis and candidate segmentation coefficients of the at least one segmentation axis, wherein the segmentation method corresponding to the candidate segmentation coefficient can maximize the use of the number of available computing resources; The second determining module is configured to determine that the merging operation is valid in response to a candidate segmentation coefficient of at least one segmentation axis in the segmentation information of the second subgraph being a factor of the maximum segmentation coefficient.

9. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images