Calculation graph processing method and device, equipment and storage medium

Abstract

The invention discloses a computational graph processing method and device, equipment and a storage medium, and the method comprises the steps: obtaining a computational graph comprising P dynamic inputs and a bucket-dividing mark of the computational graph, the bucket-dividing mark being used for indicating a group to which each dynamic input belongs and a bucket-dividing mode of each group, the dynamic shapes of the dynamic inputs in the same group have the same value range, and the value range of the dynamic shapes of the dynamic inputs is subjected to barrel separation processing according to the barrel separation mode of the corresponding group; furthermore, according to the indication of the bucket dividing mark, the value range of the dynamic shape of the P dynamic inputs can be subjected to bucket dividing processing to obtain a plurality of buckets, and the calculation graph is subjected to static compiling based on the numerical value in each bucket, so that after the input data of the calculation graph is obtained, the calculation graph is executed on the input data based on the compiling results corresponding to the plurality of buckets, and the calculation graph is obtained. Therefore, the overhead of the processor is saved, and the execution efficiency of the computational graph is improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, specifically to the field of artificial intelligence technology, and in particular to a method, device, equipment and storage medium for processing a computational graph. Background Art

[0002] With the development of artificial intelligence (AI) technology, computation graphs have been widely used; the so-called computation graph is a graph structure used to represent mathematical calculations or program execution. It is particularly important in the implementation of AI models and can be used to describe the calculation process in AI models. The input of the computation graph can use a dynamic shape. The so-called dynamic shape refers to a shape that can change dynamically, which can indicate that the shape of the data input to the computation graph each time is not fixed, that is, the computation graph can support the input of data of various shapes. Such a computation graph can be called a dynamic shape graph.

[0003] For dynamic shape graphs, a common processing method is compile execution, that is, dynamically compile the dynamic shape graph in advance to obtain a compilation result that supports dynamic input, so that when the dynamic shape graph needs to be executed, the input data of the dynamic shape graph is obtained, and the dynamic shape graph is executed on the input data based on the compilation result. Since the shape of the input data is often different each time it is executed, such a processing method easily causes a large processor overhead, resulting in low execution efficiency. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, device and storage medium for processing a computational graph, which can save processor overhead and improve the execution efficiency of the computational graph.

[0005] On the one hand, an embodiment of the present application provides a method for processing a computational graph, the method comprising:

[0006] Obtain a computation graph, wherein the computation graph includes P dynamic inputs, where P is a positive integer; wherein each dynamic input has a dynamic shape, and each dynamic shape has a value range;

[0007] Obtaining a bucket mark of the computation graph, the bucket mark being used to indicate: the group to which each dynamic input belongs and the bucketing method of each group; wherein the dynamic shapes of each dynamic input in the same group have the same value range, and the value range of the dynamic shape of each dynamic input is bucketed according to the bucketing method of the corresponding group;

[0008] According to the indication of the bucket mark, the value ranges of the dynamic shapes of the P dynamic inputs are bucketed to obtain a plurality of buckets; each bucket includes P values, and different values ​​of the P values ​​are selected from different value ranges;

[0009] Static compile the computation graph based on the value in each bucket to obtain a compilation result corresponding to each bucket;

[0010] After the input data of the calculation graph is acquired, the calculation graph is executed on the input data based on the compilation results corresponding to the multiple buckets to obtain a data calculation result of the input data.

[0011] On the other hand, an embodiment of the present application provides a computing graph processing device, the device comprising:

[0012] An acquisition unit, used to acquire a computation graph, wherein the computation graph includes P dynamic inputs, where P is a positive integer; wherein each dynamic input has a dynamic shape, and each dynamic shape has a value range;

[0013] The acquisition unit is further used to acquire a bucket mark of the computation graph, wherein the bucket mark is used to indicate: the group to which each dynamic input belongs and the bucketing method of each group; wherein the dynamic shapes of the dynamic inputs in the same group have the same value range, and the value range of the dynamic shape of each dynamic input is bucketed according to the bucketing method of the corresponding group;

[0014] a processing unit, configured to perform bucket processing on the value ranges of the dynamic shapes of the P dynamic inputs according to the indication of the bucket marking, to obtain a plurality of buckets; each bucket includes P values, and different values ​​among the P values ​​are selected from different value ranges;

[0015] The processing unit is further used to statically compile the calculation graph based on the value in each bucket to obtain a compilation result corresponding to each bucket;

[0016] The processing unit is further configured to, after acquiring the input data of the calculation graph, execute the calculation graph on the input data based on the compilation results corresponding to the multiple buckets to obtain a data calculation result of the input data.

[0017] In another aspect, an embodiment of the present application provides a computer device, the computer device comprising an input interface and an output interface, the computer device further comprising:

[0018] processor and computer storage media;

[0019] Among them, the processor is suitable for implementing one or more instructions, and the computer storage medium stores one or more instructions, and the one or more instructions are suitable for the processor to load and execute the above-mentioned calculation graph processing method.

[0020] On the other hand, an embodiment of the present application provides a computer storage medium, which stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the above-mentioned calculation graph processing method.

[0021] On the other hand, an embodiment of the present application provides a computer program product, which includes one or more instructions; when one or more instructions in the computer program product are executed by a processor, the above-mentioned method for processing the computational graph is implemented.

[0022] The embodiment of the present application can obtain a calculation graph including P dynamic inputs and a bucket mark of the calculation graph, wherein the bucket mark is used to indicate the group to which each dynamic input belongs and the bucket method of each group, and the dynamic shapes of each dynamic input in the same group have the same value range, and the value range of the dynamic shape of each dynamic input is bucketed according to the bucket method of the corresponding group; it can be seen that the embodiment of the present application sets the bucket mark in the form of grouping, so that the bucket method of the value range of the dynamic shape of each dynamic input in the same group can be recorded with one piece of information, which can avoid occupying some memory to separately record the bucket method of the value range of the dynamic shape of each dynamic input, thereby effectively saving processor memory. Furthermore, before executing the calculation graph, the value range of the dynamic shape of P dynamic inputs can be bucketed as indicated by the bucket mark to obtain multiple buckets, and the calculation graph can be statically compiled based on the value in each bucket. After obtaining the input data of the calculation graph, the calculation graph is executed on the input data based on the compilation results corresponding to the multiple buckets to obtain the data calculation results of the input data. This can avoid the situation where processor resources are consumed to compile the calculation graph every time the input data is obtained, thereby effectively saving the overhead generated by the processor and shortening the execution time of the calculation graph, thereby improving the execution efficiency of the calculation graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1This is a schematic diagram of a process of executing a dynamic computation graph by static bucket compilation provided by an embodiment of the present application;

[0025] Figure 2 It is a flowchart of a method for processing a computational graph provided in an embodiment of the present application;

[0026] Figure 3 It is a schematic diagram of a bucket processing provided in an embodiment of the present application;

[0027] Figure 4 It is a flowchart of a method for processing a computational graph provided by another embodiment of the present application;

[0028] Figure 5a is a schematic diagram of extended data provided by an embodiment of the present application;

[0029] Figure 5b is a schematic diagram of an expanded data and corresponding data calculation results provided in an embodiment of the present application;

[0030] Figure 5c It is a schematic diagram of a generation process of a target shape calculation formula provided in an embodiment of the present application;

[0031] Figure 5d is a schematic diagram of a generation process of another target shape calculation formula provided in an embodiment of the present application;

[0032] Figure 5e It is a schematic diagram of a framework of a method for processing a computational graph provided in an embodiment of the present application;

[0033] Figure 6 It is a structural schematic diagram of a computing graph processing device provided in an embodiment of the present application;

[0034] Figure 7 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0036] In the embodiments of the present application, a computational graph is a graph structure that can be used to describe the computational process in an AI model; the AI ​​model here refers to a model built based on AI technology, and AI technology refers to the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Exemplarily, the AI ​​model can be, for example: an ordinary neural network model (such as a convolutional neural network model), a large model, etc.; the so-called large model can be a large language model (Large Language Model), which specifically refers to a deep neural network with large parameters.

[0037] Specifically, the computation graph may include, but is not limited to: Q input nodes, R operator nodes, and at least one output node, where Q and R are positive integers. The input node is used to represent the input of the computation graph, which is responsible for inputting the data to be calculated into the computation graph; the operator node is used to represent the operator, which is a general term for a certain mathematical operation, and an operator can represent at least one operation in a network layer in the AI ​​model; the output node is used to represent the output of the computation graph, which is responsible for outputting the execution result of the computation graph (i.e., the data calculation result obtained after calculating the input data). It can be seen that the computation graph may include Q inputs, R operators, and at least one output.

[0038] Moreover, the input and output of the computation graph may have their own shapes. The input shape specifically refers to the shape of the data supported by the input, and the output shape specifically refers to the shape of the data supported by the output. The so-called shape is an array used to represent the composition structure of data, which may specifically include the value of at least one dimension to describe the size (the number of data elements) of the data in at least one dimension in the multidimensional space; the data here may be, for example, images, voices, texts, etc., which are not limited to this. For example, if the shape of data A is <23xf32>, then the shape may indicate that data A is a one-dimensional data, which has 1 dimension in the multidimensional space and contains 23 data elements in the dimension, and the format of each data element is f32 (i.e., 32-bit floating point number); for another example, if the shape of data B is <23x10xf32>, then the shape may indicate that data B is a two-dimensional data, which has 2 dimensions in the multidimensional space, and contains 23 data elements in the first dimension and 10 data elements in the second dimension, and the format of each data element is f32.

[0039] Furthermore, the above shape can be specifically divided into dynamic shape and static shape; the so-called dynamic shape refers to a shape that contains dynamic dimensions, and the static shape refers to a shape that does not contain dynamic dimensions. Among them, the dynamic dimension refers to the dimension whose value supports dynamic change, that is, the value of the dynamic dimension is unknown, and the target value (such as -1) can be used in the shape to represent the unknown value. In other words, when the value of one or several dimensions in the shape is -1, it means that the values ​​of these dimensions are unknown and can change dynamically. These dimensions can be called dynamic dimensions. In this case, the shape can be called a dynamic shape. For example, the shape [-1, 30] is a dynamic shape; correspondingly, when the values ​​of each dimension in the shape are not -1, it means that the values ​​of each dimension are known and fixed. In this case, the shape is a static shape. For example, the shape [1, 30] is a static shape.

[0040] Based on the above-mentioned shape classification, the calculation graph can be specifically divided into a dynamic shape graph (dynamic calculation graph) and a static shape graph (static calculation graph). Among them, a dynamic shape graph refers to a calculation graph in which there is at least one input whose shape is a dynamic shape; that is, when the shape of at least one input in the calculation graph is a static shape, the calculation graph can be called a dynamic shape graph. Correspondingly, a static shape graph refers to a calculation graph in which the shapes of all inputs are static shapes; that is, when the shapes of all inputs in the calculation graph are static shapes, the calculation graph can be called a static shape graph. It should be noted that, for the sake of convenience, the input with dynamic shape will be referred to as dynamic input, and the input with static shape will be referred to as static input; and the dynamic shape of each dynamic input has its own value range.

[0041] In AI scenarios, for dynamic shape graphs, since the input shapes involved in each execution are often different, it is easy for the operators in the dynamic shape graph to add a lot of additional judgments during the implementation process; and most operators that support multiple inputs usually have requirements for the shape of each input, such as the add operator, which requires the shape of the two inputs to be the same, or the shape of one input can be broadcasted to be the same as the shape of another input. This easily leads to the fact that in actual calculations, in addition to processing the operators in the calculation graph, the processor also needs to add and process some shape calculation-related operators to ensure correctness. However, for static shape graphs, since the input shape of each operator is known, there is no need to involve the two additional overheads involved in the execution of the aforementioned dynamic shape graph (i.e., the overhead required for additional judgment and the overhead required for processing additional operators), which can make the running time of static shape graphs much faster than that of dynamic shape graphs, that is, the execution efficiency of static shape graphs is higher than that of dynamic shape graphs.

[0042] Research has shown that: for dynamic shape graphs and static shape graphs with the same calculation logic, if the dynamic shape graph needs to be executed, the shape of the input data of the dynamic shape graph is expanded to be the same as the shape of the input of the static shape graph, so that the static shape graph is executed using the expanded input data, and the corresponding output shape (that is, the shape of the data calculation result obtained after the dynamic shape graph is calculated) is predicted based on the shape of the input data, so that the data calculation result of the input data is extracted from the execution result of the static shape graph based on the predicted output shape, so that the extracted data calculation result can be consistent with the data calculation result calculated by the dynamic shape graph.

[0043] For example, if a dynamic shape graph has two inputs and the shapes of the two inputs are <-1xf32> and <-1xf32>, where the value range of the dimension where -1 is located is [1, 10]; there is also a static shape graph with two inputs and the shapes of the two inputs are <10xf32> and <10xf32>; and the calculation logic in the dynamic shape graph and the static shape graph is add. Then when the dynamic shape graph needs to be executed, the shape of the input data of the dynamic shape graph can be expanded to <10xf32>, and the static shape graph can be executed using this expanded input data, and the output shape can be predicted based on the shape of the actual input data, and the data calculation result of the input data can be extracted from the execution result of the static shape graph based on the predicted output shape, so that the extracted data calculation result is consistent with the data calculation result obtained by calculating the input data through the dynamic shape graph.

[0044] Based on the above research results, the embodiment of the present application proposes a method for processing a calculation graph for a dynamic shape graph, which can accelerate the execution of the dynamic shape graph by static bucket compilation. Among them, the static bucket compilation method refers to: bucketing the value range of the dynamic shape possessed by at least one input of the dynamic shape graph to obtain multiple buckets, each bucket includes at least one numerical value and different numerical values ​​are selected from the value ranges of different dynamic shapes, and statically compile the dynamic shape graph based on the numerical values ​​in each bucket to obtain the compilation results corresponding to each bucket. It can be understood that static compilation refers to compilation using static shape; based on this definition, the principle of statically compiling a dynamic shape graph based on the numerical value in any bucket is roughly as follows: determine the specific values ​​of each dimension in the dynamic shape possessed by at least one input of the dynamic shape graph based on the numerical value in any bucket, obtain the static shape of the corresponding bucket, and then compile the dynamic shape graph using the static shape of the bucket.

[0045] Specifically, the method can compile the dynamic shape graph in advance by static bucket compilation before executing the dynamic shape graph, and obtain compilation results corresponding to multiple buckets, and different compilation results correspond to the static shapes of different buckets, so that after obtaining the actual input data of the dynamic shape graph, a compilation result can be selected from multiple compilation results as the target compilation result, and the static shape corresponding to the target compilation result is greater than or equal to the shape of the actual input data, and the dynamic shape graph is executed on the actual input data based on the target compilation result to convert the input of the dynamic shape into the input of the static shape, and convert the execution of the dynamic shape graph into the execution of the static shape graph, so as to accelerate the execution of the dynamic shape graph, prompt the execution efficiency of the dynamic shape graph, and avoid the additional overhead introduced by the dynamic shape processing, thereby saving the processor overhead and improving the operating efficiency of the processor. It can be understood that: any compilation result can be understood as an executable dynamic shape graph with a known shape, and the known shape is the static shape corresponding to the compilation result; therefore, when executing the dynamic shape graph on the actual input data based on any compilation result, the essence is to use the actual input data to execute the corresponding compilation result.

[0046] Combine the following Figure 1 , a specific example is used to illustrate the general process of executing a dynamic shape graph through static bucket compilation:

[0047] Assume that a dynamic shape graph has at least two inputs with dynamic shapes, such as the dynamic shape of Arg0 (the first input) is <-1x1x16xf32>, the dynamic shape of Arg1 (the second input) is <-1x1280xf32>, and the value range of the first dimension in these dynamic shapes is 1 to 100. If the value range is bucketed by adopting an exponential bucketing strategy with base 2, 1, 2, 4, 8, ... 64, 100 can be selected from the value range as the values ​​in different buckets to obtain multiple buckets, and the values ​​in each bucket are used as the actual values ​​(specific values) of the dynamic values ​​of the first dimension in the dynamic shapes of each input to obtain the static shape of each bucket. For example, the static shapes of the first bucket include <1x1x16xf32> corresponding to Arg0, <1x1280xf32> corresponding to Arg1, etc., and the static shapes of the second bucket include <2x1x16xf32> corresponding to Arg0, <2x1280xf32> corresponding to Arg1, etc.

[0048] Furthermore, the dynamic shape graph can be statically compiled based on the static shape of each bucket to obtain the compilation result corresponding to each bucket; and in actual operation, according to the shape size of each actual input data, the compilation result corresponding to the bucket whose static shape is closest to and not smaller than the shape of the actual input data is selected for execution to obtain the data calculation result of the actual input data. For example, if the value of the first dimension in the shape of the actual input data is 5, that is, the shape of the actual input data includes <5x1x16xf32> corresponding to Arg0, <5x1280xf32> corresponding to Arg1, etc., then it can be determined that the static shape (<8x1x16xf32>, <8x1280xf32>, etc.) of the compilation result corresponding to the bucket containing the value 8 is closest to and not smaller than the shape of the actual input data, so the compilation result corresponding to the bucket containing the value 8 can be selected for execution to obtain the data calculation result of the actual input data. Similarly, if the value of the first dimension in the shape of the actual input data is 1, that is, the shape of the actual input data includes <1x1x16xf32> corresponding to Arg0 and <1x1280xf32> corresponding to Arg1, etc., then the compilation result corresponding to the bucket containing the value 1 can be selected for execution; if the value of the first dimension in the shape of the actual input data is 3, that is, the shape of the actual input data includes <3x1x16xf32> corresponding to Arg0 and <3x1280xf32> corresponding to Arg1, etc., then the compilation result corresponding to the bucket containing the value 4 can be selected for execution, and so on.

[0049] Based on the above description, the following points need to be explained:

[0050] (1) The method proposed in the embodiment of the present application can effectively reduce the processor overhead when the dynamic shape graph is running by converting the calculation of the dynamic shape graph into the calculation of the static shape graph, thereby improving the operating efficiency of the processor. Although the bucketing process itself may cause some redundant calculations, the overhead of this part of redundant calculations can be reduced by reasonable bucketing, and finally the execution efficiency of the dynamic shape graph can be improved.

[0051] (2) The embodiment of the present application does not limit the application scenario of the method proposed by the embodiment of the present application, that is, the method proposed by the embodiment of the present application can be applied to various AI scenarios. For example, in the reasoning scenario, multiple input data are often grouped into batches, that is, a batch contains multiple input data, and a batch is provided to the neural network model for execution; since the number of input data contained in a batch is often not fixed, from the execution perspective of the calculation graph corresponding to the neural network model, it can be considered that this is a case where the input adopts a dynamic shape, and the number of input data contained in the batch usually has a range of values, so the method proposed by the embodiment of the present application can be used to accelerate the execution of the corresponding calculation graph. For example, in the large model scenario, since the number of tokens (the smallest unit or basic element of text processing, such as a word, a phrase, a punctuation mark or a character, etc.) in the input text is often not fixed, but there is often an upper limit, so this scenario also belongs to the case where the input of the calculation graph corresponding to the large model has a dynamic shape and the dynamic shape has a range of values, so the method proposed by the embodiment of the present application can be used to accelerate the execution of the corresponding calculation graph.

[0052] (3) The embodiments of the present application do not limit the execution subject of the method proposed in the embodiments of the present application. For example, the method proposed in the embodiments of the present application can be executed by a computer device, which can be a terminal or a server; or, the method proposed in the embodiments of the present application can also be executed by a terminal and a server together, which is not limited. Among them, the terminal can be a smart phone, a computer (such as a tablet computer, a laptop computer, a desktop computer, etc.), a smart wearable device (such as a smart watch, smart glasses), an intelligent voice interaction device, a smart home appliance (such as a smart TV), a car terminal or an aircraft, etc.; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms, etc.

[0053] (4) In the embodiments of the present application, if relevant data such as user information is involved, when any method embodiment proposed in the embodiments of the present application is applied to a specific product or technology, these relevant data are collected with the user's permission or consent, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant region.

[0054] Combine the following Figure 2The method flow chart shown in the figure illustrates the specific implementation process of the method for processing the computational graph proposed in the embodiment of the present application; in the embodiment of the present application, the method for processing the computational graph executed by a computer device is used as an example for illustration. Figure 2 As shown, the processing method of the computation graph may generally include S201-S205:

[0055] S201, obtain a computation graph.

[0056] In the embodiment of the present application, the calculation graph obtained by step S201 specifically refers to a dynamic shape graph (i.e., a dynamic calculation graph). The calculation graph may include multiple nodes, and the multiple nodes may specifically include: P input nodes, R operator nodes, and at least one output node, where P and R are both positive integers; wherein an input node represents a dynamic input of the calculation graph, an operator node represents an operator in the calculation graph, and an output node represents an output of the calculation graph.

[0057] It can be seen that the calculation graph mentioned in the embodiment of the present application may include P dynamic inputs. It is understandable that the calculation graph may include a total of Q input nodes (Q is a positive integer), and the P input nodes mentioned above for representing P dynamic inputs belong to all or part of the Q input nodes, that is, the calculation graph may include a total of Q inputs (Q is a positive integer), if each of the Q inputs is a dynamic input, then in this case, P = Q (that is, the value of P is equal to the total number of inputs included in the calculation graph); or, if some of the Q inputs included in the calculation graph are dynamic inputs and the other part of the inputs are static inputs, then in this case, P < Q (the value of P is less than the total number of inputs included in the calculation graph).

[0058] Among them, each dynamic input has a dynamic shape, and the dynamic shape of any dynamic input may include numerical values ​​of N dimensions, and there is a dynamic dimension among the N dimensions (that is, the dimension whose numerical value supports dynamic change), and N is a positive integer. It is understandable that the number of dimensions in the dynamic shapes of different dynamic inputs (that is, the value of N) may be the same or different, and there is no limitation on this. For example, the dynamic shape of the first dynamic input includes numerical values ​​of 3 dimensions (that is, the number of dimensions in the first dynamic shape is 3), and the dynamic shape of the second dynamic input includes the value of 1 dimension (that is, the number of dimensions in the dynamic shape of the second dynamic input is 1. In this case, the number of dimensions in the first dynamic input is different from the number of dimensions in the second dynamic input.

[0059] Furthermore, each dynamic shape has a value range. Specifically, the value range of a dynamic shape may refer to: the value range of a dynamic dimension in a dynamic shape; for example, if the dynamic shape is <-1x10xf32>, the dynamic dimension is the first dimension and the value range of the first dimension is 0-100, then the value range of the dynamic shape is 0-100. Alternatively, the value range of a dynamic shape may refer to: a range consisting of a plurality of preset static shapes, any static shape in the range is obtained by setting the value of the dynamic dimension in the dynamic shape to a preset value; for example, if the dynamic shape is still <-1x10xf32>, the preset values ​​may include 1, 5, 8, 10, etc., then the value range of the dynamic shape may include the following static shapes: <1x10xf32>, <5x10xf32>, <8x10xf32>, <10x10xf32>, etc.

[0060] S202, obtaining bucket labels of the computation graph.

[0061] For a given computational graph (dynamic shape graph), in order to use static bucket compilation to accelerate the execution of the computational graph, it is necessary to obtain the bucket mark of the computational graph, so as to know the bucket method of the value range of the dynamic shape of each dynamic input of the computational graph through the bucket mark, so that the value range of each dynamic shape can be accurately bucketed based on the corresponding bucket method.

[0062] In a specific implementation, the embodiment of the present application proposes a bucket marking method for shape groups to simply and efficiently mark how to bucket the value ranges of dynamic shapes of each dynamic input of a dynamic shape graph. Specifically, the principle of the bucket marking method is as follows:

[0063] s11, grouping each dynamic input based on the value range of the dynamic shape of each dynamic input to obtain the group to which each dynamic input belongs. Specifically, dynamic inputs corresponding to dynamic shapes with the same value range can be grouped into the same group to obtain the group to which each dynamic input belongs.

[0064] s12, configure a bucketing method for each group. Specifically, for any group, a bucketing method can be configured for the group based on business needs or experience values. The bucketing method can be, for example, an exponential bucketing method based on a first value (such as value 2), a multiple bucketing method based on a second value (such as value 5), and so on. Alternatively, a bucketing method can be configured for the corresponding group based on the historical value distribution of the dynamic shape of each dynamic input in the group, so that after the value range of the dynamic shape of the corresponding dynamic input is bucketed based on the bucketing method, the values ​​in each bucket obtained are historically frequently appearing values, so that the static shapes determined based on the values ​​in each bucket are historically frequently appearing shapes, and these shapes are highly likely to be the same or similar to the shapes of the actual input data of the calculation graph, which can reduce the subsequent expansion of the input data due to shape differences to a certain extent, thereby saving the processing resources of the processor and improving the performance of the processor.

[0065] s13, based on the groups to which each dynamic input belongs and the bucketing method of each group, generate the bucketing mark of the computation graph. Specifically, the corresponding dynamic input can be marked based on the groups to which each dynamic input belongs, to obtain the group mark of the corresponding dynamic input, and the corresponding group can be marked based on the bucketing method of each group, to obtain the attribute mark of the corresponding group, so as to construct the bucketing mark of the computation graph using the group mark of each dynamic input and the attribute mark of each group.

[0066] Based on the description of s11-s13 above, it can be known that the bucket mark generated by the bucket mark method of shape group can be used to indicate: the group to which each dynamic input belongs and the bucket method of each group; it may specifically include: the group mark of each dynamic input and the attribute mark of at least one group. Among them, the dynamic shapes of each dynamic input in the same group have the same value range, and the value range of the dynamic shape of each dynamic input is bucketed according to the bucket method of the corresponding group. It can be seen that the embodiment of the present application can realize the bucket method of the value range of the dynamic shape of each dynamic input in the same group by using one piece of information to record, which can avoid occupying some memory to separately record the bucket method of the value range of the dynamic shape of each dynamic input, thereby effectively saving the memory of the processor.

[0067] The group to which the dynamic input belongs may refer to: the group to which the dynamic dimension in the dynamic shape of the dynamic input belongs; in this case, the group tag of the dynamic input is used to indicate: the dynamic dimension in the dynamic shape of the dynamic input, and the group to which the corresponding dynamic dimension belongs. For example, the group tag of the dynamic input may include but is not limited to at least one of the following information: a node identifier (such as a node name) for representing the input node of the dynamic input, the output information of the input node, the dynamic shape, and the group to which the dynamic dimension in the dynamic shape belongs, etc. Exemplarily, the group tag of the dynamic input may be expressed as follows:

[0068] {name:NODE_NAME,index:0,shape:{-1,10},shape_group:{0,-1}}

[0069] name: represents the node name field, which is used to store the node name of the input node.

[0070] index: represents the output sequence number field, which is used to store the sequence number of the output of the input node, so as to indicate the output number of the input node through the stored value. Index: 0 represents the first output.

[0071] Shape: represents the shape field, which is used to store the shape of the input node output; -1 indicates that the value of the dimension is dynamically changing, and other values ​​indicate that the value of the dimension is static. It can be understood that for a computational graph, the data output by the input node is the input data of the computational graph, so the shape of the input node output is the shape of the input of the computational graph.

[0072] shape_group: represents the group field, which is used to store the grouping information of shape, which specifically includes the grouping indication value of each dimension in shape; among them, when the grouping indication value of any dimension is a specified value (such as -1), it means that the dimension is not grouped. When the grouping indication information of any dimension is a non-specified value (that is, a value other than the specified value), it indicates the group to which the dimension belongs. If the grouping indication information is 0, it means that the corresponding dimension belongs to the 0th group.

[0073] In addition, a group corresponds to a value range, and the attribute tag of the group is used to indicate: the value range corresponding to the group and the bucketing method of the group. For example, the attribute tag of the group may include but is not limited to at least one of the following information: group, bucketing method (such as bucketing algorithm and data required by bucketing algorithm), maximum value and minimum value of the value range, etc. Exemplarily, the attribute tag of the group can be expressed as follows:

[0074] {shape_group: 0, algo: exponent, value: 2, min: 1, max: 100}

[0075] shape_group: represents the group field, which is used to store the group and indicate the group number through the stored group;

[0076] algo: represents the algorithm field, which is used to store the bucketing algorithm. exponent represents the exponential bucketing algorithm.

[0077] value: indicates the algorithm basis field, which is used to store the data required by the bucketing algorithm. For example, the 2 stored therein indicates that the base of the exponential bucketing algorithm is 2.

[0078] max_shape: represents the maximum value field of the range, which is used to store the maximum value of the value range, thereby indicating the maximum value of the shape.

[0079] min_shape: represents the maximum value field of the range, which is used to store the minimum value of the value range, thereby indicating the minimum value of the shape.

[0080] Based on the above description, for example, the computation graph may contain the following input nodes:

[0081] input0: <-1x10x16xf32>

[0082] input1: <-1x2xf32>

[0083] input2: <-1xf32>

[0084] Then, the bucket marking of the computation graph can be as follows:

[0085] {name: input0, index: 0, shape: {-1, 10, 16}, shape_group: {0, -1, -1}}

[0086] {name: input1, index: 0, shape: {-1, 2}, shape_group: {0, -1}}

[0087] {name: input2, index: 0, shape: {-1}, shape_group: {0}}

[0088] {shape_group: 0, algo: exponent, value: 2, min: 1, max: 100}

[0089] Through the above bucket marking, we can know that the first dimension in the shape of input0 (i.e. the 0th input node), the first dimension in the shape of input1 (i.e. the 1st input node), and the first dimension in the shape of input2 (i.e. the 2nd input node) are all in group 0. And based on the attribute marking of group 0, we can know that the minimum value of the value range of the first dimension in the shape of each input node is 1 and the maximum value is 100, that is, the value range is 1-100; and each value range is bucketed according to the exponential bucketing algorithm with base 2, that is, bucketing is performed according to 1, 2, 4, 8, 16, 32, 64, 100 as the bucket value.

[0090] It should be noted that in the above bucket marking, the bucketing algorithm can be configured as needed, and the embodiment of the present application does not limit this. In addition, the above is only an exemplary description of the specific content of the bucket marking, and does not limit this; for example, in other embodiments, each dynamic input group mark in the bucket marking can respectively include the value range of the corresponding dynamic input. In this case, the attribute mark of the group does not need to indicate the value range corresponding to the group.

[0091] S203 , performing bucket processing on the value ranges of the dynamic shapes of the P dynamic inputs according to the instructions of the bucket marks, to obtain a plurality of buckets.

[0092] In a specific implementation, the computer device can determine the bucketing method of the value range of the dynamic shape of the p-th dynamic input (p∈[1, P]) according to the indication of the bucketing mark, and then bucket the value range of the dynamic shape of the p-th dynamic input based on the determined bucketing method to obtain multiple numerical values ​​corresponding to the p-th dynamic input; after obtaining the multiple numerical values ​​corresponding to each dynamic input based on this principle, the multiple numerical values ​​corresponding to the P dynamic inputs can be combined to obtain multiple buckets.

[0093] Each bucket includes P values, and different values ​​in the P values ​​are selected from different value ranges. Figure 3 As shown: there are 2 dynamic inputs in total. The value range of the dynamic shape of the first dynamic input is bucketed to obtain two values, value 1 and value 2. The value range of the dynamic shape of the second dynamic input is bucketed to obtain two values, value 4 and value 8. Then, by combining the values ​​corresponding to different dynamic inputs, 4 buckets can be obtained: bucket 1 includes value 1 and value 4, bucket 2 includes value 1 and value 8, bucket 3 includes value 2 and value 4, and bucket 4 includes value 2 and value 8.

[0094] S204, statically compile the computation graph based on the values ​​in each bucket to obtain a compilation result corresponding to each bucket.

[0095] In a specific implementation, if the value range of each dynamic shape of the dynamic input is a range composed of multiple preset static shapes, then any value in each bucket obtained in this case is a static shape. Then when the computer device executes S204, it can directly compile the calculation graph based on each static shape in each bucket to obtain the compilation result corresponding to each bucket. It is understandable that the embodiment of the present application does not limit the specific method of compiling the calculation graph based on the static shape. For example, an existing compiler can be used to compile the calculation graph based on the static shape.

[0096] In another specific implementation, if the value range of the dynamic shape of each dynamic input is the value range of the dynamic dimension in the corresponding dynamic shape, then any value in each bucket obtained in this case is a specific value of the dynamic dimension. Then when the computer device executes S204, it can traverse multiple buckets to determine the kth bucket currently traversed, where k is a positive integer and is less than or equal to the total number of buckets; each value in the kth bucket is used as the value of the dynamic shape of the corresponding dynamic input to obtain the static shape of the kth bucket; wherein, when the pth value in the kth bucket is selected from the value range of the dynamic shape of the pth dynamic input, the pth value corresponds to the pth dynamic input, p∈[1,P]; and the shape determined after the pth value is used as the value of the dynamic shape of the pth dynamic input can be called the static shape of the pth dynamic input, which shows that the static shape of the kth bucket includes a static shape of each of the P dynamic inputs. Furthermore, after multiple buckets are traversed, the computation graph can be statically compiled based on the static shape of each bucket to obtain the compilation result corresponding to each bucket.

[0097] It should be noted that if the input of the computation graph includes at least one static input (i.e., an input with a static shape) in addition to P dynamic inputs, the computer device may also add the static shape of each static input to the static shape of each bucket, so that the static shape of each bucket includes not only a static shape for each dynamic input, but also the static shape of each static input, so as to improve the comprehensiveness of the static shape of each bucket, thereby accurately statically compiling the computation graph based on the static shape of each bucket and obtaining the compilation result corresponding to each bucket.

[0098] S205, after obtaining the input data of the calculation graph, based on the compilation results corresponding to the multiple buckets, execute the calculation graph on the input data to obtain the data calculation result of the input data.

[0099] As can be seen from the above description, each bucket has a static shape. Moreover, the difference between the static shape of any bucket and the shape of the input data lies in the different values ​​of the dynamic dimension, that is, the values ​​of the static shape of any bucket and the shape of the input data under the static dimension are the same. For example, if the dynamic dimension is the first dimension, then the static shape of any bucket can be <4x10x16xf32>, <8x10x16xf32>, etc., while the shape of the input data can be <5x10x16xf32>;.

[0100] Based on this, when executing S205, the computer device can select a target bucket from multiple buckets based on the shape of the input data, and the static shape of the target bucket is greater than or equal to the shape of the input data; wherein, the static shape of the target bucket is greater than or equal to the shape of the input data means that the value of the static shape of the target bucket in the dynamic dimension is greater than or equal to the value of the input data in the dynamic dimension. Further, when the static shape of the target bucket is equal to or greater than the shape of the input data, it indicates that the compilation result corresponding to the target bucket can recognize and process the input data, so the calculation graph can be directly executed on the input data based on the compilation result corresponding to the target bucket (that is, the compilation result corresponding to the target bucket is directly executed using the input data) to obtain the data calculation result of the input data.

[0101] When the static shape of the target bucket is larger than the shape of the input data, it indicates that the shape of the input data is not compatible with the shape of the data supported for recognition and processing by the compilation result corresponding to the target bucket. Therefore, the input data can be expanded to obtain expanded data (i.e., expanded input data) so that the shape of the expanded data is the same as the static shape in the target bucket. Based on the compilation result corresponding to the target bucket, the calculation graph is executed on the expanded data to obtain the data calculation result of the expanded data, and reference information for shape prediction is obtained. The reference information is generated based on the calculation logic of the calculation graph, so as to predict the target output shape (i.e., the shape of the data calculation result corresponding to the input data) according to the shape of the input data and the reference information. Then, based on the target output shape, the data calculation result of the expanded data is extracted to obtain the data calculation result corresponding to the input data.

[0102] Among them, the specific implementation method of the above-mentioned step of executing the calculation graph on the extended data based on the compilation result corresponding to the target bucket is: using the extended data to execute the compilation result corresponding to the target bucket. It can be seen that when the static shape of the target bucket is larger than the shape of the input data, the extended data whose shape is compatible with the static shape of the target bucket is obtained by expanding the input data, so that the compilation result corresponding to the target bucket is executed using the extended data, which can ensure the successful execution of the target compilation result. By predicting the shape of the data calculation result corresponding to the input data, data extraction of the data calculation result of the extended data based on the predicted shape can ensure the accuracy of the data calculation result corresponding to the input data obtained in the end.

[0103] It is understandable that in the above implementation, although the input data is expanded, resulting in additional redundant operations, experiments have shown that through reasonable bucketing, the execution efficiency of static bucketing compilation can still exceed the execution efficiency of dynamic shape graphs.

[0104] The embodiment of the present application can obtain a calculation graph including P dynamic inputs and a bucket mark of the calculation graph, wherein the bucket mark is used to indicate the group to which each dynamic input belongs and the bucket method of each group, and the dynamic shapes of each dynamic input in the same group have the same value range, and the value range of the dynamic shape of each dynamic input is bucketed according to the bucket method of the corresponding group; it can be seen that the embodiment of the present application sets the bucket mark in the form of grouping, so that the bucket method of the value range of the dynamic shape of each dynamic input in the same group can be recorded with one piece of information, which can avoid occupying some memory to separately record the bucket method of the value range of the dynamic shape of each dynamic input, thereby effectively saving processor memory. Furthermore, before executing the calculation graph, the value range of the dynamic shape of P dynamic inputs can be bucketed as indicated by the bucket mark to obtain multiple buckets, and the calculation graph can be statically compiled based on the value in each bucket. After obtaining the input data of the calculation graph, the calculation graph is executed on the input data based on the compilation results corresponding to the multiple buckets to obtain the data calculation results of the input data. This can avoid the situation where processor resources are consumed to compile the calculation graph every time the input data is obtained, thereby effectively saving the overhead generated by the processor and shortening the execution time of the calculation graph, thereby improving the execution efficiency of the calculation graph.

[0105] Based on the above Figure 2 The related description of the method embodiment shown, the embodiment of the present application proposes another method for processing a computational graph; in the embodiment of the present application, the computer device is still used as the execution subject for explanation.

[0106] See also Figure 4 As shown, the processing method of the computation graph may generally include S401-S408:

[0107] S401, obtaining a computation graph and obtaining a bucket mark of the computation graph.

[0108] The computation graph may include multiple nodes, each node has its own data processing logic, and the data processing logic of multiple nodes constitutes the operation logic of the computation graph. Specifically, the multiple nodes may include Q input nodes, R operator nodes, and at least one output node, where Q and R are both positive integers; the data processing logic of the input node may be the logic of the input node inputting data to the computation graph, the data processing logic of the operator node may be the data operation logic adopted by the operator node, and the data processing logic of the output node may be the logic of outputting data from the computation graph. An input node represents an input, an operator node represents an operator, and an output node represents an output; based on this, it can be seen that a computer device may include Q inputs, R operators, and at least one output. Moreover, there are P dynamic inputs among the Q inputs, each dynamic input has a dynamic shape, and each dynamic shape has a value range, P∈[1,Q].

[0109] The bucket mark of the computation graph is used to indicate: the group to which each dynamic input belongs and the bucketing method of each group; wherein the dynamic shapes of each dynamic input in the same group have the same value range, and the value range of the dynamic shape of each dynamic input is bucketed according to the bucketing method of the corresponding group. It is understandable that the generation method of the bucket mark of the computation graph can refer to the relevant description of the aforementioned method embodiment, which will not be repeated here.

[0110] Optionally, considering that the value range of the dynamic shape of each dynamic input of the calculation graph is bucketed, and the calculation graph is statically compiled based on the values ​​in each bucket, when the calculation graph is executed on the actual input data based on the compilation results corresponding to multiple buckets, there may be an operation to expand the input data, and if there may be some operation logic of operators in the calculation graph, the accuracy of the final data calculation result of the input data will be affected by the expansion operation. In order to avoid this problem, it can be considered that this type of calculation graph does not have the bucketability (that is, the calculation graph supports the characteristic of bucketing the value range of the dynamic shape of the dynamic input), thereby prohibiting the execution of bucketing processing on this type of calculation graph. Based on this, after obtaining the calculation graph, the computer device can detect whether the calculation graph has bucketability, and then perform bucketing processing when it is detected that the calculation graph has bucketability; that is, the bucketing processing in the embodiment of the present application is performed when it is detected that the calculation graph has bucketability.

[0111] The following method can be used to determine whether a computation graph is bucketable (i.e., whether the computation graph can be executed using bucketized static compilation):

[0112] Assume that the dynamic input of a computation graph is <-1x10xf32>, where the dimension where -1 is located (i.e., the first dimension) is a dynamic dimension, and the value range of the dynamic dimension is [1,100]. If in one execution, the shape of the input data of the computation graph is <23x10xf32>, and the compiled result corresponding to the bucket with a static shape of <100x10xf32> is used to execute the input data, then the input data needs to be expanded using 77 preset data elements in the first dimension, so that the shape of the expanded input data is <100x10xf32>, that is, the expanded input data has 100 data elements in the first dimension, such as Figure 5a In this case, among the 100 data elements of the expanded input data in the first dimension, only the first 23 data elements (i.e., the data elements in the solid box) are meaningful inputs, and the last 77 data elements (i.e., the data elements in the dotted box) are meaningless; then, for this expanded input data, if during the execution of the computation graph, each operator in the computation graph can meet the following conditions:

[0113] (1) Any data element in a solid-line box is only calculated with other data elements in a solid-line box, and any data element in a dotted-line box is only calculated with other data elements in a dotted-line box (i.e., the data elements in the dotted-line box (i.e., the preset data elements) do not participate in the calculation of the data elements in the solid-line box (i.e., the data elements in the input data), that is, the calculation operation on the data elements in the dotted-line box is not related to the calculation operation on the data elements in the solid-line box, and the two calculation operations are independent of each other), such as:

[0114] add input[0][1], input[0][9]. This means adding the first data element in the second dimension of the input data to the first data element in the second dimension of the input data. It can be seen that when this operator is used to calculate the expanded data, the preset data elements used for expansion in the first dimension do not participate in the calculation process of the input data, that is, the calculation operation on the preset data elements is independent of the calculation operation on the input data.

[0115] reduce input, axis=1. It means to reduce (such as accumulate) the data elements in the second dimension of the input data. It can be seen that when this operator is used to calculate the expanded data, the preset data elements used for expansion in the first dimension do not participate in the calculation process of the input data, that is, the calculation operation on the preset data elements is independent of the calculation operation on the input data.

[0116] (2) The operation process does not involve the expanded shape (that is, after the input data is expanded using the preset data elements, the operator does not involve the calculation operation on the preset data elements when calculating the expanded input data), such as:

[0117] average=(input[0][0]+input[1][1]+……+input[g-1][0]) / g. It means to perform average operation on the first g data elements in the first dimension of the input data. The value of g refers to the number of data elements in the first dimension of the input data. For example, if the shape of the input data is <23x10xf32>, then g=23. It can be seen that when this operator is used to calculate the expanded data, the operation process does not involve calculation operations on the preset data elements.

[0118] It can be seen that if the operator meets the above conditions, then for the input data with the original input shape of <23x10xf32>, whether to expand it in the first dimension does not affect its final calculation result. It is just that after the expansion, there are some redundant data in its result. After removing these redundant data, the result is consistent with the direct execution of the dynamic shape graph.

[0119] Based on this, each operator in the computation graph can be analyzed to determine whether the execution of each operator meets the above conditions in the scenario of static bucketing compilation, so that it can be automatically determined whether a computation graph can be executed by static bucketing compilation (that is, whether a computation graph is bucketable).

[0120] The following are some examples of operators:

[0121] ①Matmul (matrix multiplication): This operator has two inputs (i.e. lhs and rhs), the shapes of these two inputs are <axb> <bxc>; Where B is called contracting dim (convertible value). According to the semantics of the Matmul operator, all data elements of the dimension where the contracting dim is located will be multiplied and then accumulated. If expansion is performed in this dimension, the data elements used for expansion will participate in the calculation process of the input data, resulting in incorrect data calculation results of the input data. Therefore, for the Matmul operator, its two input contracting dims cannot be bucketed, that is, the operator does not meet the bucketing conditions. Similar operators include general matmul, batching matmul, etc.

[0122] ②Reduce (accumulation operation): This operator will perform a reduce operation on a certain dimension of the input. For example: reduce_suminput<100x10>, 1. It will perform an accumulation operation on the data elements in the second dimension of the input data. Specifically, for each data element in the first dimension, the 10 data elements corresponding to the second dimension are accumulated to obtain the data calculation result. The shape of the data calculation result is <100> , then for such a reduce operator, it can be determined that it cannot perform static bucketing on the dimension where the reduce operation is performed, and thus it is determined that the operator does not meet the bucketing conditions.

[0123] ③Gather: This operator has two main parameters, input and indices. Its logic is mainly to select appropriate data elements from input and fill them into the data calculation result through the value of the indices parameter. It can be seen that when using this operator to calculate the data after expanding the input data based on the preset data elements, it may involve calculation operations (index search) for the preset data elements. Therefore, such an operator cannot perform static bucketing, and thus it is determined that the operator does not meet the bucketing conditions.

[0124] Based on the above description, the input data includes at least one data element; the computation graph records at least one operator (that is, the operator represented by R operator nodes). The method of detecting whether the computation graph is bucketable can be summarized as including the following steps s21-s24:

[0125] s21, traverse each operator recorded in the computation graph, and take the currently traversed operator as the current operator.

[0126] s22, according to the data processing logic of the current operator, predict whether the preset data element affects the original calculation result of the input data when the current operator calculates the expanded input data after the preset data element is used to expand the input data, and the original calculation result refers to: the result obtained by the current operator calculating the input data. Specifically, according to the data processing logic of the current operator, it can be predicted whether there is a calculation operation for the preset data element when the current operator calculates the expanded input data after the preset data element is used to expand the input data; if there is no calculation operation for the preset data element, it is determined that the preset data element does not affect the original calculation result of the input data; if there is a calculation operation for the preset data element, according to the data processing logic of the current operator, the calculation association between the preset data element and the input data (that is, the association between the calculation operation for the preset data element and the calculation result for the input data) is detected; if the calculation association is detected, it is determined that the preset data element affects the original calculation result of the input data; if the calculation association is not detected, it is determined that the preset data element does not affect the original calculation result of the input data.

[0127] s23: If the preset data element affects the original calculation result of the input data, it is determined that the current operator does not meet the bucketing condition; if the preset data element does not affect the original calculation result of the input data, it is determined that the current operator meets the bucketing condition.

[0128] s24, after all operators are traversed, if all operators meet the bucketing conditions, it is determined that the computation graph is bucketable; if there is at least one operator that does not meet the bucketing conditions, it is determined that the computation graph is not bucketable.

[0129] Based on the above descriptions of s21-s24, it can be seen that the embodiment of the present application can automatically determine whether a given computational graph is bucketable to determine whether the computational graph can be executed through static bucketing compilation, which can effectively reduce manual workload and improve the adaptation efficiency of the computational graph.

[0130] S402 , bucketing the value ranges of the dynamic shapes of the P dynamic inputs according to the instructions of the bucketing marks to obtain a plurality of buckets.

[0131] S403: statically compile the computation graph based on the value in each bucket to obtain a compilation result corresponding to each bucket.

[0132] In a specific implementation, the specific implementation of steps S402-S403 can refer to the relevant description of steps S203-S204 in the aforementioned method example, which will not be repeated here. In addition, based on the relevant description of the aforementioned steps S203-S204, it can be known that each bucket has a static shape, and the static shape of any bucket can be a set composed of the static shapes of each input, that is, the static shape of any bucket includes the static shapes of Q inputs. For example, if the calculation graph includes 2 inputs, then the static shape of any bucket includes the static shape <4x1280xf32> of the first input and a static shape <8x1280xf32> of the second input.

[0133] S404, after obtaining the input data of the computation graph, based on the shape of the input data, a target bucket is selected from multiple buckets, and the static shape in the target bucket is greater than or equal to the shape of the input data.

[0134] Wherein, the computation graph may include Q inputs, one input is used to input one data into the computation graph, so the number of input data of the computation graph is Q. It is understandable that when Q is greater than 1, each of the Q input data has its own shape, and the shape of the qth input data matches the shape of the qth input; for example, the computation graph includes 2 inputs and each input is a dynamic input, and the dynamic shapes of each dynamic input are <-1x1280xf32> and <-1x1280xf32>, respectively, then the computation graph has 2 input data, the shape of the first input data can be <3x1280xf32> and the shape of the input data corresponding to the second input can be <6x1280xf32>. Wherein, any input data can include but is not limited to: image, text, voice, etc.

[0135] Since the difference between the static shape of any bucket and the shape of the input data lies in the different values ​​of the dynamic dimension, the computer device can determine the size relationship between the corresponding bucket and the input data by the size between the static shape of each bucket and the shape of the input data under the dynamic dimension, and select a target bucket from multiple buckets based on the size relationship, so that the static shape of the target bucket is greater than or equal to the shape of the input data. Specifically, the computer device can determine each bucket whose static shape is greater than or equal to the shape of the input data from multiple buckets based on the size relationship, and randomly select a bucket from each of the selected buckets as the target bucket. Alternatively, select the bucket with the smallest static shape from each of the selected buckets as the target bucket, so that the static shape of the target bucket is closest to the shape of the input data, which can reduce the subsequent expansion operation on the input data, thereby saving the processing resources of the processor and improving the operating efficiency of the processor.

[0136] It is understandable that when the calculation graph includes 1 input, the static shape of any bucket includes the static shape of 1 input and the shape of the input data is also 1; in this case, the static shape of the target bucket is greater than or equal to the shape of the input data means: the value of the static shape of the target bucket in the dynamic dimension is greater than or equal to the value of the input data in the dynamic dimension. When the calculation graph includes at least two inputs, the static shape of any bucket includes the static shapes of at least two inputs and the shape of the input data is also at least two; in this case, the static shape of the target bucket is greater than or equal to the shape of the input data means: the value of the static shape of any dynamic input contained in the target bucket in the dynamic dimension is greater than or equal to the value of the static shape of the input data corresponding to the corresponding dynamic input in the dynamic dimension.

[0137] For example, a computation graph includes two dynamic inputs, whose static shapes in bucket 1 are <6x10x80xf32> and <10x1280xf32>, and whose static shapes in bucket 2 are <10x10x80xf32> and <10x1280xf32> respectively; if the shapes of the two input data of the computation graph are <7x10x80xf32> and <8x1280xf32> respectively, then since the value of the static shape of the first dynamic input contained in bucket 2 in the dynamic dimension (the first dimension) is greater than the value of the static shape of the first input data in the dynamic dimension, and the value of the static shape of the second dynamic input contained in bucket 2 in the dynamic dimension (the first dimension) is also greater than the value of the static shape of the second input data in the dynamic dimension, bucket 2 can be used as the target bucket.

[0138] S405, when the static shape of the target bucket is larger than the shape of the input data, the input data is expanded to obtain expanded data, and the shape of the expanded data is the same as the static shape in the target bucket.

[0139] In a specific implementation, if the static shape of the target bucket is larger than the shape of the input data, the computer device may use the value of the static shape of the target bucket in the dynamic dimension as the first value, and the value of the shape of the input data in the dynamic dimension as the second value, so as to use the difference between the first value and the second value as the target number, and then expand the input data using the preset data elements of the target number in the dynamic dimension to obtain expanded data, so that the value of the shape of the expanded data in the dynamic dimension is the same as the first value. For example, if the static shape of the target bucket is <10x1280xf32>, and the shape of the input data is <8x1280xf32>, it can be determined that the first value is 10, the second value is 8, and the difference between the first value and the second value is 2, so the input data can be expanded using 2 preset data elements in the first dimension (i.e., the dynamic dimension) to obtain expanded data, and the value of the shape of the expanded data in the first dimension is 10, even if the expanded data has 10 elements in the first dimension.

[0140] It can be understood that the above is explained using the example that the static shape of the target bucket includes the static shape of 1 input and the number of input data is 1. When the static shape of the target bucket includes the static shapes of at least two inputs and the number of input data is at least two, the computer device can expand the input data corresponding to each input using the above method for each input, thereby obtaining expanded data.

[0141] S406 , based on the compilation result corresponding to the target bucket, execute the calculation graph on the extended data to obtain the data calculation result of the extended data.

[0142] S407, obtaining reference information for shape prediction, and predicting a target output shape according to the shape of the input data and the reference information, where the target output shape is the shape of a data calculation result corresponding to the input data.

[0143] In a specific implementation, since the extended data is obtained by extending the input data, it is necessary to obtain reference information for shape prediction, and predict the target output shape based on the shape of the input data and the reference information, so as to determine which data is calculated based on the input data and which data is calculated based on the preset data elements for extension in the data calculation results of the extended data based on the target output shape. For example, see Figure 5b As shown, the input data is expanded from <23xf32> to <100xf32> in the first dimension. After executing the calculation graph on the expanded data (i.e., the expanded input data) based on the compilation result corresponding to the target bucket, the shape of the data calculation result of the expanded data is <200xf32>. Therefore, it is necessary to determine which data is calculated based on the actual input data and which data is calculated based on the preset data elements in the data calculation result of the expanded data, so as to accurately extract the data calculation result of the input data.

[0144] The reference information mentioned above is generated based on the computational logic of the computational graph, and the computational logic includes the data processing logic of each node in the computational graph.

[0145] In a specific implementation, the reference information may include the computational logic of the computational graph. In this case, when the computer device predicts the target output shape based on the shape of the input data and the reference information, it may obtain a shape prediction model pre-trained based on AI technology, and then call the shape prediction model to predict the output shape based on the shape of the input data and the computational logic of the computational graph to obtain the target output shape.

[0146] In another specific implementation, the reference information may include a target shape calculation formula, in which the variables in the target shape calculation formula are numerical values ​​in the shape of the input data, and the value of the target shape calculation formula represents the value of the t-th dimension in the target output shape, where t is a positive integer and is less than or equal to the number of dimensions in the target output shape. In this case, when predicting the target output shape based on the shape of the input data and the reference information, the computer device may call the target shape calculation formula, calculate the value of the t-th dimension in the target output shape based on the numerical values ​​in the shape of the input data, obtain the target numerical value, and obtain the numerical values ​​of each dimension except the t-th dimension from the shape of the data calculation result of the extended data, thereby integrating the obtained numerical values ​​of each dimension with the target numerical value to obtain the target output shape.

[0147] In one implementation, if the computation graph includes only one operator node, the computer device may generate a target shape calculation formula directly based on the data processing logic of the operator represented by the operator node.

[0148] For example, the input to the computation graph is:

[0149] Input0: <-1xf32>

[0150] Input1: <-1x10xf32>

[0151] The output is:

[0152] Output0: <-1xf32>

[0153] Output1: <-1x30xf32>

[0154] If the data processing logic of the operator in the calculation graph includes: directly outputting the input data corresponding to the 0th input (Input0), and performing a 2-fold multiplication operation on the data elements in the 1st dimension in the input data corresponding to the 1st input (Input1); then based on the data processing logic, it can be known that the shape of the 0th output (Output0) is equal to the shape of the 0th input in each dimension, and the shape of the 1st output (Output1) is twice the shape of the 1st input in the 1st dimension.

[0155] Based on this, the generated target shape calculation formula can be as follows:

[0156] Output_0_dim_0=input_0_dim_0

[0157] Output_1_dim_0=input_1_dim_1*2

[0158] The formula means that the dynamic shape graph has two outputs, each containing a dynamic dim (i.e., the value under the dynamic dimension). The formula uses the dynamic dim in the input to calculate the dynamic dim of the output. According to the formula, the value of dim0 of Output0 (i.e., the value of the shape of Output0 in the first dimension) and the value of dim0 of input0 (i.e., the value of the shape of Input0 in the first dimension) are equal, and the value of dim0 of Output1 (i.e., the value of the shape of Output1 in the first dimension) is twice the value of dim0 of input1 (i.e., the value of the shape of Input1 in the first dimension).

[0159] In another implementation, the computation graph includes multiple nodes, and the multiple nodes may specifically include: P input nodes, R operator nodes, and output nodes, etc. The process of generating the target shape calculation formula based on the computational logic of the computation graph (i.e., the data processing logic of the multiple nodes) may include the following s31-s33:

[0160] s31, based on the topological structure of the computational graph, sorting multiple nodes in the computational graph to obtain a topological order of the multiple nodes. Specifically, the topological structure may include a connection relationship between each node. When the computer device sorts multiple nodes in the computational graph based on the topological structure, the input node may be used as a starting point (i.e., a start node) and the output node may be used as an end point (i.e., a last node), so as to obtain a topological order of the multiple nodes.

[0161] s32, according to the topological order of multiple nodes, the corresponding nodes are processed in turn according to the data processing logic of each node to obtain the shape calculation formula of each node. Among them, the variables in the shape calculation formula of any node are the values ​​in the shape of the input data. Specifically:

[0162] ① The processing of the i-th input node includes: based on the variable x i and the dimension j of the dynamic dimension in the dynamic shape of the i-th input node, generate a shape calculation formula for the i-th input node, record the generated shape calculation formula on the output shape of the i-th input node, and pass the output shape of the i-th input node to the next node of the i-th input node. The shape calculation formula of the i-th input node is used to indicate that the value of the j-th dimension in the dynamic shape of the i-th input node is the variable x i ;i∈[1,P], j is a positive integer. For example, the shape calculation formula of the i-th input node can be: f(input i , j) = x i ; i represents the number of input nodes with dynamic shapes, j represents the number of dimensions in this dynamic shape that are dynamic dimensions, and x i Represents the i-th variable (i.e. the i-th unknown quantity).

[0163] ② The processing of the rth operator node includes: according to the data processing logic of the rth operator node and the shape received by the rth operator node, deriving the output shape of the rth operator node, and passing the derived output shape to the next node of the rth operator node, r∈[1,R]. Among them, the output formula of the rth operator node (i.e., the shape calculation formula in the output shape) compared with its input formula (i.e., the received shape calculation formula) may change as follows: the output formula is equal to the input formula, or the output formula performs arithmetic operations on the basis of the input formula.

[0164] ③ The processing of the output node includes: using the shape received by the output node as the output shape of the output node, recording the shape calculation formula on the output shape of the output node, and saving the shape calculation formula recorded on the output shape (that is, saving the dynamic dim formula of the output shape).

[0165] s33, taking the shape calculation formula of the last node as the target shape calculation formula. It is understandable that, since each input node is the starting node indicated by the topological order, and the output node is the last node indicated by the topological order, the specific implementation of step s33 is: taking the shape calculation formula of the output node as the target shape calculation formula.

[0166] Based on the above description of s31-s33, Figure 5c To illustrate the generation process of the target shape calculation formula:

[0167] The design graph has two dynamic inputs: input_1 and input_2; and the dynamic shapes of input_1 and input_2 are both <-1x12xf32>. Then the shape calculation formula for input_1 can be generated as f(input1, 1) = x1, and the shape calculation formula for input_2 can be generated as f(input2, 1) = x2, so that the shape calculation formula of each dynamic input is recorded on the output shape of the corresponding dynamic input, and the output shape is passed to the next node add.

[0168] Furthermore, we can continue to deduce the corresponding output shape as <-1x12xf32> based on the data processing logic of the add operator and the received shape, and obtain the corresponding shape calculation formula as f(add, 1) = broadcast(x1, x2), and pass the derived output shape (with the shape calculation formula) to the next node reshape. It is understandable that when processing the add operator, the value of the first dimension in the output shape may be x1 or x2; assuming that the add operator here has the semantics of broadcast, then when generating the formula, the broadcast semantics also need to be added to the formula.

[0169] Furthermore, based on the data processing logic of the reshape operator (such as reshaping the data elements in the first dimension into data elements in two dimensions) and the received shape, we can deduce that the corresponding output shape is <2x-1x12xf32> and obtain the corresponding shape calculation formula f(reshape, 2) = broadcast(x1, x2) / 2, and pass the derived output shape (carrying the shape calculation formula) to the next node transpose.

[0170] Furthermore, we can continue to deduce the corresponding output shape as <-1x2x12xf32> and obtain the corresponding shape calculation formula as f(transpose, 1) = broadcast(x1, x2) / 2 based on the data processing logic of the transpose operator (such as transposing the data elements in the first dimension and the data elements in the second dimension) and the received shape, and pass the derived output shape (carrying the shape calculation formula) to the next node tile (splicing).

[0171] Furthermore, based on the data processing logic of the tile operator and the received shape, we can deduce that the corresponding output shape is <-1x2x24xf32> and obtain the corresponding shape calculation formula f(tile, 1) = broadcast(x1, x2) / 2*3, and pass the derived output shape (carrying the shape calculation formula) to the next node output.

[0172] Furthermore, the shape received by the output node can be used as the output shape of the output node, and the shape calculation formula recorded on the output shape of the output node (i.e. <-1x2x24xf32>) can be f(output, 1) = broadcast(x1, x2) / 2*3, so that the shape calculation formula can be used as the target shape calculation formula.

[0173] Based on the above, it can be seen that by continuously analyzing each node, a formula containing the dynamic dim of the input as a variable can be accurately generated. In this way, in actual execution, as long as the actual shape of the input data is obtained, the actual shape of the output (that is, the target output shape) can be directly calculated by applying the formula, which can improve the efficiency of obtaining the target output shape. It is understandable that if there are nodes that cannot be analyzed, it can be determined that the formula generation has failed; in this case, the calculation graph can be statically bucketed without compilation.

[0174] Based on the above description, it should be noted that the algorithm used in the embodiment of the present application to generate the target shape calculation formula needs to deduce the output shape based on the input shape. In addition to the above basic algorithm, some operators involved in shape calculation need to be specially processed. For example:

[0175] Shape operator, which obtains the input shape as output data.

[0176] That is to say, for the processing of common operators, the algorithm only needs to derive the formula generation according to the shape of the operator input and output. However, when encountering the shape operator above, it passes the input shape as data to the next operator, so that the shape participates in the subsequent calculation as data. At this time, if the formula is derived only according to the shape of the operator input and output, then a large number of subsequent operators will not be able to generate the shape calculation formula because the shape calculation formula does not exist in the output shape. In this scenario, the algorithm for generating the target shape calculation formula proposed in the embodiment of the present application can bind the already generated shape calculation formula in its input and its output data together when encountering special operators such as the shape operator, and pass the output data to the next node, so that the next node can derive the shape calculation formula through the output data, and the subsequent operation of the shape operator output will also perform corresponding processing on the shape calculation formula at the same time until encountering operators such as reshape.

[0177] Based on this, it can be known that the shape received by the rth operator node is represented as the target shape. When the data processing logic of the rth operator node does not include: when the calculation operation is performed on the target shape, the shape calculation formula in the target shape is recorded on the output shape of the rth operator node. When the data processing logic of the rth operator node includes: when the calculation operation is performed on the target shape, the shape calculation formula in the target shape is recorded on the output data of the rth operator node, then the processing of the rth operator node also includes: passing the output data of the rth operator node to the next node of the rth operator node, so as to realize the normal transmission of the shape calculation formula in the calculation graph, thereby ensuring that each subsequent node can generate its own shape calculation formula, and improving the success rate and accuracy of generating the final target shape calculation formula.

[0178] See for example Figure 5d As shown: Formula f() represents the shape calculation formula of the operator recorded on the output shape of the operator, f value () indicates that the shape calculation formula of the operator is recorded on the output data of the operator. Figure 5d It can be seen that:

[0179] The shape calculation formula f(add, 1) = x of the add operator is recorded in the output shape of the add operator. However, the output shape of the next operator of the add operator (i.e., the shape operator) is <2xi32> (where i32 represents a 32-bit integer). This output shape does not carry the shape calculation formula or any unknown quantity. The shape calculation formula (f value (shape, 1) = x) is recorded on the output data of the shape operator, so the output data and output shape of the shape operator need to be passed to the next node slice.

[0180] Furthermore, according to the data processing logic of the slice operator and the received shape and the output data of the shape operator, the corresponding output shape is deduced to be <1xi32> and the corresponding shape calculation formula is f value (slice, 1) = x, and pass the derived output shape and the output data recording the corresponding shape calculation formula to the next node concat (concatenation).

[0181] Furthermore, according to the data processing logic of the concat operator and the received shape and the output data of the slice operator, the corresponding output shape is deduced to be <2xi32> and the corresponding shape calculation formula is f value (concat, 1) = x, and pass the derived output shape and the output data recording the corresponding shape calculation formula to the next node reshape.

[0182] Furthermore, according to the data processing logic of the reshape operator and the received shape and the output data of the concat operator, the corresponding output shape is deduced to be <-1x6xf32> and the corresponding shape calculation formula is f(reshape, 1)=x, and the deduced output shape (recording the corresponding shape calculation formula) is passed to the next node, and so on, until the target shape calculation formula is obtained.

[0183] It can be seen that from the shape operator (inclusive) to the end of the reshape operator (excluding), the shape calculation formulas of each operator are recorded on the output data of the operator. Therefore, when transferring, the output data and output shape of each operator need to be passed to the next node together. Only in this way can the shape calculation formula of the add operator be transferred to the output shape of the reshape operator, thereby avoiding the problem that the shape calculation formulas of the reshape and subsequent operators cannot be deduced, thereby ensuring the success rate and accuracy of the derivation of the final target shape calculation formula.

[0184] S408, based on the target output shape, extract the data calculation result of the expanded data to obtain the data calculation result corresponding to the input data.

[0185] Specifically, data corresponding to the target output shape can be extracted from the data calculation results of the extended data as the data calculation results corresponding to the input data. For example, if the target output shape is <2x10xf32>, the first two data elements can be extracted from the first dimension of the data calculation results of the extended data, and the first ten data elements can be extracted from the second dimension, so that the data calculation results corresponding to the input data are formed using the data elements extracted from each dimension.

[0186] Based on the above description of steps S401-S408, it can be seen that the calculation graph processing method proposed in the embodiment of the present application may include multiple module contents. Figure 5e As shown, the contents of these multiple modules may include but are not limited to: input legality check (i.e., checking whether the input data is legal to ensure the legality of the input data), bucketability analysis (i.e., analyzing whether the calculation graph is bucketable to avoid executing optimization of inappropriate calculation graphs based on static bucketing compilation), output data space calculation (i.e., predicting the shape of the data calculation result corresponding to the input data to ensure that the correct data calculation result is obtained based on the predicted shape), custom operator processing (e.g., processing the dynamic output of the custom operator), bucketing method selection (e.g., based on the distribution of the input, selecting a more targeted bucketing method for the value range of each dynamic input to improve performance), static bucketing compilation (i.e., using the corresponding bucketing method to bucket the value range of each dynamic input, and statically compiling the calculation graph based on the value in each bucket), execution (i.e., selecting the compilation result corresponding to the appropriate bucket according to the shape of the input data and executing it), and result extraction (i.e., extracting the data calculation result of the input data from the execution result of the compilation result based on the shape calculated by the output data space).

[0187] It can be seen that the calculation graph processing method proposed in the embodiment of the present application can confirm that the input calculation graph can be statically bucketed compiled through legality checks and bucketability analysis (for example, through code analysis, it is determined that the calculations of multiple input data will not interfere with each other). Furthermore, the data that the compilation process depends on is determined by modules such as output data space calculation, bucketing strategy selection, and custom operator processing. During the compilation process, multiple static compilations are performed using the bucket values ​​of the bucketing strategy to generate multiple compilation results, so that when the calculation graph is actually executed, it can be judged according to the shape of the input data, and the appropriate compilation result can be selected to execute the corresponding input data, thereby obtaining the data calculation result of the input data.

[0188] Based on the above, it can be known that the method proposed in the embodiment of the present application can not only support the execution of dynamic shape graphs using static bucket compilation, but also provide several automatic and efficient methods to ensure the correctness of bucketing, so that it can better cope with the actual production environment, and when the dynamic shape graph needs to be executed, it can reduce the additional overhead caused by directly executing the dynamic shape graph, so that better execution efficiency can be obtained. In addition, the embodiment of the present application starts from practical applications, considers multiple scenarios such as commercial applications and testing, and adopts a simple, convenient, automated, and high-performance design, so that it has high feasibility in actual production applications.

[0189] Based on the description of the above-mentioned various method embodiments, the present application embodiment also discloses a computational graph processing device; the computational graph processing device may be a computer program (including one or more instructions) running in a computer device, and the computational graph processing device may execute each step in any of the above-mentioned method flows. Figure 6 , the processing device of the computational graph can run the following units:

[0190] An acquisition unit 601 is used to acquire a computation graph, wherein the computation graph includes P dynamic inputs, where P is a positive integer; wherein each dynamic input has a dynamic shape, and each dynamic shape has a value range;

[0191] The acquisition unit 601 is further used to acquire a bucket mark of the computation graph, wherein the bucket mark is used to indicate: the group to which each dynamic input belongs and the bucketing method of each group; wherein the dynamic shapes of each dynamic input in the same group have the same value range, and the value range of the dynamic shape of each dynamic input is bucketed according to the bucketing method of the corresponding group;

[0192] The processing unit 602 is used to perform bucket processing on the value ranges of the dynamic shapes of the P dynamic inputs according to the indication of the bucket mark to obtain multiple buckets; each bucket includes P values, and different values ​​of the P values ​​are selected from different value ranges;

[0193] The processing unit 602 is further configured to statically compile the computation graph based on the value in each bucket to obtain a compilation result corresponding to each bucket;

[0194] The processing unit 602 is further configured to, after acquiring the input data of the calculation graph, execute the calculation graph on the input data based on the compilation results corresponding to the multiple buckets to obtain a data calculation result of the input data.

[0195] In one implementation, when the processing unit 602 is used to statically compile the computation graph based on the value in each bucket to obtain the compilation result corresponding to each bucket, it can be specifically used to:

[0196] Traversing the multiple buckets, determining the kth bucket currently traversed, where k is a positive integer and is less than or equal to the total number of buckets;

[0197] Taking each value in the k-th bucket as a value of the dynamic shape of the corresponding dynamic input, respectively, to obtain the static shape of the k-th bucket; wherein, when the p-th value in the k-th bucket is selected from the value range of the dynamic shape of the p-th dynamic input, the p-th value corresponds to the p-th dynamic input, p∈[1,P];

[0198] After the multiple buckets are traversed, the computation graph is compiled based on the static shape of each bucket to obtain a compilation result corresponding to each bucket.

[0199] In another implementation, each of the buckets has a static shape; accordingly, when the processing unit 602 is used to execute the calculation graph on the input data based on the compilation results corresponding to the multiple buckets to obtain the data calculation result of the input data, it can be specifically used to:

[0200] Based on the shape of the input data, selecting a target bucket from the multiple buckets, wherein the static shape of the target bucket is greater than or equal to the shape of the input data;

[0201] When the static shape of the target bucket is larger than the shape of the input data, the input data is expanded to obtain expanded data, and the shape of the expanded data is the same as the static shape in the target bucket;

[0202] Based on the compilation result corresponding to the target bucket, executing the calculation graph on the extended data to obtain a data calculation result of the extended data;

[0203] Acquire reference information for shape prediction, the reference information being generated based on the computational logic of the computational graph; and predict a target output shape according to the shape of the input data and the reference information, the target output shape being the shape of a data computation result corresponding to the input data;

[0204] Based on the target output shape, data extraction is performed on the data calculation results of the expanded data to obtain the data calculation results corresponding to the input data.

[0205] In another embodiment, the reference information includes a target shape calculation formula, the variables in the target shape calculation formula are numerical values ​​in the shape of the input data, and the value of the target shape calculation formula represents the value of the t-th dimension in the target output shape, t is a positive integer and is less than or equal to the number of dimensions in the target output shape;

[0206] Accordingly, when the processing unit 602 is used to predict the target output shape according to the shape of the input data and the reference information, it can be specifically used to:

[0207] Calling the target shape calculation formula, based on the value in the shape of the input data, calculates the value of the tth dimension in the target output shape to obtain the target value;

[0208] From the shape of the data calculation result of the extended data, obtaining the values ​​of each dimension except the t-th dimension;

[0209] The acquired values ​​of each dimension and the target value are integrated to obtain the target output shape.

[0210] In another embodiment, the computation graph includes a plurality of nodes, each node has its own data processing logic, and the data processing logics of the plurality of nodes constitute the operation logic of the computation graph;

[0211] Accordingly, when the processing unit 602 generates the target shape calculation formula based on the operation logic of the calculation graph, it can be specifically used to:

[0212] Based on the topological structure of the computation graph, sorting multiple nodes in the computation graph to obtain a topological order of the multiple nodes;

[0213] According to the topological order of the plurality of nodes, the corresponding nodes are processed in turn according to the data processing logic of each node to obtain a shape calculation formula of each node; wherein the variables in the shape calculation formula of any node are the values ​​in the shape of the input data;

[0214] The shape calculation formula of the last node is used as the target shape calculation formula.

[0215] In another implementation, the bucketing process is performed when it is detected that the computation graph is bucketizable;

[0216] The input data includes at least one data element; the computation graph records at least one operator. Accordingly, when the processing unit 602 detects whether the computation graph has the bucketability, it can be specifically used to:

[0217] Traverse each operator recorded in the computation graph, and use the currently traversed operator as the current operator;

[0218] According to the data processing logic of the current operator, predict whether the preset data element affects the original calculation result of the input data when the current operator calculates the expanded input data after the input data is expanded using the preset data element; wherein the original calculation result refers to: the result obtained by the current operator calculating the input data;

[0219] If the preset data element affects the original calculation result of the input data, it is determined that the current operator does not meet the bucketing condition; if the preset data element does not affect the original calculation result of the input data, it is determined that the current operator meets the bucketing condition;

[0220] After all the operators are traversed, if all the operators meet the bucketing condition, it is determined that the computation graph is bucketable; if there is at least one operator that does not meet the bucketing condition, it is determined that the computation graph is not bucketable.

[0221] In another implementation, the processing unit 602 is used to predict, according to the data processing logic of the current operator, whether the preset data element affects the original calculation result of the input data when the current operator calculates the expanded input data after the input data is expanded using the preset data element, and can be specifically used to:

[0222] According to the data processing logic of the current operator, predict whether there is a calculation operation for the preset data element when the current operator calculates the expanded input data after the input data is expanded using the preset data element;

[0223] If there is no calculation operation for the preset data element, it is determined that the preset data element does not affect the original calculation result of the input data.

[0224] In another implementation, the processing unit 602 may also be used to:

[0225] If there is a calculation operation for the preset data element, detecting the calculation association between the preset data element and the input data according to the data processing logic of the current operator;

[0226] If the computational association is detected, determining that the preset data element affects the original computational result of the input data;

[0227] If the calculation association is not detected, it is determined that the preset data element does not affect the original calculation result of the input data.

[0228] According to another embodiment of the present application, Figure 6 The various units in the processing device of the calculation graph shown can be respectively or all merged into one or several other units to constitute, or some (some) units therein can also be split into multiple smaller units in function to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the functions of a unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the processing device based on the calculation graph can also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0229] According to another embodiment of the present application, a computer program (including one or more instructions) capable of executing each step involved in any of the above methods can be constructed by running the computer program (including one or more instructions) on a general computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM) and other processing elements and storage elements. Figure 6 The computer program may be recorded on a computer-readable storage medium, loaded into the computing device through the computer-readable storage medium, and run therein.

[0230] It is worth noting that in the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can contain a part of the overall module or unit of the module or unit function.

[0231] The embodiment of the present application can obtain a calculation graph including P dynamic inputs and a bucket mark of the calculation graph, wherein the bucket mark is used to indicate the group to which each dynamic input belongs and the bucket method of each group, and the dynamic shapes of each dynamic input in the same group have the same value range, and the value range of the dynamic shape of each dynamic input is bucketed according to the bucket method of the corresponding group; it can be seen that the embodiment of the present application sets the bucket mark in the form of grouping, so that the bucket method of the value range of the dynamic shape of each dynamic input in the same group can be recorded with one piece of information, which can avoid occupying some memory to separately record the bucket method of the value range of the dynamic shape of each dynamic input, thereby effectively saving processor memory. Furthermore, before executing the calculation graph, the value range of the dynamic shape of P dynamic inputs can be bucketed as indicated by the bucket mark to obtain multiple buckets, and the calculation graph can be statically compiled based on the value in each bucket. After obtaining the input data of the calculation graph, the calculation graph is executed on the input data based on the compilation results corresponding to the multiple buckets to obtain the data calculation results of the input data. This can avoid the situation where processor resources are consumed to compile the calculation graph every time the input data is obtained, thereby effectively saving the overhead generated by the processor and shortening the execution time of the calculation graph, thereby improving the execution efficiency of the calculation graph.

[0232] Based on the description of the above method embodiment and device embodiment, the present application embodiment also provides a computer device. Figure 7 , the computer device at least includes a processor 701, an input interface 702, an output interface 703 and a computer storage medium 704. Among them, the processor 701, the input interface 702, the output interface 703 and the computer storage medium 704 in the computer device can be connected via a bus or other means. The computer storage medium 704 can be stored in the memory of the computer device, the computer storage medium 704 is used to store a computer program, the computer program includes one or more instructions, and the processor 701 is used to execute one or more instructions in the computer program stored in the computer storage medium 704. The processor 701 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to realize the corresponding method flow or corresponding function.

[0233] In one embodiment, the processor 701 described in the embodiment of the present application can be used to perform a series of processing on the calculation graph, specifically including: obtaining a calculation graph, the calculation graph including P dynamic inputs, P is a positive integer; wherein each dynamic input has a dynamic shape, and each dynamic shape has a value range; obtaining a bucket mark of the calculation graph, the bucket mark is used to indicate: the group to which each dynamic input belongs and the bucket method of each group; wherein the dynamic shapes of each dynamic input in the same group have the same value range, and the value range of the dynamic shape of each dynamic input is bucketed according to the bucket method of the corresponding group; according to the indication of the bucket mark, the value range of the dynamic shape of the P dynamic inputs is bucketed to obtain multiple buckets; each bucket includes P values, and different values ​​of the P values ​​are selected from different value ranges; statically compiling the calculation graph based on the values ​​in each bucket to obtain the compilation result corresponding to each bucket; after obtaining the input data of the calculation graph, executing the calculation graph on the input data based on the compilation results corresponding to the multiple buckets to obtain the data calculation result of the input data, and so on.

[0234] The embodiment of the present application also provides a computer storage medium (Memory), which is a memory device in a computer device for storing computer programs and data. It is understandable that the computer storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer storage medium provides a storage space, which stores the operating system of the computer device. In addition, a computer program is also stored in the storage space, which includes one or more instructions suitable for being loaded and executed by the processor 701, and these instructions can be one or more program codes. It should be noted that the computer storage medium here can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage; optionally, it can also be at least one computer storage medium located away from the aforementioned processor.

[0235] In one embodiment, one or more instructions stored in a computer storage medium may be loaded and executed by a processor to implement the corresponding steps in any of the above method embodiments; in a specific implementation, one or more instructions in a computer storage medium may be loaded and executed by a processor as follows:

[0236] Obtain a computation graph, wherein the computation graph includes P dynamic inputs, where P is a positive integer; wherein each dynamic input has a dynamic shape, and each dynamic shape has a value range;

[0237] Obtaining a bucket mark of the computation graph, the bucket mark being used to indicate: the group to which each dynamic input belongs and the bucketing method of each group; wherein the dynamic shapes of each dynamic input in the same group have the same value range, and the value range of the dynamic shape of each dynamic input is bucketed according to the bucketing method of the corresponding group;

[0238] According to the indication of the bucket mark, the value ranges of the dynamic shapes of the P dynamic inputs are bucketed to obtain a plurality of buckets; each bucket includes P values, and different values ​​of the P values ​​are selected from different value ranges;

[0239] Static compile the computation graph based on the value in each bucket to obtain a compilation result corresponding to each bucket;

[0240] After the input data of the calculation graph is acquired, the calculation graph is executed on the input data based on the compilation results corresponding to the multiple buckets to obtain a data calculation result of the input data.

[0241] In one embodiment, when the computation graph is statically compiled based on the value in each bucket to obtain the compilation result corresponding to each bucket, the one or more instructions may be loaded and specifically executed by the processor:

[0242] Traversing the multiple buckets, determining the kth bucket currently traversed, where k is a positive integer and is less than or equal to the total number of buckets;

[0243] Taking each value in the k-th bucket as a value of the dynamic shape of the corresponding dynamic input, respectively, to obtain the static shape of the k-th bucket; wherein, when the p-th value in the k-th bucket is selected from the value range of the dynamic shape of the p-th dynamic input, the p-th value corresponds to the p-th dynamic input, p∈[1,P];

[0244] After the multiple buckets are traversed, the computation graph is compiled based on the static shape of each bucket to obtain a compilation result corresponding to each bucket.

[0245] In another implementation, each of the buckets has a static shape; accordingly, when the processing unit 602 is used to execute the calculation graph on the input data based on the compilation results corresponding to the multiple buckets to obtain the data calculation result of the input data, it can be specifically used to:

[0246] Based on the shape of the input data, selecting a target bucket from the multiple buckets, wherein the static shape of the target bucket is greater than or equal to the shape of the input data;

[0247] When the static shape of the target bucket is larger than the shape of the input data, the input data is expanded to obtain expanded data, and the shape of the expanded data is the same as the static shape in the target bucket;

[0248] Based on the compilation result corresponding to the target bucket, executing the calculation graph on the extended data to obtain a data calculation result of the extended data;

[0249] Acquire reference information for shape prediction, the reference information being generated based on the computational logic of the computational graph; and predict a target output shape according to the shape of the input data and the reference information, the target output shape being the shape of a data computation result corresponding to the input data;

[0250] Based on the target output shape, data extraction is performed on the data calculation results of the expanded data to obtain the data calculation results corresponding to the input data.

[0251] In another embodiment, the reference information includes a target shape calculation formula, the variables in the target shape calculation formula are numerical values ​​in the shape of the input data, and the value of the target shape calculation formula represents the value of the t-th dimension in the target output shape, t is a positive integer and is less than or equal to the number of dimensions in the target output shape;

[0252] Accordingly, when predicting the target output shape according to the shape of the input data and the reference information, the one or more instructions may be loaded and specifically executed by the processor:

[0253] Calling the target shape calculation formula, based on the value in the shape of the input data, calculates the value of the tth dimension in the target output shape to obtain the target value;

[0254] From the shape of the data calculation result of the extended data, obtaining the values ​​of each dimension except the t-th dimension;

[0255] The acquired values ​​of each dimension and the target value are integrated to obtain the target output shape.

[0256] In another embodiment, the computation graph includes a plurality of nodes, each node has its own data processing logic, and the data processing logics of the plurality of nodes constitute the operation logic of the computation graph;

[0257] Accordingly, when the target shape calculation formula is generated based on the operation logic of the calculation graph, the one or more instructions may be loaded and specifically executed by the processor:

[0258] Based on the topological structure of the computation graph, sorting multiple nodes in the computation graph to obtain a topological order of the multiple nodes;

[0259] According to the topological order of the plurality of nodes, the corresponding nodes are processed in turn according to the data processing logic of each node to obtain a shape calculation formula of each node; wherein the variables in the shape calculation formula of any node are the values ​​in the shape of the input data;

[0260] The shape calculation formula of the last node is used as the target shape calculation formula.

[0261] In another implementation, the bucketing process is performed when it is detected that the computation graph is bucketizable;

[0262] The input data includes at least one data element; the computation graph records at least one operator, and accordingly, when detecting whether the computation graph has the bucketability, the one or more instructions can be loaded and specifically executed by the processor:

[0263] Traverse each operator recorded in the computation graph, and use the currently traversed operator as the current operator;

[0264] According to the data processing logic of the current operator, predict whether the preset data element affects the original calculation result of the input data when the current operator calculates the expanded input data after the input data is expanded using the preset data element; wherein the original calculation result refers to: the result obtained by the current operator calculating the input data;

[0265] If the preset data element affects the original calculation result of the input data, it is determined that the current operator does not meet the bucketing condition; if the preset data element does not affect the original calculation result of the input data, it is determined that the current operator meets the bucketing condition;

[0266] After all the operators are traversed, if all the operators meet the bucketing condition, it is determined that the computation graph is bucketable; if there is at least one operator that does not meet the bucketing condition, it is determined that the computation graph is not bucketable.

[0267] In another embodiment, when predicting, based on the data processing logic of the current operator, whether the preset data element affects the original calculation result of the input data when the current operator calculates the expanded input data after the input data is expanded using the preset data element, the one or more instructions may be loaded and specifically executed by the processor:

[0268] According to the data processing logic of the current operator, predict whether there is a calculation operation for the preset data element when the current operator calculates the expanded input data after the input data is expanded using the preset data element;

[0269] If there is no calculation operation for the preset data element, it is determined that the preset data element does not affect the original calculation result of the input data.

[0270] In another implementation, the one or more instructions may be loaded and specifically executed by a processor:

[0271] If there is a calculation operation for the preset data element, detecting the calculation association between the preset data element and the input data according to the data processing logic of the current operator;

[0272] If the computational association is detected, determining that the preset data element affects the original computational result of the input data;

[0273] If the calculation association is not detected, it is determined that the preset data element does not affect the original calculation result of the input data.

[0274] The embodiment of the present application can obtain a calculation graph including P dynamic inputs and a bucket mark of the calculation graph, wherein the bucket mark is used to indicate the group to which each dynamic input belongs and the bucket method of each group, and the dynamic shapes of each dynamic input in the same group have the same value range, and the value range of the dynamic shape of each dynamic input is bucketed according to the bucket method of the corresponding group; it can be seen that the embodiment of the present application sets the bucket mark in the form of grouping, so that the bucket method of the value range of the dynamic shape of each dynamic input in the same group can be recorded with one piece of information, which can avoid occupying some memory to separately record the bucket method of the value range of the dynamic shape of each dynamic input, thereby effectively saving processor memory. Furthermore, before executing the calculation graph, the value range of the dynamic shape of P dynamic inputs can be bucketed as indicated by the bucket mark to obtain multiple buckets, and the calculation graph can be statically compiled based on the value in each bucket. After obtaining the input data of the calculation graph, the calculation graph is executed on the input data based on the compilation results corresponding to the multiple buckets to obtain the data calculation results of the input data. This can avoid the situation where processor resources are consumed to compile the calculation graph every time the input data is obtained, thereby effectively saving the overhead generated by the processor and shortening the execution time of the calculation graph, thereby improving the execution efficiency of the calculation graph.

[0275] It should be noted that, according to one aspect of the present application, a computer program product or computer program is also provided, and the computer program product or computer program includes one or more instructions, and the one or more instructions are stored in a computer storage medium. The processor of the computer device reads one or more instructions from the computer storage medium, and the processor executes the one or more instructions, so that the computer device executes the method provided in various optional ways in any of the above-mentioned method embodiments. It should be understood that what is disclosed above is only a preferred embodiment of the present application, and of course it cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.< / bxc> < / axb>

Claims

1. A method for processing a computational graph, characterized in that: include: Obtain a computation graph, wherein the computation graph includes P dynamic inputs, where P is a positive integer; wherein each dynamic input has a dynamic shape, and each dynamic shape has a value range; Obtaining a bucket mark of the computation graph, the bucket mark being used to indicate: the group to which each dynamic input belongs and the bucketing method of each group; wherein the dynamic shapes of each dynamic input in the same group have the same value range, and the value range of the dynamic shape of each dynamic input is bucketed according to the bucketing method of the corresponding group; According to the indication of the bucket mark, the value ranges of the dynamic shapes of the P dynamic inputs are bucketed to obtain a plurality of buckets; each bucket includes P values, and different values ​​of the P values ​​are selected from different value ranges; Static compile the computation graph based on the value in each bucket to obtain a compilation result corresponding to each bucket; After the input data of the calculation graph is acquired, the calculation graph is executed on the input data based on the compilation results corresponding to the multiple buckets to obtain a data calculation result of the input data.

2. The method according to claim 1, characterized in that The dynamic shape of the dynamic input includes values ​​of N dimensions, and there is a dynamic dimension in the N dimensions, and N is a positive integer; The dynamic dimension refers to a dimension whose value supports dynamic changes; the value range of the dynamic shape refers to: the value range of the dynamic dimension in the dynamic shape.

3. The method according to claim 2, characterized in that The bucket mark includes: a group mark of each dynamic input, and an attribute mark of at least one group; wherein: The group to which the dynamic input belongs refers to: the group to which the dynamic dimension in the dynamic shape of the dynamic input belongs; the group tag of the dynamic input is used to indicate: the dynamic dimension in the dynamic shape of the dynamic input, and the group to which the corresponding dynamic dimension belongs; A group corresponds to a value range, and the attribute tag of the group is used to indicate: the value range corresponding to the group and the bucketing method of the group.

4. The method according to any one of claims 1 to 3, characterized in that: The statically compiling the computation graph based on the value in each bucket to obtain the compilation result corresponding to each bucket includes: Traversing the multiple buckets, determining the kth bucket currently traversed, where k is a positive integer and is less than or equal to the total number of buckets; Taking each value in the k-th bucket as a value of the dynamic shape of the corresponding dynamic input, respectively, to obtain the static shape of the k-th bucket; wherein, when the p-th value in the k-th bucket is selected from the value range of the dynamic shape of the p-th dynamic input, the p-th value corresponds to the p-th dynamic input, p∈[1,P]; After the multiple buckets are traversed, the computation graph is compiled based on the static shape of each bucket to obtain a compilation result corresponding to each bucket.

5. The method according to any one of claims 1 to 3, characterized in that: Each of the buckets has a static shape; and executing the calculation graph on the input data based on the compilation results corresponding to the multiple buckets to obtain a data calculation result of the input data includes: Based on the shape of the input data, selecting a target bucket from the multiple buckets, wherein the static shape of the target bucket is greater than or equal to the shape of the input data; When the static shape of the target bucket is larger than the shape of the input data, the input data is expanded to obtain expanded data, and the shape of the expanded data is the same as the static shape in the target bucket; Based on the compilation result corresponding to the target bucket, executing the calculation graph on the extended data to obtain a data calculation result of the extended data; Acquire reference information for shape prediction, the reference information being generated based on the computational logic of the computational graph; and predict a target output shape according to the shape of the input data and the reference information, the target output shape being the shape of a data computation result corresponding to the input data; Based on the target output shape, data extraction is performed on the data calculation results of the expanded data to obtain the data calculation results corresponding to the input data.

6. The method according to claim 5, characterized in that The reference information includes a target shape calculation formula, the variables in the target shape calculation formula are numerical values ​​in the shape of the input data, and the value of the target shape calculation formula represents the value of the t-th dimension in the target output shape, where t is a positive integer and is less than or equal to the number of dimensions in the target output shape; The predicting the target output shape according to the shape of the input data and the reference information comprises: Calling the target shape calculation formula, based on the value in the shape of the input data, calculates the value of the tth dimension in the target output shape to obtain the target value; From the shape of the data calculation result of the extended data, obtaining the values ​​of each dimension except the t-th dimension; The acquired values ​​of each dimension and the target value are integrated to obtain the target output shape.

7. The method according to claim 6, characterized in that The computation graph includes a plurality of nodes, each node has its own data processing logic, and the data processing logics of the plurality of nodes constitute the operation logic of the computation graph; The process of generating the target shape calculation formula based on the operation logic of the calculation graph includes: Based on the topological structure of the computation graph, sorting multiple nodes in the computation graph to obtain a topological order of the multiple nodes; According to the topological order of the plurality of nodes, the corresponding nodes are processed in turn according to the data processing logic of each node to obtain a shape calculation formula of each node; wherein the variables in the shape calculation formula of any node are the values ​​in the shape of the input data; The shape calculation formula of the last node is used as the target shape calculation formula.

8. The method according to claim 7, characterized in that The multiple nodes include: P input nodes, R operator nodes and output nodes, where R is a positive integer; wherein an input node represents a dynamic input of the computation graph, and the dynamic shape of each input node includes a dynamic dimension; Each of the input nodes is a starting node indicated by the topological order, and the output node is the last node indicated by the topological order; wherein: The processing of the i-th input node includes: based on the variable x i and the dimension j of the dynamic dimension in the dynamic shape of the i-th input node, generate a shape calculation formula for the i-th input node, record the generated shape calculation formula on the output shape of the i-th input node, and pass the output shape of the i-th input node to the next node of the i-th input node; wherein the shape calculation formula of the i-th input node is used to indicate that the value of the j-th dimension in the dynamic shape of the i-th input node is the variable x i ; i∈[1, P], j is a positive integer; The processing of the rth operator node includes: deriving an output shape of the rth operator node according to the data processing logic of the rth operator node and the shape received by the rth operator node, and passing the derived output shape to the next node of the rth operator node, r∈[1,R]; The processing of the output node includes: using the shape received by the output node as the output shape of the output node, and recording a shape calculation formula on the output shape of the output node.

9. The method according to claim 8, characterized in that The shape received by the rth operator node is represented as the target shape; When the data processing logic of the rth operator node includes: when performing a calculation operation on the target shape, the shape calculation formula in the target shape is recorded on the output data of the rth operator node, then the processing of the rth operator node also includes: passing the output data of the rth operator node to the next node of the rth operator node.

10. The method according to claim 1, characterized in that The bucketing process is performed when it is detected that the computation graph is bucketizable; The input data includes at least one data element; the computation graph records at least one operator, and the method for detecting whether the computation graph has the bucketability includes: Traverse each operator recorded in the computation graph, and use the currently traversed operator as the current operator; According to the data processing logic of the current operator, predict whether the preset data element affects the original calculation result of the input data when the current operator calculates the expanded input data after the input data is expanded using the preset data element; wherein the original calculation result refers to: the result obtained by the current operator calculating the input data; If the preset data element affects the original calculation result of the input data, it is determined that the current operator does not meet the bucketing condition; if the preset data element does not affect the original calculation result of the input data, it is determined that the current operator meets the bucketing condition; After all the operators are traversed, if all the operators meet the bucketing condition, it is determined that the computation graph is bucketable; if there is at least one operator that does not meet the bucketing condition, it is determined that the computation graph is not bucketable.

11. The method according to claim 10, characterized in that The predicting, according to the data processing logic of the current operator, whether the preset data element affects the original calculation result of the input data when the current operator calculates the expanded input data after the input data is expanded by using the preset data element, includes: According to the data processing logic of the current operator, predict whether there is a calculation operation for the preset data element when the current operator calculates the expanded input data after the input data is expanded using the preset data element; If there is no calculation operation for the preset data element, it is determined that the preset data element does not affect the original calculation result of the input data.

12. The method according to claim 11, characterized in that The method further comprises: If there is a calculation operation for the preset data element, detecting the calculation association between the preset data element and the input data according to the data processing logic of the current operator; If the computational association is detected, determining that the preset data element affects the original computational result of the input data; If the calculation association is not detected, it is determined that the preset data element does not affect the original calculation result of the input data.

13. A computing graph processing device, characterized in that: include: An acquisition unit, used to acquire a computation graph, wherein the computation graph includes P dynamic inputs, where P is a positive integer; wherein each dynamic input has a dynamic shape, and each dynamic shape has a value range; The acquisition unit is further used to acquire a bucket mark of the computation graph, wherein the bucket mark is used to indicate: the group to which each dynamic input belongs and the bucketing method of each group; wherein the dynamic shapes of the dynamic inputs in the same group have the same value range, and the value range of the dynamic shape of each dynamic input is bucketed according to the bucketing method of the corresponding group; a processing unit, configured to perform bucket processing on the value ranges of the dynamic shapes of the P dynamic inputs according to the indication of the bucket marking, to obtain a plurality of buckets; each bucket includes P values, and different values ​​among the P values ​​are selected from different value ranges; The processing unit is further used to statically compile the calculation graph based on the value in each bucket to obtain a compilation result corresponding to each bucket; The processing unit is further configured to, after acquiring the input data of the calculation graph, execute the calculation graph on the input data based on the compilation results corresponding to the multiple buckets to obtain a data calculation result of the input data.

14. A computer device comprising an input interface and an output interface, characterized in that: Also includes: processor and computer storage media; The processor is suitable for implementing one or more instructions, the computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by the processor and executed by the method for processing a computational graph as described in any one of claims 1-12.

15. A computer storage medium, characterized in that The computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the method for processing a computational graph as described in any one of claims 1-12.

16. A computer program product, characterized in that The computer program product includes one or more instructions; when one or more instructions in the computer program are executed by a processor, the method for processing a computational graph as described in any one of claims 1 to 12 is implemented.

Images