An intermediate representation method and apparatus for compute graph compilation

Abstract

The application discloses an intermediate representation method for computing graph compilation, comprising the following steps: step 1, compiling a neural network into a computing graph for neural network computation; step 2, constructing a node for each tensor variable in the computing graph; step 3, associating the node representing the tensor variable in the computing graph to a pointer set pointing to the tensor variable; step 4, analyzing the constraint relationship between the tensor variables in the computing graph; step 5, iteratively constructing a topology graph of the intermediate representation based on the constraint relationship of the tensor variables in the computing graph; and step 6, analyzing the tensor variables with different aliases pointing to the same memory location based on the intermediate representation and allocating registers for the tensor variables. The application provides an analysis method for the tensor variables with alias relationship pointing to the same memory location in the computing graph, and the intermediate representation method for computing graph compilation optimizes the compiling efficiency of the tensor variables pointing to the same memory location in the computing graph.

Description

Technical Field

[0001] This invention relates to the field of computer systems based on specific computational models, and in particular to an intermediate representation method and apparatus for computational graph compilation. Background Technology

[0002] With the deployment of neural network models in recent years, technologies for compiling computational graphs have become increasingly important. Currently, existing graph compilation technologies for neural network computation lack the ability to analyze tensor variables within the computational graph tensor flow that point to the same memory address. Therefore, existing compilation technologies have high memory resource requirements. Summary of the Invention

[0003] The purpose of this invention is to provide an intermediate representation method and apparatus for computation graph compilation, so as to overcome the shortcomings of the prior art.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] This invention discloses an intermediate representation method for computation graph compilation, comprising the following steps:

[0006] Step 1: Compile the neural network into a computation graph for neural network computation;

[0007] Step 2: Construct nodes for each tensor variable in the computation graph;

[0008] Step 3: Associate the nodes representing tensor variables in the computation graph with a set of pointers pointing to the tensor variables;

[0009] Step 4: Analyze and calculate the constraint relationships between tensor variables in the graph, including the following sub-steps:

[0010] Step 4.1: Define the constraint representation for address assignment operations between tensor variables in the computation graph;

[0011] Step 4.2: Define the constraint representation for assignment operations between tensor variables in the computation graph;

[0012] Step 4.3: Define the constraint representation for the loading operation of the tensor variable pointer set in the computation graph;

[0013] Step 4.4: Define the constraint representation for the storage operations of the tensor variable pointer set in the computation graph;

[0014] Step 5: Iteratively construct the intermediate representation of the topological graph based on the constraint relationships of the computation graph tensor variables, including the following sub-steps:

[0015] Step 5.1: The propagation process of constructing the intermediate representation of the computation graph based on the constraint representation of assignment operations between tensor variables;

[0016] Step 5.2: The process of loading the tensor variable pointer set in the intermediate representation of the computation graph based on the constraint representation of the tensor variable pointer set loading operation;

[0017] Step 5.3: The storage procedure for constructing the tensor variable pointer set in the intermediate representation of the computation graph based on the constraint representation of the tensor variable pointer set storage operation;

[0018] Step 6: Analyze tensor variables with different aliases pointing to the same memory location based on intermediate representations, and allocate registers for them.

[0019] Preferably, step 4.1 defines the constraint representation of address assignment operations between tensor variables in the computation graph as a constraint representation method of assigning the address of tensor variable b to tensor variable a. Specifically, if there is a relationship between tensor variable b and tensor variable a of assigning the address of tensor variable b to tensor variable a, then a basic constraint relationship is defined between tensor variable b and tensor variable a: the set containing tensor variable b is contained in the set of pointers to tensor variable a.

[0020] Preferably, step 4.2 defines the constraint representation of assignment operations between tensor variables in the computation graph as a constraint representation method for assigning tensor variable b to tensor variable a. Specifically, if tensor variable b and tensor variable a have a relationship of assigning tensor variable b to tensor variable a, then a constraint relationship of assignment operation is defined between tensor variable b and tensor variable a: the set of pointers pointing to tensor variable b is included in the set of pointers pointing to tensor variable a.

[0021] Preferably, step 4.3 defines the constraint representation of the tensor variable pointer set loading operation in the computation graph as a constraint representation method for loading elements from the pointer set pointing to tensor variable b into the pointer set pointing to tensor variable a. Specifically, if tensor a and tensor b satisfy the condition that elements from the pointer set pointing to tensor variable b are assigned to tensor variable a, and the pointer set pointing to tensor variable b contains elements of tensor variable t, then a constraint relationship is defined between tensor variables a and b: the constraint relationship of the operation of loading the pointer set pointing to tensor variable b into the pointer set pointing to tensor variable a is defined as the pointer set pointing to tensor variable t being contained in the pointer set pointing to tensor variable a.

[0022] Preferably, step 4.4 defines the constraint representation of the storage operation of the tensor variable pointer set in the computation graph as a constraint representation method for storing the pointer set pointing to tensor variable b into the pointer set of elements of the pointer set pointing to tensor variable a. Specifically, if tensor variable b and tensor variable a satisfy the following: the pointer set pointing to tensor variable b is assigned to the elements of the pointer set pointing to tensor variable a, and the pointer set pointing to tensor variable a contains the element of tensor variable t, then tensor a and b are defined to have a constraint relationship: the constraint relationship of storing the pointer set pointing to tensor variable b into the pointer set of elements of the pointer set pointing to tensor variable a is defined as the pointer set pointing to tensor variable b being contained in the pointer set of element t of the pointer set pointing to tensor variable a.

[0023] Preferably, step 5.1 refers to the propagation of the set containing tensor variables along the direction of the edges of the computation graph based on the constraint representation, and the specific sub-steps are as follows:

[0024] Step 5.1.1 Constructing the intermediate representation of assignment operations in the computation graph: For the constraint representation of assignment operations in the computation graph, if there is a relationship between tensor variable b and tensor variable a that assigns tensor variable b to tensor variable a, then there is a constraint representation between tensor variable b and tensor variable a: the set of pointers to tensor variable b is contained in the set of pointers to tensor variable a.

[0025] Step 5.1.2: Construct a topology graph based on the constraint representation of the assignment operation. Specifically, for the constraint relationship that the set of pointers to tensor variable b is included in the set of pointers to tensor variable a, based on the constraint relationship, an edge is generated in the topology graph that connects the nodes of the pointer set of tensor variable b to the nodes of the pointer set of tensor variable a.

[0026] The propagation process of tensor variables in the assignment operation in the above topology graph is as follows: the execution flow of the computation graph passes through the constraint relationship edge representing the assignment operation. The constraint relationship edge propagates from the pointer set node pointing to the tensor variable b to the pointer set node pointing to the tensor variable a. The tail node pointer of the constraint relationship also points to the set containing tensor variables pointed to by the pointer of the starting node of the constraint relationship edge.

[0027] Preferably, step 5.2 refers to loading the elements from the set of pointers to tensor variable b into the set of pointers to tensor variable a, and the elements in the set of pointers to tensor variable b propagate along the direction of the constraint-based edge representation of the computation graph. The specific sub-steps are as follows:

[0028] Step 5.2.1: Construct the intermediate representation of the loading operation in the computation graph: For the constraint representation of the loading operation in the computation graph, if tensor variable b and tensor variable a exist, and the elements in the pointer set pointing to tensor variable b are assigned to tensor variable a, and the pointer set pointing to tensor variable b contains elements of tensor variable t, then the constraint representation of the operation of loading the pointer set pointing to tensor variable b into the pointer set pointing to tensor variable a is: the pointer set pointing to tensor variable t is contained in the pointer set pointing to tensor variable a;

[0029] Step 5.2.2: Construct a topology graph based on the constraint representation of the loading operation. Specifically, for the constraint relationship that the pointer set of element tensor variable t pointing to the pointer set of tensor variable b is included in the pointer set of tensor variable a, based on the constraint relationship, an edge is generated in the topology graph from the node of the pointer set of tensor variable t to the node of the pointer set of tensor variable a.

[0030] The propagation process of the tensor variable in the loading operation in the above topology graph is as follows: the execution flow of the computation graph passes through the constraint relationship edge representing the loading operation, and the constraint relationship edge propagates from the pointer set node of the corresponding element to the pointer set node pointing to the tensor variable a.

[0031] Preferably, step 5.3 refers to storing the set of pointers to tensor variable b into the set of pointers to elements of the set of pointers to tensor variable a, and the set of pointers to tensor variable b propagates along the direction of the constraint-based edge of the computation graph. The specific sub-steps are as follows:

[0032] Step 5.3.1: Construct the intermediate representation of storage operations in the computation graph: For the constraint representation of storage operations in the computation graph, if there exists a set of pointers to tensor variable b that is assigned to an element of the set of pointers to tensor variable a, and the set of pointers to tensor variable a contains an element of tensor variable t, then the constraint representation of storing the set of pointers to tensor variable b into the set of pointers to the elements of the set of pointers to tensor variable a is: represented as the set of pointers to tensor variable b being contained in the set of pointers to the element t of the set of pointers to tensor variable a.

[0033] Step 5.3.2: Construct a topology graph based on the constraint representation of the storage operation. Specifically, for the constraint relationship that the set of pointers to tensor variable b is included in the set of pointers to tensor variable t, an edge is generated in the topology graph based on the constraint relationship, connecting the nodes of the pointer set of tensor variable b to the nodes of the pointer set of tensor variable t.

[0034] The propagation process of tensor variables in the storage operation in the above topology graph is as follows: the execution flow of the computation graph passes through the constraint relationship edge representing the storage operation, and the constraint relationship edge propagates from the pointer set node pointing to the tensor variable b to the pointer set node of the corresponding element t in the pointer set pointing to the tensor variable a.

[0035] Preferably, step 6 specifically involves: if it is found that the set of pointers to a certain tensor variable in the intermediate topology graph contains tensor variables with different aliases, since they have the same memory address, these tensor variables with different aliases are regarded as the same tensor variable, and the same free register is allocated to these tensor variables with different aliases.

[0036] The present invention discloses an intermediate representation device for computation graph compilation. The device includes a memory and one or more processors. The memory stores executable code. When the one or more processors execute the executable code, they are used for the intermediate representation method for computation graph compilation described above.

[0037] The beneficial effects of this invention are as follows: This invention provides an intermediate representation method and apparatus for compiling computation graphs. It offers an analysis method for tensor variables in the computation graph that have alias relationships and point to the same memory location. After analysis, these tensor variables are stored in the same register. This intermediate representation method for compiling computation graphs optimizes the compilation efficiency of tensor variables pointing to the same memory location in the computation graph, reduces the hardware memory resource requirements during computation graph execution, and improves the execution efficiency of the computation graph. Researchers and engineers using this intermediate representation method and apparatus for compiling computation graphs can optimize models during algorithm model development, improve the compilation efficiency of computation graphs, and promote the development and application of deep neural network models. Attached Figure Description

[0038] Figure 1 This is a schematic diagram illustrating the constraint representation of the address assignment operation in the computation diagram in an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of the constraint representation of the assignment operation in an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of the constraint representation of the tensor variable pointer set loading operation in an embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram illustrating the constraint representation of the storage operation of the tensor variable pointer set in an embodiment of the present invention;

[0042] Figure 5This is a schematic diagram of the propagation process of constructing the intermediate representation of the computation graph in an embodiment of the present invention;

[0043] Figure 6 This is a schematic diagram of the loading process of the tensor variable pointer set in the intermediate representation of the computation graph in an embodiment of the present invention;

[0044] Figure 7 This is a schematic diagram of the storage process for constructing the tensor variable pointer set in the intermediate representation of the computation graph in an embodiment of the present invention;

[0045] Figure 8 This is a schematic diagram illustrating an intermediate representation of the address assignment operation in an embodiment of the present invention;

[0046] Figure 9 This is a schematic diagram illustrating the intermediate representation of the assignment operation in an embodiment of the present invention;

[0047] Figure 10 This is a schematic diagram illustrating an intermediate representation of storage operations in an embodiment of the present invention;

[0048] Figure 11 This is a schematic diagram illustrating an intermediate representation of the loading operation in an embodiment of the present invention;

[0049] Figure 12 This is a schematic diagram illustrating an intermediate representation of the loading operation in an embodiment of the present invention;

[0050] Figure 13 This is a schematic diagram of the overall architecture of the intermediate representation method in an embodiment of the present invention;

[0051] Figure 14 This is a schematic diagram of the device in an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0053] This invention provides an intermediate representation method for computation graph compilation, the architecture of which is shown in the figure below. Figure 13 As shown, it includes the following steps:

[0054] Step 1: Compile the neural network into a computation graph for neural network computation;

[0055] Step 2: Construct a node for each tensor variable v in the computation graph;

[0056] Step 3: Associate the node representing the tensor variable v in the computation graph with a set of pointers P(v) pointing to the tensor variable v;

[0057] Step 4: Analyze and calculate the constraint relationships between tensor variables in the graph, including the following sub-steps:

[0058] Step 4.1: Define the constraint representation for address assignment operations between tensor variables in the computation graph;

[0059] Step 4.2: Define the constraint representation for assignment operations between tensor variables in the computation graph;

[0060] Step 4.3: Define the constraint representation for the loading operation of the tensor variable pointer set in the computation graph;

[0061] Step 4.4: Define the constraint representation for the storage operations of the tensor variable pointer set in the computation graph;

[0062] Step 5: Iteratively construct the intermediate representation of the topological graph based on the constraint relationships of the computation graph tensor variables, including the following sub-steps:

[0063] Step 5.1: The propagation process of constructing the intermediate representation of the computation graph based on the constraint representation of assignment operations between tensor variables;

[0064] Step 5.2: The process of loading the tensor variable pointer set in the intermediate representation of the computation graph based on the constraint representation of the tensor variable pointer set loading operation;

[0065] Step 5.3: The storage procedure for constructing the tensor variable pointer set in the intermediate representation of the computation graph based on the constraint representation of the tensor variable pointer set storage operation;

[0066] Step 6: Analyze tensor variables with different aliases pointing to the same memory location based on intermediate representations, and allocate registers for them.

[0067] Step 4.1 defines the constraint representation of address assignment operations in the computation graph. Defining the constraint representation of address assignment operations between tensor variables in the computation graph refers to the method of representing the constraint of assigning the address of tensor variable b to tensor variable a. If tensor variable b and tensor variable a have a relationship of assigning the address of tensor variable b to tensor variable a, such as a=&b, then a basic constraint relationship is defined between tensor variable b and tensor variable a: the set containing tensor variable b is contained in the set of pointers to tensor variable a. The constraint representation of the address assignment operation is as follows: . Figure 1 This demonstrates the constraint representation process of address assignment operations in a computation graph.

[0068] Step 4.2 defines the constraint representation of assignment operations in the computation graph. Defining the constraint representation of assignment operations between tensor variables in the computation graph refers to the method of representing the constraint of assigning tensor variable b to tensor variable a. If tensor variable b and tensor variable a have a relationship of assigning tensor variable b to tensor variable a, such as a=b, then a constraint relationship of assignment operation is defined between tensor variable b and tensor variable a: the set of pointers to tensor variable b is contained in the set of pointers to tensor variable a. The constraint representation of the assignment operation is as follows: . Figure 2 This demonstrates the constraint representation process of assignment operations in a computation graph.

[0069] Step 4.3 defines the constraint representation of the tensor variable pointer set loading operation in the computation graph. The constraint representation of the tensor variable pointer set loading operation refers to the method of loading elements from the pointer set P(b) pointing to tensor variable b into the pointer set P(a) pointing to tensor variable a. The constraint representation of the loading operation is as follows:

[0070] If tensors a and b satisfy the following conditions: elements in the set of pointers P(b) pointing to tensor variable b are assigned to tensor variable a, and the set of pointers pointing to tensor variable b contains elements of tensor variable t: a = *b and Therefore, there is a constraint relationship between tensor variables a and b: the constraint relationship for loading the set of pointers P(b) pointing to tensor variable b into the set of pointers P(a) pointing to tensor variable a is that the set of pointers pointing to tensor variable t is contained within the set of pointers pointing to tensor variable a. . Figure 3 This demonstrates the constraint representation process of loading the tensor variable pointer set in the computation graph.

[0071] Step 4.4 defines the constraint representation of the storage operation of tensor variable pointer sets in the computation graph. The constraint representation of the storage operation of tensor variable pointer sets refers to the method of storing the set of pointers P(b) pointing to tensor variable b into the set of pointers to the elements of the set of pointers P(a) pointing to tensor variable a. The constraint relationship of the storage operation is as follows:

[0072] If tensor variables b and a satisfy the following conditions: The set of pointers P(b) pointing to tensor variable b is assigned to an element of the set of pointers pointing to tensor variable a, and the set of pointers pointing to tensor variable a contains elements of tensor variable t, such as a = *b and Therefore, tensors a and b have a constraint relationship: the constraint that stores the set of pointers P(b) pointing to tensor variable b into the set of pointers to elements of the set of pointers P(a) pointing to tensor variable a is defined as follows: the set of pointers to tensor variable b is contained in the set of pointers P(t) to elements t of the set of pointers to tensor variable a. The constraint of this storage operation is expressed as: Constraint relationships. Figure 4 This demonstrates the constraint representation process for storing tensor variable pointer sets in a computation graph.

[0073] Step 5.1 describes the propagation process for constructing the intermediate representation of the computational graph. This propagation process refers to the propagation of a set of tensor variables along the edges of the computational graph based on constraint representations. The modeling process for this intermediate representation propagation is as follows:

[0074] (i) Intermediate representation of the computation graph: For the constraint representation of assignment operations in the computation graph, if there is a relationship between tensor variable b and tensor variable a such that tensor variable b is assigned to tensor variable a, such as a=b, then tensor variable b and tensor variable a have a constraint representation: This means that the set of pointers to tensor variable b is included in the set of pointers to tensor variable a.

[0075] (ii) The process of constructing a graph for constraint representation: For a constraint relationship, the set of pointers to tensor variable b is included in the set of pointers to tensor variable a. The constraint representation graph construction process refers to the generation of an edge from node P(b) of the pointer set of tensor variable b to node P(a) of the pointer set of tensor variable a based on the constraint relationship.

[0076] (iii) Propagation of tensor variables: Since the set of pointers to tensor variable b is contained within the set of pointers to tensor variable a, all tensor elements contained in the set of pointers to tensor variable b flow to the set of pointers to tensor variable a. Therefore, as long as the execution flow of the computation graph passes through the constraint edge representing the assignment operation, the constraint edge propagates from node P(b) of the set of pointers to tensor variable b to node P(a) of the set of pointers to tensor variable a. That is, the tail node pointer of the constraint also points to the set containing tensor variables pointed to by the pointer of the starting node of the constraint edge. Figure 5 It demonstrates the propagation process of constructing the intermediate representation of the computation graph.

[0077] Step 5.2 describes the loading process for constructing the tensor variable pointer set in the intermediate representation of the computation graph. This loading process refers to loading elements from the pointer set P(b) pointing to tensor variable b into the pointer set P(a) pointing to tensor variable a, where elements in the pointer set P(b) pointing to tensor variable b propagate along the direction of the constraint-based edges of the computation graph. The modeling process for this intermediate representation loading process is as follows:

[0078] (i) Intermediate representation of the computation graph: For the constraint representation of the loading operation in the computation graph, if tensor variable b and tensor variable a exist, elements in the pointer set P(b) pointing to tensor variable b are assigned to tensor variable a, and the pointer set pointing to tensor variable b contains elements of tensor variable t: a = *b and The constraint representation of the operation of loading the set of pointers P(b) pointing to tensor variable b into the set of pointers P(a) pointing to tensor variable a is as follows: and , represents the element contained in the set of pointers to the tensor variable b. The set of pointers to tensor variable t is contained in the set of pointers to tensor variable a.

[0079] (ii) The graph construction process of constraint representation: For constraint relations that point to (elements contained in the set of pointers to tensor variable b) The set of pointers to tensor variable t is contained within the set of pointers to tensor variable a. and The constraint representation graph construction process refers to the generation of an edge from the node of the pointer set of tensor variable t to the node of the pointer set of tensor variable a based on the constraint relationship.

[0080] (iii) Propagation process of tensor variables: because it points to (elements contained in the set of pointers to tensor variable b) The set of pointers to tensor variable t is contained within the set of pointers to tensor variable a. and Therefore, the pointer sets of all tensor elements contained in the pointer set pointing to tensor variable b all flow to the pointer set pointing to tensor variable a. Thus, as long as the execution flow of the computation graph passes through the constraint edges representing the load operation, the constraint edges propagate from the pointer set node P(t) of the corresponding element to the pointer set node P(a) pointing to tensor variable a. Figure 6 The process of loading the set of tensor variable pointers in constructing the intermediate representation of the computation graph is demonstrated.

[0081] Step 5.3 involves constructing a stored procedure for the set of tensor variable pointers in the intermediate representation of the computation graph. This procedure involves storing the set of pointers P(b) pointing to tensor variable b into the set of pointers to elements of the set of pointers P(a) pointing to tensor variable a, and the set of pointers P(b) pointing to tensor variable b propagates along the direction of the edges of the constraint-based representation of the computation graph. The modeling process for this intermediate representation stored procedure is as follows:

[0082] (i) Intermediate representation of the computation graph: For the constraint representation of storage operations in the computation graph, the following conditions must be met: tensor variable b and tensor variable a exist such that the set of pointers to tensor variable b, P(b), is assigned to an element of the set of pointers to tensor variable a, and the set of pointers to tensor variable a contains elements of tensor variable t, such as *a=b. The constraint that stores the set of pointers P(b) pointing to tensor variable b into the set of pointers to elements of the set of pointers P(a) pointing to tensor variable a is as follows: This means that the set of pointers to tensor variable b is contained in the set of pointers to element t of the set of pointers to tensor variable a.

[0083] (ii) The process of constructing a graph representing constraints: For constraints where the set of pointers to tensor variable b is contained within the set of pointers to tensor variable a (containing elements of the set of pointers to tensor variable a), the following applies. In the pointer set of tensor variable t and The constraint representation graph construction process refers to the generation of an edge from the node of the pointer set of tensor variable b to the node of the pointer set of tensor variable t based on the constraint relationship.

[0084] (iii) Propagation process of tensor variables: Since the set of pointers to tensor variable b is contained in the set of pointers to (elements contained in the set of pointers to tensor variable a) In the pointer set of tensor variable t and Therefore, the pointer set node pointing to tensor variable b flows to the pointer set node pointing to tensor variable t. Thus, as long as the execution flow of the computation graph passes through the constraint edge representing the storage operation, the constraint edge propagates from the pointer set node P(b) pointing to tensor variable b to the pointer set node P(t) of the corresponding element t in the pointer set pointing to tensor variable a. Figure 7 The stored procedure for constructing the set of tensor variable pointers in the intermediate representation of the computation graph is shown.

[0085] Step 6 specifically involves: if it is found that in the topology graph of the intermediate representation, the set of pointers to a certain tensor variable contains tensor variables with different aliases, since they have the same memory address, these tensor variables with different aliases are regarded as the same tensor variable, and the same free register is allocated to these tensor variables with different aliases.

[0086] Specifically, the process for the intermediate representation method used for compiling the computation graph is as follows:

[0087] The first step is to construct a node for each tensor variable in the computation graph, representing a set of pointers to those tensor variables. The computation graph is shown below:

[0088] y=&x;

[0089] x=&z;

[0090] w=x;

[0091] *w=y;

[0092] x = *w;

[0093] The operations in the computation graph have the following meanings:

[0094] b=&a: This means that the pointer variable b points to the address of the tensor variable a, where &a means taking the memory address of the tensor variable a;

[0095] b=a: This means that pointer variable b points to tensor variable a;

[0096] *b=a: This means that the elements of pointer variable b point to tensor variable a. Here, *b represents dereferencing the pointers to tensor variable b, that is, retrieving the elements contained in the set of pointers to tensor variable b.

[0097] b = *a: This means that pointer variable b points to an element of pointer variable a.

[0098] The second step is to analyze and calculate the constraint relationships between tensor variables in the graph.

[0099] ;

[0100] The meanings of the constraint relationships between tensor variables in the computation graph are as follows:

[0101] (1) Address assignment operation If there exists a relationship between tensor variable b and tensor variable a where the address of tensor variable b is assigned to tensor variable a, such as... Therefore, a fundamental constraint exists between tensor variable b and tensor variable a: the set {b} containing tensor variable b is contained within the set of pointers to tensor variable a. ,like .

[0102] (2) Assignment operation If there's a relationship between tensor variable b and tensor variable a where tensor variable b is assigned to tensor variable a (e.g., a = b), then there's an assignment constraint between tensor variable b and tensor variable a: a set of pointers pointing to tensor variable b. Contained in a set of pointers to the tensor variable a ,like .

[0103] (3) Storage operations The set of pointers to tensor variable a contains elements of tensor variable t, and the set of pointers to tensor variable b. Assigning an element t to the set of pointers pointing to the tensor variable a, such as and Therefore, tensors a and b have a constraint relationship: define a set of pointers that will point to tensor variable b. Stored in a set of pointers to the tensor variable a The set of pointers to element t The constraint relationship in the code is a set of pointers to the tensor variable b. The set of pointers contained in the set of pointers to the tensor variable a, and the set of pointers to the element t. In, such as and .

[0104] (4) Loading operation : The set of pointers to tensor variable b contains elements of tensor variable t, such as A set of pointers to the tensor variable b The element t in the tensor is assigned to the tensor variable a, such as and Therefore, tensor variables a and b have a constraint relationship: the definition will point to the set of pointers of tensor variable b. Loaded into the set of pointers to the tensor variable a. The constraint relationship of the operation is a set of pointers to the tensor variable b. The set of pointers to the element t contained therein It is contained in a set of pointers to the tensor variable a, such as and .

[0105] The third step is to iteratively construct the intermediate representation's topology based on the constraints of the computation graph tensor variables. This iterative construction of the intermediate representation's topology based on the constraints of the computation graph tensor variables refers to iteratively constructing the intermediate representation's topology according to the constraints of the computation graph tensor variables until the structure of the intermediate representation's topology no longer changes.

[0106] The first round of iterations constructs the intermediate representation topology graph: This process includes the following steps:

[0107] (1) Constructing an intermediate representation of address assignment operations: Based on the address assignment operations y=&x and x=&z, it is analyzed that there exists a relationship between the tensor variables x, y, and z. and The constraint relationship is established. Therefore, a topology is constructed in which the set containing tensor variable x is contained within the set of pointer nodes pointing to tensor variable y, and similarly, a topology is constructed in which the set containing tensor variable z is contained within the set of pointer nodes pointing to tensor variable x. Figure 8 It demonstrates the intermediate representation process of the address assignment operation.

[0108] (2) Constructing the intermediate representation of the assignment operation: Based on the assignment operation w=x, it is analyzed that there exists a relationship between the tensor variables x and w. Constraint relationships. Therefore, the topology of the node containing the pointer set to the tensor variable x is contained within the node containing the pointer set to the tensor variable w. In other words, execution flow information flows from the node containing the pointer set to the tensor variable x to the node containing the pointer set to the tensor variable w.

[0109] (3) Propagation process of tensor variables in assignment operation: After the assignment operation steps described above, the set {z} containing tensor variable z is propagated to the pointer set node P(w) pointing to tensor variable w, therefore, there exists . Figure 9 It demonstrates the intermediate representation process of the assignment operation.

[0110] (4) Constructing the intermediate representation of the storage operation: Based on the storage operation *w=y, since the pointer set node P(w) pointing to the tensor variable w contains the element {z}, it can be deduced that the pointer set node pointing to the tensor variable y will flow to the pointer set node pointing to the tensor variable z. and .

[0111] (5) Propagation process of tensor variables in storage operations: After the storage operation steps described above, the pointer set node P(y) pointing to tensor variable y will flow to the pointer set node P(z) pointing to tensor variable z. Furthermore, due to... The set of pointers to tensor variable y contains the element tensor variable x, so the set {x} containing tensor variable x also propagates from node P(y) to node P(z), thus obtaining... . Figure 10 It demonstrates the intermediate representation process of storage operations.

[0112] (6) Constructing the intermediate representation of the loading operation: Based on the storage operation x=*w, since the pointer set node P(w) pointing to the tensor variable w contains the element {z}, it can be deduced that the pointer set node pointing to the tensor variable z will flow to the pointer set node pointing to the tensor variable x. and .

[0113] (7) Propagation process of tensor variables in the loading operation: After the loading operation steps described above, the pointer set node P(z) pointing to tensor variable z will flow to the pointer set node P(x) pointing to tensor variable x. Furthermore, due to... and The set of pointers to tensor variable z contains the element tensor variable x. Therefore, the set {x} containing tensor variable x also propagates from node P(z) to node P(x), and then continues to node P(w), thus obtaining... and . Figure 11 It illustrates the intermediate representation process of the loading operation.

[0114] The second round of iterations constructs the intermediate representation topology graph: This process includes the following steps:

[0115] Since the first iteration added element x to pointer node P(x) pointing to tensor variable x and element x to pointer node P(w) pointing to tensor variable w, the computational operations involving P(x) and P(w) concerning *x and *w need to be updated iteratively. Because the computation graph only contains operations involving *w, the operations *w=y and x = *w need to be updated.

[0116] (i) For the operation x = *w: After the first iteration, since P(w) is updated to Therefore, the primitive update for the operation involving the pointer set node P(w) pointing to the tensor variable w is: based on the storage operation x = *w, since the update of the constraint relationship of the pointer set node P(w) pointing to the tensor variable w only adds... Therefore, we can conclude that and ,because The pointer set node P(x) pointing to the tensor variable x flows to its own node, so the intermediate representation structure graph does not need to be updated.

[0117] (ii) For the operation *w=y: After the first iteration, since P(w) is updated to Therefore, the primitive update for the operation involving the pointer set node P(y) pointing to the tensor variable y is: based on the storage operation *w=y, since the update of the constraint relationship of the pointer set node P(w) pointing to the tensor variable w only adds... Therefore, we can conclude that and ,because The intermediate representation structure graph needs to be updated to include an edge that propagates from the pointer set node P(y) pointing to the pointer set node P(x) pointing to the pointer set node P(x) pointing to the pointer variable x. Figure 12 It illustrates the intermediate representation process of the loading operation.

[0118] After the second round of iterative updates to the intermediate representation topology graph, the structure of the topology graph no longer changes, thus completing the process of compiling the computation graph into an intermediate representation based on the constraint relationships of the computation graph tensor variables.

[0119] Analysis of tensor variables with different aliases pointing to the same memory location based on intermediate representation: Analysis of the topology graph of the intermediate representation reveals that the set of pointers to tensor variable x contains elements of tensor variables x and z. Therefore, tensor variables x and z have an alias relationship in the computation graph, and tensor variables x and z have the same memory address, indicating that tensor variables x and z are the same tensor variable.

[0120] Allocate registers for the tensor variables in the computation graph. Since tensor variables x and z share the same memory address, only one free register needs to be allocated for both tensor variables x and z.

[0121] See Figure 14 The present invention also provides an intermediate representation device for computation graph compilation, which further includes a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the intermediate representation method for computation graph compilation in the above embodiments.

[0122] An embodiment of the intermediate representation device for computation graph compilation of the present invention can be applied to any device with data processing capabilities, such as a computer. The device embodiment can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 14 The diagram shown is a hardware structure diagram of any device with data processing capabilities, where an intermediate representation device for computation graph compilation is located, according to the present invention. Except for... Figure 14In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of that data processing device, which will not be elaborated further. The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0123] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0124] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the intermediate representation method for computation graph compilation described in the above embodiments.

[0125] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0126] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An intermediate representation method for computation graph compilation, characterized in that: Includes the following steps: Step 1: Compile the neural network into a computation graph for neural network computation; Step 2: Construct nodes for each tensor variable in the computation graph; Step 3: Associate the nodes representing tensor variables in the computation graph with a set of pointers pointing to the tensor variables; Step 4: Analyze and calculate the constraint relationships between tensor variables in the graph, including the following sub-steps: Step 4.1: Define the constraint representation for address assignment operations between tensor variables in the computation graph; Step 4.2: Define the constraint representation for assignment operations between tensor variables in the computation graph; Step 4.3: Define the constraint representation for the loading operation of the tensor variable pointer set in the computation graph; Step 4.4: Define the constraint representation for the storage operations of the tensor variable pointer set in the computation graph; Step 5: Iteratively construct the intermediate representation of the topological graph based on the constraint relationships of the computation graph tensor variables, including the following sub-steps: Step 5.1: The propagation process of constructing the intermediate representation of the computation graph based on the constraint representation of assignment operations between tensor variables; Step 5.2: The process of loading the tensor variable pointer set in the intermediate representation of the computation graph based on the constraint representation of the tensor variable pointer set loading operation; Step 5.3: The storage procedure for constructing the tensor variable pointer set in the intermediate representation of the computation graph based on the constraint representation of the tensor variable pointer set storage operation; Step 6: Analyze tensor variables with different aliases pointing to the same memory location based on intermediate representations, and allocate registers for them.

2. The intermediate representation method for computation graph compilation as described in claim 1, characterized in that: Step 4.1 defines the constraint representation of address assignment operations between tensor variables in the computation graph as a constraint representation method for assigning the address of tensor variable b to tensor variable a. Specifically, if there is a relationship between tensor variable b and tensor variable a in which the address of tensor variable b is assigned to tensor variable a, then a basic constraint relationship is defined between tensor variable b and tensor variable a: the set containing tensor variable b is contained in the set of pointers to tensor variable a.

3. The intermediate representation method for computation graph compilation as described in claim 1, characterized in that: Step 4.2 defines the constraint representation of assignment operations between tensor variables in the computation graph as the constraint representation method of assigning tensor variable b to tensor variable a. Specifically, if there is a relationship between tensor variable b and tensor variable a that assigns tensor variable b to tensor variable a, then the constraint relationship of assignment operation between tensor variable b and tensor variable a is defined as follows: the set of pointers pointing to tensor variable b is included in the set of pointers pointing to tensor variable a.

4. The intermediate representation method for computation graph compilation as described in claim 1, characterized in that: Step 4.3 defines the constraint representation of the tensor variable pointer set loading operation in the computation graph as a constraint representation method for loading elements from the pointer set pointing to tensor variable b into the pointer set pointing to tensor variable a. Specifically, if tensor a and tensor b satisfy the condition that elements from the pointer set pointing to tensor variable b are assigned to tensor variable a, and the pointer set pointing to tensor variable b contains elements of tensor variable t, then tensor variables a and b are defined to have a constraint relationship: the constraint relationship of the operation of loading the pointer set pointing to tensor variable b into the pointer set pointing to tensor variable a is defined as the pointer set pointing to tensor variable t being contained in the pointer set pointing to tensor variable a.

5. The intermediate representation method for computation graph compilation as described in claim 1, characterized in that: Step 4.4 defines the constraint representation of the storage operation of the tensor variable pointer set in the computation graph as a constraint representation method for storing the pointer set pointing to tensor variable b into the pointer set of elements of the pointer set pointing to tensor variable a. Specifically, if tensor variable b and tensor variable a satisfy the following: the pointer set pointing to tensor variable b is assigned to the elements of the pointer set pointing to tensor variable a, and the pointer set pointing to tensor variable a contains the element of tensor variable t, then tensor a and b are defined to have a constraint relationship: the constraint relationship of storing the pointer set pointing to tensor variable b into the pointer set of elements of the pointer set pointing to tensor variable a is defined as the pointer set pointing to tensor variable b being contained in the pointer set of element t of the pointer set pointing to tensor variable a.

6. The intermediate representation method for computation graph compilation as described in claim 1, characterized in that: Step 5.1 refers to the propagation of the set containing tensor variables along the direction of the constraint-based edges of the computation graph, with the specific sub-steps as follows: Step 5.1.1 Constructing the intermediate representation of assignment operations in the computation graph: For the constraint representation of assignment operations in the computation graph, if there is a relationship between tensor variable b and tensor variable a that assigns tensor variable b to tensor variable a, then there is a constraint representation between tensor variable b and tensor variable a: the set of pointers to tensor variable b is contained in the set of pointers to tensor variable a. Step 5.1.2: Construct a topology graph based on the constraint representation of the assignment operation. Specifically, for the constraint relationship that the set of pointers pointing to tensor variable b is included in the set of pointers pointing to tensor variable a, based on the constraint relationship, an edge is generated in the topology graph from the node of the pointer set of tensor variable b to the node of the pointer set of tensor variable a. The propagation process of tensor variables in the assignment operation in the above topology graph is as follows: the execution flow of the computation graph passes through the constraint relationship edge representing the assignment operation. The constraint relationship edge propagates from the pointer set node pointing to the tensor variable b to the pointer set node pointing to the tensor variable a. The tail node pointer of the constraint relationship also points to the set containing tensor variables pointed to by the pointer of the starting node of the constraint relationship edge.

7. The intermediate representation method for computation graph compilation as described in claim 1, characterized in that: Step 5.2 refers to loading the elements from the set of pointers to tensor variable b into the set of pointers to tensor variable a, and the elements in the set of pointers to tensor variable b propagate along the direction of the constraint-based edges of the computation graph. The specific sub-steps are as follows: Step 5.2.1: Construct the intermediate representation of the loading operation in the computation graph: For the constraint representation of the loading operation in the computation graph, if tensor variable b and tensor variable a exist, and the elements in the pointer set pointing to tensor variable b are assigned to tensor variable a, and the pointer set pointing to tensor variable b contains elements of tensor variable t, then the constraint representation of the operation of loading the pointer set pointing to tensor variable b into the pointer set pointing to tensor variable a is: the pointer set pointing to tensor variable t is contained in the pointer set pointing to tensor variable a; Step 5.2.2: Construct a topology graph based on the constraint representation of the loading operation. Specifically, for the constraint relationship that the pointer set of element tensor variable t pointing to the pointer set of tensor variable b is included in the pointer set of tensor variable a, based on the constraint relationship, an edge is generated in the topology graph from the node of the pointer set of tensor variable t to the node of the pointer set of tensor variable a. The propagation process of the tensor variable in the loading operation in the above topology graph is as follows: the execution flow of the computation graph passes through the constraint relationship edge representing the loading operation, and the constraint relationship edge propagates from the pointer set node of the corresponding element to the pointer set node pointing to the tensor variable a.

8. The intermediate representation method for computation graph compilation as described in claim 1, characterized in that: Step 5.3 refers to storing the set of pointers to tensor variable b into the set of pointers to elements of the set of pointers to tensor variable a, and the set of pointers to tensor variable b propagates along the direction of the constraint-based edge of the computation graph. The specific sub-steps are as follows: Step 5.3.1: Construct the intermediate representation of storage operations in the computation graph: For the constraint representation of storage operations in the computation graph, if there exists a set of pointers to tensor variable b that is assigned to an element of the set of pointers to tensor variable a, and the set of pointers to tensor variable a contains an element of tensor variable t, then the constraint representation of storing the set of pointers to tensor variable b into the set of pointers to the elements of the set of pointers to tensor variable a is: represented as the set of pointers to tensor variable b being contained in the set of pointers to the element t of the set of pointers to tensor variable a. Step 5.3.2: Construct a topology graph based on the constraint representation of the storage operation. Specifically, for the constraint relationship that the set of pointers to tensor variable b is included in the set of pointers to tensor variable t, an edge is generated in the topology graph based on the constraint relationship, connecting the nodes of the pointer set of tensor variable b to the nodes of the pointer set of tensor variable t. The propagation process of tensor variables in the storage operation in the above topology graph is as follows: the execution flow of the computation graph passes through the constraint relationship edge representing the storage operation, and the constraint relationship edge propagates from the pointer set node pointing to the tensor variable b to the pointer set node of the corresponding element t in the pointer set pointing to the tensor variable a.

9. The intermediate representation method for computation graph compilation as described in claim 1, characterized in that: Step 6 specifically involves: if it is found that in the topology graph of the intermediate representation, the set of pointers to a certain tensor variable contains tensor variables with different aliases, since they have the same memory address, these tensor variables with different aliases are regarded as the same tensor variable, and the same free register is allocated to these tensor variables with different aliases.

10. An intermediate representation device for computation graph compilation, characterized in that: The device includes a memory and one or more processors, the memory storing executable code, which, when executed by the one or more processors, is used to implement the intermediate representation method for computation graph compilation as described in any one of claims 1-9.

Images