Operator mapping method and apparatus for a computational graph

Abstract

This application relates to an operator mapping method and apparatus for computation graphs. The method includes: based on a first footprint matrix and a second footprint matrix, repeating the following process until a stopping condition is met: calculating a first matrix based on the first footprint matrix and the second footprint matrix, adjusting the order of rows and columns in the first footprint matrix according to the first matrix, and recalculating the first matrix based on the adjusted first footprint matrix and the second footprint matrix; and obtaining the corresponding operators of multiple operators in the first computation graph in the second computation graph based on the first footprint matrix and the second footprint matrix at the time of the stopping condition. The operator mapping method and apparatus provided in this application can support operator mapping between computation graphs with similar structures and are applicable to scenarios where deep learning frameworks perform model transfer.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an operator mapping method and apparatus for computation graphs. Background Technology

[0002] Computational graphs, serving as an intermediate representation between the front-end and back-end of deep learning frameworks, offer excellent interactivity. A computational graph is a way to represent data functions using graph theory. In graph theory, nodes are connected by edges, and nodes represent entities; the edge connecting two nodes represents the relationship between the two entities. Similarly, in computational graphs, nodes represent input data or operators in a neural network, and the edge connecting two nodes represents the input-output relationship between them.

[0003] Deep learning frameworks can convert the scripts of models written according to the framework into computational graphs, and the operators within the model can be converted into corresponding nodes in the computational graph. The computational graphs generated by the same model script often differ when processed under different deep learning frameworks. When migrating models between different deep learning frameworks, such as from TensorFlow (TF) to MindSpore (MS), the structure and operators of the computational graphs for the models in the different frameworks before and after the migration may differ. Therefore, it may be necessary to compare whether the structure and operators of the computational graphs for the models before and after the migration are consistent.

[0004] Computational graphs are complex in structure and contain numerous operators. For example, the ResNet50 (Residual Network) computational graph in the TensorFlow deep learning framework has 2000 operators, while the computational graph in the MindSpore deep learning framework has 5000 operators. Therefore, it is essential to design automated computational graph comparison tools to compare two computational graphs and map operator relationships. Summary of the Invention

[0005] In view of this, an operator mapping method and apparatus for computation graphs are proposed, which can support operator mapping between computation graphs with similar structures and can be applied to scenarios where deep learning frameworks perform model transfer.

[0006] In a first aspect, embodiments of this application provide an operator mapping method for a computation graph, used to perform operator mapping on operators in a first computation graph and a second computation graph, wherein the first computation graph and the second computation graph are computation graphs of the same neural network model, and the operator mapping method includes:

[0007] The processor uses a first footprint matrix and a second footprint matrix, and repeats the following process until a stopping condition is met: Calculate a first matrix based on the first footprint matrix and the second footprint matrix, adjust the order of rows and columns in the first footprint matrix based on the first matrix, and recalculate the first matrix based on the adjusted first footprint matrix and the second footprint matrix; wherein, the first footprint matrix indicates the topological relationship between operators in the first computation graph, the second footprint matrix indicates the topological relationship between operators in the second computation graph, and the first matrix indicates the similarity of the topological relationships between the first and second computation graphs; the processor obtains the corresponding operators of each operator in the first computation graph in the second computation graph based on the first footprint matrix and the second footprint matrix when the stopping condition is met.

[0008] The operator mapping method provided in this application constructs footprint matrices for two computation graphs, where each footprint matrix represents the topological relationship between operators in the computation graphs. A first matrix can be iteratively calculated based on the footprint matrices of the two computation graphs, indicating the similarity of the topological relationships between the two graphs. After iteration stops, one or more preferred mapping relationships can be determined based on the footprint matrices. Therefore, the operator mapping method provided in this application can support operator mapping between computation graphs with similar structures and is applicable to scenarios involving model transfer within deep learning frameworks.

[0009] According to a first possible implementation of the first aspect, the processor calculates a first matrix based on a first footprint matrix and a second footprint matrix, comprising: the processor obtaining the first matrix based on a plurality of first eigenvectors and a plurality of second eigenvectors, wherein each first eigenvector corresponds to an operator in the first computation graph, and the row vector and column vector corresponding to each first eigenvector and the operator corresponding to the first eigenvector in the first footprint matrix are related; each second eigenvector corresponds to an operator in the second computation graph, and the row vector and column vector corresponding to each second eigenvector and the operator corresponding to the second eigenvector in the second footprint matrix are related; the first eigenvectors and the second eigenvectors are used to calculate the similarity of the topological relationship between the operator corresponding to the first eigenvector and the operator corresponding to the second eigenvector.

[0010] According to the first possible implementation of the first aspect, in the second possible implementation, the value of the element of the first matrix represents the similarity of the topological relationship between the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located, wherein the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located are respectively the operators in the first computation graph and the operators in the second computation graph.

[0011] The eigenvectors of operators are constructed using the row and column vectors corresponding to the footprint matrix of the operator in the computation graph, thereby extracting the topological relationship features of the operator in the computation graph. The similarity of the topological relationships of the operators is calculated based on their eigenvectors. A similarity matrix is ​​formed by comparing the similarity of the topological relationships of the operators in the first and second computation graphs. This similarity matrix is ​​used to measure the similarity of the topological relationships between the two computation graphs.

[0012] According to the first aspect, or the first or second possible implementation of the first aspect, in the third possible implementation, the operators corresponding to the same row positions of the first footprint matrix, the adjusted first footprint matrix, and the second footprint matrix are of the same type, and the tensors output by the operators are of the same dimension. That is, the operators corresponding to the same row positions of the first footprint matrix and the second footprint matrix are of the same type, and the tensors output by the operators are of the same dimension. The operators corresponding to the same row positions of the adjusted first footprint matrix and the second footprint matrix are of the same type, and the tensors output by the operators are of the same dimension.

[0013] When constructing the first footprint matrix and the second footprint matrix, and adjusting the first footprint matrix, it is ensured that the operators corresponding to the same rows and columns in the first and second footprint matrices have the same type and the same dimension of the tensor output by the operators. In other words, the operators corresponding to the same rows and columns in the first and second footprint matrices are functionally identical. A similarity matrix is ​​used to measure the similarity of the topological relationships between the operators. According to the operator mapping method provided in this application, the mapping relationship between two operators can be reflected in terms of both function and topological relationship, thereby ensuring the correctness of the mapping result.

[0014] According to the first aspect, or the first or second possible implementation of the first aspect, in the fourth possible implementation, the operators in the first computation graph and the second computation graph belong to multiple first partitions, the operators in the same first partition have the same type and the tensors output by the operators have the same dimension, and each first partition may include at least some operators of the first computation graph and / or the second computation graph; some operators in the first computation graph and the second computation graph belong to second partitions, the operators in the second partitions have different types and / or the tensors output by the operators have different dimensions; the operators corresponding to the same row in the first footprint matrix and the second footprint matrix are in the same first partition, and the operators corresponding to the same column in the first footprint matrix are in the same first partition; and the operators of the first computation graph belonging to the same first partition are arranged continuously in the corresponding rows or columns in the first footprint matrix, and the operators of the second computation graph belonging to the same first partition are arranged continuously in the corresponding rows or columns in the second footprint matrix.

[0015] Because there is a higher probability of mapping relationships between homogeneous operators, and in the embodiments of this application, there is a mapping relationship between operators corresponding to the same row or column in the first footprint matrix and the second footprint matrix. Therefore, according to the method of the above embodiments provided in this application, by homogeneously partitioning the operators in the first and second computation graphs, and adjusting the order of rows and columns in the first footprint matrix by adjusting the order of rows and columns corresponding to operators located in the same homogeneous partition, the iteration process can converge more quickly, thus accelerating the mapping process.

[0016] For operators without homogeneous operators, the corresponding operator similarity is obtained through classification by setting mixed partitions and participating in iterative calculations. When determining the mapping operator, mapping suggestions can be output based on the calculated similarity. This supports operator mapping between computation graphs with similar structures, and also supports one-to-many or many-to-many mappings, making it suitable for scenarios involving model transfer in deep learning frameworks.

[0017] According to the third possible implementation of the first aspect, in the fifth possible implementation, the homogeneity depth of the operators corresponding to the same position in the first footprint matrix and the second footprint matrix is ​​the same, the homogeneity depth of the operators corresponding to the same position in the columns is the same, and the rows or columns corresponding to the operators of the first computation graph belonging to the same first partition in the first footprint matrix are sorted according to the homogeneity depth of the operators, and the rows or columns corresponding to the operators of the second computation graph belonging to the same first partition in the second footprint matrix are sorted according to the homogeneity depth of the operators; wherein, the homogeneity depth is: the maximum number of operators with the same type and the same output tensor dimension as the operator before the operator in all branches to which the operator belongs in the computation graph.

[0018] Operators within homogeneous partitions are sorted according to their homogeneity depth. Based on the determined homogeneous partitions, the first footprint matrix of the first computational graph and the second footprint matrix of the second computational graph are determined respectively. Since operators with the same homogeneity depth are more likely to have mapping relationships, the process of calculating the similarity matrix according to the above embodiment of this application can converge quickly, determine the mapping relationships between operators more rapidly, and accelerate the mapping process. Furthermore, distinguishing operators with the same attributes by homogeneity depth avoids the unreliability of distinguishing operators with the same attributes by their names.

[0019] According to the third possible implementation of the first aspect, in the sixth possible implementation, if the number of operators in the first computation graph in the first partition is different from the number of operators in the second computation graph, the first partition also includes pseudo-operators; wherein, pseudo-operators are operators without operator type and operands; if the number of operators in the first computation graph in the first partition is less than the number of operators in the second computation graph in the first partition, the row or column corresponding to the pseudo-operator in the first footprint matrix is ​​located after the row or column corresponding to the operator belonging to the first partition in the first computation graph; if the number of operators in the second computation graph in the first partition is less than the number of operators in the first computation graph in the first partition, the row or column corresponding to the pseudo-operator in the second footprint matrix is ​​located after the row or column corresponding to the operator belonging to the first partition in the second computation graph.

[0020] According to the third or fifth possible implementation of the first aspect, in the seventh possible implementation, if the number of operators in the first computation graph in the second partition is different from the number of operators in the second computation graph, the second partition also includes the pseudo-operator; if the number of operators in the first computation graph in the second partition is less than the number of operators in the second computation graph in the second partition, the rows and columns corresponding to the pseudo-operator in the first footprint matrix are randomly sorted with the rows and columns corresponding to the operators in the first computation graph belonging to the second partition, and the random sorting of the rows and columns is the same; if the number of operators in the second computation graph in the second partition is less than the number of operators in the second computation graph in the second partition, the rows and columns corresponding to the pseudo-operator in the second footprint matrix are randomly sorted with the rows and columns corresponding to the operators in the second computation graph belonging to the second partition, and the random sorting of the rows and columns is the same.

[0021] When the number of operators in the first and second computational graphs differs, pseudo-operators are added to make the total number of operators in the first and second computational graphs equal, thereby constructing footprint matrices of the same size so that subsequent matrix operations can proceed normally. Thus, the operator mapping method provided in this application can be applied to scenarios involving the transfer of models between frameworks or systems employing different computational graph representations.

[0022] According to any one of the third to sixth possible implementations of the first aspect, in the eighth possible implementation, adjusting the order of rows and columns in the first footprint matrix according to the first matrix includes: adjusting the order of rows and columns corresponding to operators located in the same first partition in the first footprint matrix according to the first matrix.

[0023] According to any one of the third to sixth possible implementations of the first aspect, in the ninth possible implementation, there is a mapping relationship between the operators of the first computation graph and the operators of the second computation graph that belong to the same first partition and have the same row or column position in the first footprint matrix and the second footprint matrix when the stopping condition is reached; there is a mapping relationship between the operators of the first computation graph belonging to the second partition and multiple operators of the second computation graph belonging to the second partition, wherein the multiple operators of the second computation graph belonging to the second partition are: the operators of the second computation graph corresponding to the largest multiple values ​​among the values ​​of the elements corresponding to the operators of the first computation graph belonging to the second partition in the first matrix when the stopping condition is reached.

[0024] According to the mapping method of the above embodiments of this application, operator mapping between computation graphs with similar structures can be supported, supporting one-to-one, one-to-many and many-to-many mapping, which can be applied to scenarios where deep learning frameworks perform model transfer.

[0025] According to the first aspect or any one of the first to eighth possible implementations of the first aspect, in the tenth possible implementation, the stopping condition is that the number of times the first matrix is ​​repeatedly calculated is greater than a first threshold.

[0026] By homogeneously partitioning the operators in the first and second computational graphs, and adjusting the order of rows and columns in the first footprint matrix by adjusting the order of rows and columns corresponding to operators located in the same homogeneous partition, the iterative process can converge more quickly, accelerating the mapping process. This is because homogeneous operators have higher similarity, and there is a greater possibility of mapping relationships between them. Therefore, the process of homogeneous partitioning can accelerate the speed of iterative convergence and the mapping process.

[0027] Secondly, embodiments of this application provide an operator mapping apparatus for a computation graph, used to perform operator mapping on operators in a first computation graph and a second computation graph, wherein the first computation graph and the second computation graph are computation graphs of the same neural network model. The operator mapping apparatus includes: an iteration module, used to repeat the following process based on a first footprint matrix and a second footprint matrix until a stopping condition is reached: calculating a first matrix based on the first footprint matrix and the second footprint matrix, adjusting the order of rows and columns in the first footprint matrix based on the first matrix, and recalculating the first matrix based on the adjusted first footprint matrix and the second footprint matrix; wherein the first footprint matrix indicates the topological relationship between operators in the first computation graph, the second footprint matrix indicates the topological relationship between operators in the second computation graph, and the first matrix indicates the similarity of the topological relationship between the first computation graph and the second computation graph; and a correspondence module, used to obtain the corresponding operators of multiple operators in the first computation graph in the second computation graph based on the first footprint matrix and the second footprint matrix when the stopping condition is reached.

[0028] The operator mapping apparatus provided in this application constructs footprint matrices for two computation graphs, where each footprint matrix represents the topological relationship between operators in the computation graphs. A first matrix, indicating the similarity of the topological relationship between the two computation graphs, can be iteratively calculated based on the footprint matrices of the two computation graphs. After iteration stops, one or more preferred mapping relationships can be determined based on the footprint matrices. Therefore, the operator mapping method provided in this application can support operator mapping between computation graphs with similar structures and is applicable to scenarios involving model transfer within deep learning frameworks.

[0029] According to the first possible implementation of the second aspect, the iterative module includes:

[0030] The calculation unit is used to obtain the first matrix based on multiple first feature vectors and multiple second feature vectors, wherein each first feature vector corresponds to an operator in the first calculation graph, and the row vector and column vector corresponding to each first feature vector and the operator corresponding to the first feature vector in the first footprint matrix are related; each second feature vector corresponds to an operator in the second calculation graph, and the row vector and column vector corresponding to each second feature vector and the operator corresponding to the second feature vector in the second footprint matrix are related; the first feature vectors and the second feature vectors are used to calculate the similarity of the topological relationship between the operator corresponding to the first feature vector and the operator corresponding to the second feature vector.

[0031] According to the first possible implementation of the second aspect, in the second possible implementation, the value of the element of the first matrix represents the similarity of the topological relationship between the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located, wherein the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located are respectively the operators in the first computation graph and the operators in the second computation graph.

[0032] The eigenvectors of operators are constructed using the row and column vectors corresponding to the footprint matrix of the operator in the computation graph, thereby extracting the topological relationship features of the operator in the computation graph. The similarity of the topological relationships of the operators is calculated based on their eigenvectors. A similarity matrix is ​​formed by comparing the similarity of the topological relationships of the operators in the first and second computation graphs. This similarity matrix is ​​used to measure the similarity of the topological relationships between the two computation graphs.

[0033] According to the second aspect, or the first or second possible implementation of the second aspect, in the third possible implementation, the operators corresponding to the same row positions of the first footprint matrix, the adjusted first footprint matrix, and the second footprint matrix are of the same type, and the tensors output by the operators are of the same dimension. That is, the operators corresponding to the same row positions of the first footprint matrix and the second footprint matrix are of the same type, and the tensors output by the operators are of the same dimension. The operators corresponding to the same row positions of the adjusted first footprint matrix and the second footprint matrix are of the same type, and the tensors output by the operators are of the same dimension.

[0034] When constructing the first footprint matrix and the second footprint matrix, and adjusting the first footprint matrix, it is ensured that the operators corresponding to the same rows and columns in the first and second footprint matrices are of the same type, and the tensors output by the operators have the same dimension. In other words, the operators corresponding to the same rows and columns in the first and second footprint matrices are functionally identical. A similarity matrix is ​​used to measure the similarity of the topological relationships between the operators. According to the operator mapping device provided in this application, the mapping relationship between two operators can be reflected in terms of both function and topological relationship, thereby ensuring the correctness of the mapping result.

[0035] According to the second aspect, or the first or second possible implementation of the second aspect, in the fourth possible implementation, the operators in the first computation graph and the second computation graph belong to multiple first partitions, the operators in the same first partition have the same type and the tensors output by the operators have the same dimension, each first partition may include at least some operators of the first computation graph and / or the second computation graph, some operators in the first computation graph and the second computation graph belong to the second partition, the operators in the second partition have different types and / or the tensors output by the operators have different dimensions; the operators corresponding to the same row in the first footprint matrix and the second footprint matrix are in the same first partition, and the operators corresponding to the same column in the first footprint matrix are in the same first partition; and the operators of the first computation graph belonging to the same first partition are arranged continuously in the corresponding rows or columns in the first footprint matrix, and the operators of the second computation graph belonging to the same first partition are arranged continuously in the corresponding rows or columns in the second footprint matrix.

[0036] Because there is a higher probability of mapping relationships between homogeneous operators, and in the embodiments of this application, there is a mapping relationship between operators corresponding to the same row or column in the first footprint matrix and the second footprint matrix. Therefore, according to the above embodiments provided by this application, by homogeneously partitioning the operators in the first and second computation graphs, and adjusting the order of rows and columns in the first footprint matrix by adjusting the order of rows and columns corresponding to operators located in the same homogeneous partition, the iteration process can converge more quickly, thus accelerating the mapping process.

[0037] For operators without homogeneous operators, the corresponding operator similarity is obtained through classification by setting mixed partitions and participating in iterative calculations. When determining the mapping operator, mapping suggestions can be output based on the calculated similarity. This supports operator mapping between computation graphs with similar structures, and also supports one-to-many or many-to-many mappings, making it suitable for scenarios involving model transfer in deep learning frameworks.

[0038] According to the third possible implementation of the second aspect, in the fifth possible implementation, the homogeneity depth of the operators corresponding to the same position in the first footprint matrix and the second footprint matrix is ​​the same, the homogeneity depth of the operators corresponding to the same position in the columns is the same, and the rows or columns corresponding to the operators of the first computation graph belonging to the same first partition in the first footprint matrix are sorted according to the homogeneity depth of the operators, and the rows or columns corresponding to the operators of the second computation graph belonging to the same first partition in the second footprint matrix are sorted according to the homogeneity depth of the operators; wherein, the homogeneity depth is: the maximum number of operators with the same type and the same output tensor dimension as the operator before the operator in all branches to which the operator belongs in the computation graph.

[0039] Operators within homogeneous partitions are sorted according to their homogeneity depth. Based on the determined homogeneous partitions, the first footprint matrix of the first computational graph and the second footprint matrix of the second computational graph are determined respectively. Since operators with the same homogeneity depth are more likely to have mapping relationships, the process of calculating the similarity matrix according to the above embodiment of this application can converge quickly, determine the mapping relationships between operators more rapidly, and accelerate the mapping process. Furthermore, distinguishing operators with the same attributes by homogeneity depth avoids the unreliability of distinguishing operators with the same attributes by their names.

[0040] According to the third possible implementation of the second aspect, in the sixth possible implementation, if the number of operators in the first computation graph in the first partition is different from the number of operators in the second computation graph, the first partition also includes pseudo-operators; wherein, pseudo-operators are operators without operator type and operands; if the number of operators in the first computation graph in the first partition is less than the number of operators in the second computation graph in the first partition, the row or column corresponding to the pseudo-operator in the first footprint matrix is ​​located after the row or column corresponding to the operator belonging to the first partition in the first computation graph; if the number of operators in the second computation graph in the first partition is less than the number of operators in the first computation graph in the first partition, the row or column corresponding to the pseudo-operator in the second footprint matrix is ​​located after the row or column corresponding to the operator belonging to the first partition in the second computation graph.

[0041] According to the third or fifth possible implementation of the second aspect, in the seventh possible implementation, if the number of operators in the first computation graph in the second partition is different from the number of operators in the second computation graph, the second partition also includes the pseudo-operator; if the number of operators in the first computation graph in the second partition is less than the number of operators in the second computation graph in the second partition, the rows and columns corresponding to the pseudo-operator in the first footprint matrix are randomly sorted with the rows and columns corresponding to the operators in the first computation graph belonging to the second partition, and the random sorting of rows and columns is the same; if the number of operators in the second computation graph in the second partition is less than the number of operators in the second computation graph in the second partition, the rows and columns corresponding to the pseudo-operator in the second footprint matrix are randomly sorted with the rows and columns corresponding to the operators in the second computation graph belonging to the second partition, and the random sorting of rows and columns is the same.

[0042] When the number of operators in the first and second computational graphs differs, pseudo-operators are added to make the total number of operators in the first and second computational graphs equal, thereby constructing footprint matrices of the same size so that subsequent matrix operations can proceed normally. Thus, the operator mapping method provided in this application can be applied to scenarios involving the transfer of models between frameworks or systems employing different computational graph representations.

[0043] According to any one of the third to sixth possible implementations of the second aspect, in the eighth possible implementation, the iteration module further includes: an adjustment unit, used to adjust the order of rows and columns corresponding to operators located in the same first partition in the first footprint matrix according to the first matrix.

[0044] According to any one of the third to sixth possible implementations of the second aspect, in the ninth possible implementation, there is a mapping relationship between the operators of the first computation graph and the operators of the second computation graph that belong to the same first partition and have the same row or column position in the first footprint matrix and the second footprint matrix when the stopping condition is reached; there is a mapping relationship between the operators of the first computation graph belonging to the second partition and multiple operators of the second computation graph belonging to the second partition, wherein the multiple operators of the second computation graph belonging to the second partition are: the operators of the second computation graph corresponding to the largest multiple values ​​among the values ​​of the elements corresponding to the operators of the first computation graph belonging to the second partition in the first matrix when the stopping condition is reached.

[0045] According to the mapping method of the above embodiments of this application, operator mapping between computation graphs with similar structures can be supported, supporting one-to-one, one-to-many and many-to-many mapping, which can be applied to scenarios where deep learning frameworks perform model transfer.

[0046] According to the second aspect or any one of the first to eighth possible implementations of the second aspect, in the tenth possible implementation, the stopping condition is that the number of times the first matrix is ​​repeatedly calculated is greater than a first threshold.

[0047] By homogeneously partitioning the operators in the first and second computational graphs, and adjusting the order of rows and columns in the first footprint matrix by adjusting the order of rows and columns corresponding to operators located in the same homogeneous partition, the iterative process can converge more quickly, accelerating the mapping process. This is because homogeneous operators have higher similarity, and there is a greater possibility of mapping relationships between them. Therefore, the process of homogeneous partitioning can accelerate the speed of iterative convergence and the mapping process.

[0048] Thirdly, embodiments of this application provide an operator mapping apparatus for a computation graph, comprising:

[0049] A processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement one or more of the operator mapping methods of the first aspect or various possible implementations of the first aspect when executing the instructions.

[0050] Fourthly, embodiments of this application provide a non-volatile computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement one or more of the operator mapping methods described in the first aspect or various possible implementations of the first aspect.

[0051] Fifthly, embodiments of this application provide a terminal device that can execute one or more of the operator mapping methods described in the first aspect or various possible implementations of the first aspect.

[0052] In a sixth aspect, embodiments of this application provide a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in an electronic device, the processor in the electronic device executes one or more of the operator mapping methods described in the first aspect or various possible implementations of the first aspect.

[0053] These and other aspects of this application will become more apparent in the description of the following embodiments(s). Attached Figure Description

[0054] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this application together with the specification and serve to explain the principles of this application.

[0055] Figure 1a and Figure 1b Schematic diagrams of application scenarios according to an embodiment of this application are shown respectively.

[0056] Figure 2 A block diagram of an apparatus for performing an operator mapping method of a computation graph according to an embodiment of this application is shown.

[0057] Figure 3 A flowchart illustrating an operator mapping method for a computation graph according to an embodiment of this application is shown.

[0058] Figure 4 A schematic diagram of a calculation graph according to an example of this application is shown.

[0059] Figure 5a A flowchart illustrating the process of calculating similarity according to an embodiment of this application is shown.

[0060] Figure 5bA flowchart illustrating a method for adjusting the order of rows and columns of a footprint matrix according to an embodiment of this application is shown.

[0061] Figure 6 A flowchart illustrating step S300 of a method according to an embodiment of this application is shown.

[0062] Figure 7 A schematic diagram of a homogeneous partition according to an embodiment of this application is shown.

[0063] Figure 8 and Figure 9 Schematic diagrams are shown for determining the homogeneity depth of a homogeneous operator according to an embodiment of this application.

[0064] Figure 10 An example of an insertion pseudo-operator according to an embodiment of this application is shown.

[0065] Figure 11 A flowchart illustrating an operator mapping method according to an embodiment of this application is shown.

[0066] Figure 12 A schematic diagram illustrating the determination of the shortest distance according to an embodiment of this application is shown.

[0067] Figure 13 A schematic diagram of adjusting the footprint matrix according to an embodiment of this application is shown.

[0068] Figure 14 A schematic diagram illustrating the effect of iterative calculation according to an embodiment of this application is shown.

[0069] Figure 15 A schematic diagram illustrating the effect of iterative calculation according to an embodiment of this application is shown.

[0070] Figure 16 A block diagram of an operator mapping apparatus for a computation graph according to an embodiment of this application is shown.

[0071] Figure 17 A block diagram of an operator mapping apparatus according to an embodiment of this application is shown. Detailed Implementation

[0072] Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0073] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0074] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented without certain specific details. In some instances, methods, means, components, and circuits well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0075] Definitions:

[0076] Operator: A computational unit in a computation graph. An operator performs a type of computational logic. In this application, an operator can be understood as a mapping from one function space to another: y = f(x), which is a generalized view of an operator. A computation graph can include various types of operators. The type of operator refers to the type of computational logic that the operator can perform. In this application, an operator can indicate a basic type of computational logic that cannot be further divided, such as Add, Multiply, etc. An operator can also indicate a complex type of computational logic, which can be expressed as a combination of a set of basic computational logics. Thus, complex computational logic can be indicated by multiple operators, each corresponding to a different combination of basic computational logics. These multiple operators have the same function, and a group of operators can be regarded as an operator that performs the aforementioned complex computational logic. This group of operators may be homogeneous operators with the same function as another group of operators. For example, the Conv (convolution) operator, the sigmoid operator, etc., where the conv operator can be composed of addition and multiplication operators.

[0077] Tensor: In this application, a tensor is a description of the features of a stored block of data. A tensor records information such as the shape and type of the data. In the embodiments of this application, a tensor can be understood as tensor data or operands of operators, which may include input operands and output operands of operators in a neural network, or feature tensor data, etc. Taking the artificial intelligence deep learning framework TensorFlow as an example, the rank, shape, and dimension number are generally used to describe the dimension of a tensor.

[0078] Numerical archives: Archives that record the output tensor data (operands) of deep learning model operators during the computation process.

[0079] Operator mapping: In two computation graphs corresponding to the same model, an operator in one computation graph and an operator in the other computation graph have an equivalent correspondence in terms of function and topology. That is to say, there is an equivalent correspondence in terms of function and topology between two operators in two computation graphs.

[0080] Homogeneous operators: Two operators with completely identical properties are called homogeneous operators. In deep learning models, the properties of an operator can refer to its type and the dimension of its output tensor. If two operators have the same type and the same dimension of their output tensors, then these two operators are homogeneous operators. That is, two homogeneous operators are functionally identical. In this application, the operators referred to as homogeneous operators can be operators that perform basic, indivisible computational logic, or they can be a group of operators that perform a complex function that can be decomposed into basic computational logic. If this group of operators performs the same function as another group, and the dimensions of the tensors output by the two groups of operators are also the same, then these two groups of operators are also homogeneous operators.

[0081] Cosine similarity: The cosine of the angle between two non-zero vectors. The larger the cosine value, the closer the two vectors are to each other and the more similar they are.

[0082] Similarity Matrix: The values ​​in the similarity matrix represent the similarity between the operator corresponding to the row containing that value and the operator corresponding to the column containing that value. These two operators can be operators in different computation graphs. For example, the operator corresponding to the row containing the value might be an operator in one computation graph, while the operator corresponding to the column containing the value might be an operator in another. The similarity between two operators can be measured using the cosine similarity method described above. For instance, the values ​​in the matrix can represent the cosine similarity between the operator corresponding to the row containing the value and the operator corresponding to the column containing the value.

[0083] Cosine similarity of operators: the cosine of the angle between the eigenvectors corresponding to two operators.

[0084] Eigenvector of an operator: A vector that indicates the topological relationship between the operator and other operators in the computation graph.

[0085] Element: Each number in a matrix is ​​called an element.

[0086] The distance from one operator to another in a computational graph is the length of the path from the node corresponding to one operator to the node corresponding to another operator. The edges connecting two nodes in a computational graph are considered reachable paths. Since the computational graph is directed, between two directly connected nodes: the path (edge) from the parent node to the child node is reachable with a length of 1; the path from the child node to the parent node is unreachable with a length of 0. The length of the path from one node to another in a computational graph is the total length of any reachable path between those two nodes.

[0087] Calculate the shortest distance from one operator to another in a graph: calculate the shortest path length from the node corresponding to one operator to the node corresponding to another operator.

[0088] In related technologies, isomorphic graph algorithms can be used to determine the mapping relationship between nodes of two graphs. However, isomorphic graph algorithms are only suitable for mapping nodes between completely isomorphic graphs. For graphs with similar structures, node mapping cannot be performed, and isomorphic graph algorithms do not support many-to-many node mappings. Furthermore, different deep learning frameworks represent computational graphs differently. Even the same neural network model may correspond to different computational graph structures after transformation in different deep learning frameworks. Therefore, isomorphic graph algorithms are not suitable for scenarios involving model transfer between different deep learning frameworks.

[0089] To address the aforementioned technical problems, this application provides an operator mapping method for computational graphs. This method constructs footprint matrices for two computational graphs. The footprint matrix represents the topological relationships between operators in the computational graphs. The value of each element in the footprint matrix is ​​the reciprocal of the shortest distance from the operator in the row of the element to the operator in the column of the element, or vice versa. A similarity matrix can be iteratively calculated based on the footprint matrices of the two computational graphs. The similarity matrix indicates the similarity of the topological relationships between the two computational graphs. After iteration stops, one or more preferred mapping relationships can be determined based on the footprint matrices. Therefore, the operator mapping method provided in this application can support operator mapping between computational graphs with similar structures and is applicable to scenarios involving model transfer within deep learning frameworks.

[0090] The operator mapping method for computational graphs in this application is mainly applied to computational graphs of neural networks, which can be neural network models used for image recognition, speech recognition, etc. This operator mapping method can be applied to scenarios involving neural network model transfer between different deep learning frameworks, between different versions of the same deep learning framework, or in scenarios involving comparison of neural networks between different hardware, as well as in scenarios involving comparison of training and inference results, comparison of integrated circuit-related design graphs, comparison of computational graphs of program code, etc. This application does not limit the specific application scenario; it only applies to scenarios involving comparison and mapping between two graphs represented in graph theory.

[0091] Take the scenario of model transfer between different deep learning frameworks as an example. Figure 1a and Figure 1b Schematic diagrams of application scenarios according to an embodiment of this application are shown respectively. Figure 2 A block diagram of an apparatus for performing an operator mapping method of a computation graph according to an embodiment of this application is shown.

[0092] like Figure 1aThe illustration shows an application scenario for manual transfer. When transferring a model from Deep Learning Framework 1 to Deep Learning Framework 2, the corresponding computation graph 1 and numerical archive 1 can be exported from the model script of Deep Learning Framework 1. The numerical archive can include data such as the tensors and weights of the operators in the computation graph. Developers can write model scripts for Deep Learning Framework 2 based on the model scripts of Deep Learning Framework 1, and export the corresponding computation graphs from the model scripts of Deep Learning Framework 2. Figure 2 Sum of numerical data 2.

[0093] like Figure 2 The device shown can be a computer, server, or other similar device. Figure 2 The device shown may include a processor and memory. The processor may run an operating system, and computational graph comparison software may be installed on top of the operating system. The processor may be a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a system-on-chip (SoC), a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0094] The memory can be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions. Memory can also include non-volatile memory, such as flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory can also include combinations of the above types of memory. Memory can also be electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having an instruction or data structure form and accessible by a computer, but is not limited to these. Memory can exist independently and be connected to the processor via communication lines, such as... Figure 2 The image shows a hard disk storage device. Memory can also be integrated with the processor, such as... Figure 2 The memory shown is typically non-volatile. The memory stores computer execution instructions related to the scheme described in this application, and its execution is controlled by a processor. The processor executes the computer execution instructions stored in the memory to implement the method provided in this application.

[0095] The computation graph comparison software can be a set of instructions that integrates the method provided in this application. The processor runs the instructions to process the input computation graph, which can realize the mapping relationship between the operators of the computation graph to obtain the mapping result. Based on the mapping result, the computation graph and the numerical file, the mapping result can be detected.

[0096] Specifically, the exported computation graph 1 and numerical archive 1 of deep learning framework 1, and the computation of deep learning framework 2 can be used. Figure 2 The numerical data file 2 is input into the processor, which executes the operator mapping method of the computation graph provided in this application to find the computation graph 1 of deep learning framework 1 and the computation graph of deep learning framework 2. Figure 2 Mapping relationships between operators in (e.g.) Figure 1a The steps shown in the solid-line box are implemented using the operator mapping method provided in this application. Figure 1a The steps shown in the dashed box are the preprocessing steps to obtain the computation graph or the subsequent adjustments and processing based on the mapping results, which are part of the application scenario of this application (the steps shown in the dashed box are not the focus of this application). Then, the processor can compare whether the Tensors corresponding to the operators in the corresponding numerical files 1 and 2 are consistent according to the mapping relationship, and output the comparison result of the mapping relationship and the numerical files as a comparison report. If the comparison result of the numerical files is inconsistent, or if the developers determine that the main structure of the computation graph is inconsistent based on the mapping relationship, they can modify the model script of the deep learning framework 2 and then repeat the above process until the comparison result of the numerical files is consistent and the main structure of the computation graph is consistent.

[0097] like Figure 1b The illustration shows an application scenario for automatic transfer. The model script of Deep Learning Framework 1 can be converted into an Open Neural Network Exchange (ONNX) format file. ONNX is a standard for representing deep learning models, enabling model transfer between different deep learning frameworks. Then, a script conversion tool is used to convert the ONNX file to obtain the model script of Deep Learning Framework 2. Computational graph 1 can be exported from the ONNX file, and computational graph 2 can be exported from the model script of Deep Learning Framework 2. Figure 2 Calculate Figure 1 and calculate Figure 2 Input to Figure 2 In the device shown, the operator mapping method provided in this application can be executed by the processor to obtain the computation diagram 1 and the computation diagram 2. Figure 2 The mapping relationship of the operators in the algorithm is determined, and the mapping result is verified based on the mapping relationship.

[0098] Figure 1a and Figure 1b In the application scenarios shown, deep learning framework 1 and deep learning framework 2 can be any of TensorFlow, PyTorch, MindSpore, etc. It should be noted that the above... Figure 1a and Figure 1b This is merely an example of an application scenario for this application and does not limit this application in any way. The operator mapping method of the embodiments provided in this application can also be applied to other application scenarios.

[0099] Figure 3A flowchart illustrating an operator mapping method for a computation graph according to an embodiment of this application is provided. The operator mapping method provided in this application can be used to perform operator mapping on operators in a first computation graph and a second computation graph. The first computation graph and the second computation graph are computation graphs of the same neural network model. The first computation graph and the second computation graph can be computation graphs corresponding to different versions of the same deep learning framework, or computation graphs corresponding to different deep learning frameworks; this application does not limit this. Figure 3 The method steps shown can be derived from Figure 2 The processor shown executes the commands.

[0100] like Figure 3 As shown, an embodiment of the operator mapping method for a computation graph provided in this application may include the following steps:

[0101] Step S300: Based on the first footprint matrix and the second footprint matrix, repeat the following process until the stopping condition is met: calculate the first matrix according to the first footprint matrix and the second footprint matrix, adjust the order of rows and columns in the first footprint matrix according to the first matrix, and recalculate the first matrix according to the adjusted first footprint matrix and the second footprint matrix.

[0102] Wherein, the first footprint matrix indicates the topological relationship between operators in the first computation graph, the second footprint matrix indicates the topological relationship between operators in the second computation graph, and the first matrix indicates the similarity of the topological relationships between the first computation graph and the second computation graph.

[0103] Step S301: Based on the first footprint matrix and the second footprint matrix when the stopping condition is met, obtain the corresponding operators of each of the multiple operators in the first computation graph in the second computation graph.

[0104] In one possible implementation, the value of each element in the first footprint matrix can be the reciprocal of the shortest distance from the operator corresponding to the row containing that element to the operator corresponding to the column containing that element. The first footprint matrix describes the topological relationships between operators in the first computation graph through the reciprocals of the shortest distances between operators. Similarly, the value of each element in the second footprint matrix can be the reciprocal of the shortest distance from the operator corresponding to the row containing that element to the operator corresponding to the column containing that element. The second footprint matrix describes the topological relationships between operators in the second computation graph through the reciprocals of the shortest distances between operators.

[0105] In one possible implementation, the processor can traverse each operator in the first and second computational graphs respectively, determining the shortest distance from each operator to other operators in its respective computational graph. Specifically, the processor can use the traversed operator as a starting point to perform a depth-first search (DFS) traversal of the computational graph to determine the shortest distance from that operator to other operators. Since both the first and second computational graphs are directed graphs, some operators are unreachable from other operators. In this case, the shortest distance from an operator to an unreachable operator can be determined to be infinity, and the reciprocal of the shortest distance from an operator to an unreachable operator is 0. Therefore, the element corresponding to the operator in the footprint matrix has a value of 0. Furthermore, if there are more than one path from one operator to another, and these paths have different distances, the shortest path can be determined as the shortest distance from that operator to the other operator. The reciprocal of the shortest distance from an operator to itself is 0.

[0106] Figure 4 A schematic diagram illustrating a calculation graph according to an example of this application is shown. Figure 4 To illustrate the process of determining the footprint matrix based on the computational graph in this application embodiment, let's assume... Figure 4 The calculation diagram shown is the first calculation diagram.

[0107] The processor performs a depth-first traversal starting from node (operator) A, and obtains the shortest distances from operator A to other operators B, C, D, E, and F as 1, 2, 3, 2, and 3, respectively. For operator E, there are two paths from A to operator E, with distances of 2 and 3, respectively. Therefore, the shortest distance from operator A to operator E is 2.

[0108] The processor performs a depth-first traversal starting from node B, and the shortest distances from node B to nodes A, C, D, E, and F are ∞, ∞, ∞, 1, and 2, respectively.

[0109] The processor performs a depth-first traversal starting from node C, and the shortest distances from node C to nodes A, B, D, E, and F are ∞, ∞, 1, 2, and 3, respectively.

[0110] The processor performs a depth-first traversal starting from node D, and obtains the shortest distances from node D to nodes A, B, C, E, and F as ∞, ∞, ∞, 1, and 2, respectively.

[0111] The processor performs a depth-first traversal starting from node E, and obtains the shortest distances from node E to nodes A, B, C, D, and F as ∞, ∞, ∞, ∞, and 1, respectively.

[0112] The processor performs a depth-first traversal starting from node F, and obtains the shortest distances from node F to nodes A, B, C, D, and E as ∞, ∞, ∞, ∞, ∞, respectively. The shortest distance from each node to itself is ∞.

[0113] Based on the above process, it can be concluded that... Figure 4 The first footprint matrix of the first computational graph shown can be:

[0114] A B C D E F A 0 1 1 / 2 1 / 3 1 / 2 1 / 3 B 0 0 0 0 1 1 / 2 C 0 0 0 1 1 / 2 1 / 3 D 0 0 0 0 1 1 / 2 E 0 0 0 0 0 1 F 0 0 0 0 0 0

[0115] Based on the above process, the first footprint matrix of the first computational graph and the second footprint matrix of the second computational graph can be determined. It should be noted that... Figure 4 The example shown is merely one example of this application and does not limit this application in any way.

[0116] In one possible implementation, when the processor iterates through the shortest distances from one operator to another, if the shortest distance exceeds the maximum access distance, the reciprocal of the shortest distance between the two operators can be set to 0. The maximum access distance can be a distance threshold set to limit computational complexity; for example, in this embodiment, the maximum access distance can be set to 15. When the shortest distance from another operator to the starting operator exceeds the maximum access distance, the impact of that other operator on the topology of the starting operator is relatively small and can therefore be ignored. This increases the sparsity of the footprint matrix and reduces computational complexity.

[0117] In one possible implementation, cosine similarity can be used to measure the similarity between two non-zero vectors. Cosine similarity refers to the cosine of the angle between two non-zero vectors; a larger cosine value indicates that the two vectors point closer together. For example, suppose two non-zero vectors... and Cosine similarity can be expressed by the following formula (1),

[0118]

[0119] In embodiments of this application, a first matrix is ​​used to indicate the similarity of the topological relationships between the first computation graph and the second computation graph. Therefore, in embodiments of this application, the first matrix can also be called a similarity matrix (similarity matrix of the first and second computation graphs). In one possible implementation, the value of an element in the first matrix represents the similarity of the topological relationships between the operator corresponding to the row and the operator corresponding to the column of the element. In embodiments of this application, features of the operators of the first and second computation graphs can be extracted and expressed in the form of feature vectors. Based on the feature vectors of the operators of the first and second computation graphs, the cosine similarity of the operators of the first and second computation graphs is obtained to describe the similarity of the topological relationships between the operators of the first and second computation graphs.

[0120] Since the footprint matrix is ​​a matrix representing the topological relationships between operators in a computation graph, in one possible implementation, the processor can construct eigenvectors of operators in the computation graph based on the footprint matrix, and the cosine similarity of the eigenvectors of two operators can be used to measure the similarity of the topological relationships between the two operators.

[0121] In one possible implementation, step S300, calculating the first matrix based on the first footprint matrix and the second footprint matrix, may include:

[0122] The first matrix is ​​obtained based on multiple first eigenvectors and multiple second eigenvectors, wherein each first eigenvector corresponds to an operator in the first computation graph, and the row vector and column vector corresponding to each first eigenvector and the operator corresponding to the first eigenvector in the first footprint matrix are related. Each second eigenvector corresponds to an operator in the second computation graph, and the row vector and column vector corresponding to each second eigenvector and the operator corresponding to the second eigenvector in the second footprint matrix are related. The first eigenvectors and the second eigenvectors are used to calculate the similarity of the topological relationship between the operator corresponding to the first eigenvector and the operator corresponding to the second eigenvector.

[0123] In other words, each operator in the first computational graph has a corresponding first eigenvector. This first eigenvector is related to the row and column vectors corresponding to the operator in the first footprint matrix. The row vector corresponding to the operator in the first computational graph in the first footprint matrix can be the row vector of the operator in the row corresponding to the operator in the first footprint matrix, and the corresponding column vector can be the column vector of the operator in the column corresponding to the operator in the first footprint matrix. Similarly, each operator in the second computational graph has a corresponding second eigenvector. This second eigenvector is related to the row and column vectors corresponding to the operator in the second footprint matrix. The row vector corresponding to the operator in the second computational graph in the second footprint matrix can be the row vector of the operator in the row corresponding to the operator in the second footprint matrix, and the corresponding column vector can be the column vector of the operator in the column corresponding to the operator in the second footprint matrix.

[0124] In other words, the processor can obtain the first eigenvector of the operator in the first computational graph based on the row vectors and column vectors in the first footprint matrix, and can obtain the second eigenvector of the operator in the second computational graph based on the row vectors and column vectors in the second footprint matrix.

[0125] In one possible implementation, the first eigenvector and the second eigenvector are used to calculate the similarity of the topological relationship between the operators corresponding to the first eigenvector and the operators corresponding to the second eigenvector. This can be achieved by calculating the cosine similarity between the first and second eigenvectors corresponding to the two operators. For example, the cosine similarity between the first eigenvector of operator A in the first computation graph and the second eigenvector of operator B in the second computation graph is used as the similarity of the topological relationship between operator A and operator B. Thus, the value of each element in the first matrix represents the similarity of the topological relationship between the operator corresponding to the row and the operator corresponding to the column of the element, where the operator corresponding to the row and the operator corresponding to the column are respectively the operators in the first and second computation graphs.

[0126] In one possible implementation, the operators corresponding to the rows of the first matrix can be operators of the first computation graph, and the operators corresponding to the columns can be operators of the second computation graph. Alternatively, in another possible implementation, the operators corresponding to the rows of the first matrix can be operators of the second computation graph, and the operators corresponding to the columns can be operators of the first computation graph.

[0127] Figure 5a A flowchart illustrating the process of calculating a first matrix according to an embodiment of this application is shown.

[0128] like Figure 5a As shown, in one possible implementation, the first matrix is ​​obtained based on multiple first eigenvectors and multiple second eigenvectors, which may specifically include the following process:

[0129] Step S3010: Determine the first eigenvector of the operator in the first computational graph based on the first footprint matrix, and determine the second eigenvector of the operator in the second computational graph based on the second footprint matrix;

[0130] Step S3011: Calculate the first matrix based on the first eigenvector and the second eigenvector.

[0131] In one possible implementation, the processor can concatenate the row vector of the operator in the corresponding footprint matrix and the column vector of the corresponding column to obtain the eigenvector of the operator. For example, the processor can concatenate the elements of the row vector to the end of the column vector to obtain the eigenvector of the operator, or it can concatenate the elements of the column vector to the end of the row vector to obtain the eigenvector of the operator. This application does not limit the specific concatenation method, but the concatenation method of the row vector and column vector of the operator in the first computation graph is the same as the concatenation method of the row vector and column vector of the operator in the second computation graph.

[0132] For example, with Figure 4 Taking operator A in the example shown, operator A is in the first row of the footprint matrix, and the first row vector is [0,1,1 / 2,1 / 3,1 / 2,1 / 3]. Operator A is in the first column of the footprint matrix, and the first column vector is [0,0,0,0,0,0]. By concatenating the column vector and the row vector, we can obtain the feature vector of operator A as [0,0,0,0,0,0,0,0,1,1 / 2,1 / 3,1 / 2,1 / 3].

[0133] After determining the eigenvectors of the operators, the cosine similarity between the first eigenvector of the operator in the first computational graph and the second eigenvector of the corresponding operator in the second computational graph can be calculated according to formula (1) to obtain the similarity of the operators. The similarity of the operators in the first computational graph and the operators in the second computational graph can form a first matrix.

[0134] For example, suppose the first computational graph is Q and the second computational graph is R.

[0135] The first footprint matrix of the first computational graph Q is:

[0136] q(1) q(2) q(3) … q(n-1) q(n) q(1) 0 … … … … … q(2) … 0 … … … … q(3) … … 0 … … … … … … … 0 … … q(n-1) … … … … 0 … q(n) … … … … … 0

[0137] The second footprint matrix of the second computational graph R is:

[0138] r(1) r(2) r(3) … r(n-1) r(n) r(1) 0 … … … … … r(2) … 0 … … … … r(3) … … 0 … … … … … … … 0 … … r(n-1) … … … … 0 … r(n) … … … … … 0

[0139] It should be noted that the operators corresponding to the rows in the footprint matrix can be used as the starting point of the access path and the operators corresponding to the columns can be used as the ending point of the access path. Alternatively, the operators corresponding to the columns in the footprint matrix can be used as the starting point of the access path and the operators corresponding to the rows can be used as the ending point of the access path. This application does not limit this.

[0140] The processor can determine the first eigenvectors of operators q(1), q(2)...q(n) in the first computation graph based on the first footprint matrix mentioned above, and determine the second eigenvectors of operators r(1), r(2)...r(n) in the second computation graph based on the second footprint matrix mentioned above.

[0141] The first eigenvector of operators q(1), q(2)...q(n) can be expressed as:

[0142]

[0143] In the table above, the vector formed by the data in the column containing the operator can represent the first eigenvector of the operator. The vector above the thick solid line can be the column vector of the column containing the operator, and the vector below the thick solid line can be obtained by concatenating the row vector of the row containing the operator into the column vector.

[0144] The second eigenvector of operators r(1), r(2)...r(n) can be expressed as:

[0145]

[0146] In the table above, the vector formed by the data in the column containing the operator can represent the second eigenvector of the operator. The vector above the thick solid line can be the column vector of the column containing the operator, and the vector below the thick solid line can be obtained by concatenating the row vector of the row containing the operator into the column vector.

[0147] For example, the similarity between operators q(1) and r(1) can be calculated using the following formula (2):

[0148]

[0149] in, Let q(1) be the first eigenvector of the operator. Let r(1) represent the second eigenvector of operator r(1). According to the above formula (2), the similarity between operators q(1) and r(1) can be calculated. The similarity between the operators of the first computation graph and the operators of the second computation graph can form the similarity matrix of the first computation graph and the second computation graph as shown in the table below:

[0150]

[0151]

[0152] In step S300, the processor adjusts the order of rows and columns in the first footprint matrix according to the first matrix. This can mean adjusting the order between rows and between columns in the first footprint matrix while keeping the order of rows and columns in the second footprint matrix unchanged. It should be noted that, in the embodiments of this application, adjusting the order of rows and columns in the first footprint matrix is ​​merely an example; the order of rows and columns in the second footprint matrix can also be adjusted while keeping the order of rows and columns in the first footprint matrix unchanged.

[0153] In one possible implementation, the specific process by which the processor adjusts the order of rows and columns in the first footprint matrix according to the similarity matrix may include: the processor determining the mapping relationship between operators in the first computation graph and the second computation graph according to the similarity matrix, and adjusting the order of rows and columns in the first footprint matrix corresponding to operators in the first computation graph according to the mapping relationship.

[0154] For example, the processor can determine the similarity between operators in the first and second computational graphs based on the values ​​of elements in the similarity matrix, and determine the mapping relationship between the operators in the first and second computational graphs. For instance, it can select the largest few element values ​​in the similarity matrix, determine if the two operators corresponding to these largest element values ​​have a mapping relationship, and if the two operators with a mapping relationship correspond to different rows and columns in the first and second footprint matrices, the relationship between the rows and columns in the first footprint matrix can be adjusted so that the two operators with a mapping relationship correspond to the same rows and columns in both matrices. If it is determined that the two operators with a mapping relationship correspond to the same rows and columns in both matrices, the order of the rows and columns in the first footprint matrix can be left unchanged, and the larger element value can be selected from the similarity matrix, and the above process can be repeated.

[0155] It should be noted that the above is merely an example of adjusting the order of rows and columns of the first footprint matrix based on the similarity matrix, and does not limit this application in any way.

[0156] When adjusting the row and column order of the first footprint matrix, rows and columns with the same index should be adjusted simultaneously. For example, to adjust the order of rows and columns in the first footprint matrix, assuming the positions of the first and second rows, and the first and second columns are adjusted, the adjusted first footprint matrix can be represented as follows:

[0157] q(2) q(1) q(3) … q(n-1) q(n) q(2) 0 … … … … … q(1) … 0 … … … … q(3) … … 0 … … … … … … … 0 … … q(n-1) … … … … 0 … q(n) … … … … … 0

[0158] After adjusting the order of rows and columns in the first footprint matrix, the mapping relationship between operators in the first and second computational graphs changes, and the eigenvectors of the operators are also different from those before the adjustment. Therefore, the similarity matrix can be recalculated based on the adjusted first and second footprint matrices.

[0159] Repeat the above process until a stopping condition is reached. This stopping condition can be a condition that indicates the cessation of adjusting the first footprint matrix and calculating the first matrix. In one possible implementation, the stopping condition could be a condition satisfied by the first matrix obtained through multiple calculations; for example, the sum of the element values ​​in the first matrix is ​​greater than a similarity threshold. Alternatively, a similarity-related objective function could be set, and the stopping condition is reached when the objective function's value is maximized.

[0160] For example, the objective function of the iteration can be set as follows: Here, M represents the set of mapping relationships between operators, and m represents the mapping relationship between two specific operators in this set of mapping relationships. If, based on the mapping relationship between the operators corresponding to the adjusted first footprint matrix and the second footprint matrix, the value of the objective function calculated satisfies a certain range, or no longer changes relative to the value of the already calculated objective function, then the processor can determine that the stopping condition has been reached.

[0161] In another possible implementation, the stopping condition could be the number of iterations. For example, the stopping condition could be that the number of times the first matrix is ​​repeatedly calculated exceeds a first threshold. That is, after calculating the first matrix a number of times corresponding to the first threshold, the order of the rows and columns of the first footprint matrix is ​​no longer adjusted. This application does not limit this.

[0162] In one possible implementation, there is a mapping relationship between the operators of the first computation graph and the operators of the second computation graph, where the corresponding rows or columns in the first and second footprint matrices are in the same position when the stopping condition is met.

[0163] After the stopping condition is met, in step S301, the processor can determine that the operators corresponding to the same rows or columns in the first and second footprint matrices at the time the stopping condition is met are mapping operators. Rows at the same position can refer to rows with the same row number. For example, the first row of the first footprint matrix and the second row of the second footprint matrix may have the same row position, indicating a mapping relationship between the operators corresponding to the first row of the first footprint matrix and the operators corresponding to the second row of the second footprint matrix. Alternatively, multiple mapping operators can be determined for one operator based on the calculated similarity matrix. For instance, the similarity of one operator to other operators can be sorted based on the calculated similarity matrix values, and the operators with the highest similarity to that operator can be determined as its mapping operators.

[0164] In one possible implementation, the operators corresponding to the same row positions in the first footprint matrix, the adjusted first footprint matrix, and the second footprint matrix are of the same type, and the tensors output by the operators have the same dimension. That is, the operators corresponding to the same row positions in the first footprint matrix and the second footprint matrix are of the same type, and the tensors output by the operators have the same dimension.

[0165] When constructing the first footprint matrix and the second footprint matrix, and adjusting the first footprint matrix, it is ensured that the operators corresponding to the same rows and columns in the first and second footprint matrices have the same type and the same dimension of the tensor output by the operators. In other words, the operators corresponding to the same rows and columns in the first and second footprint matrices are functionally identical. A similarity matrix is ​​used to measure the similarity of the topological relationships between the operators. According to the operator mapping method provided in this application, the mapping relationship between two operators can be reflected in terms of both function and topological relationship, thereby ensuring the correctness of the mapping result.

[0166] According to the operator mapping method of the computation graph provided in this application, one or more preferred mapping relationships can be obtained through iterative calculation. Therefore, it can support operator mapping between computation graphs with similar structures and is applicable to scenarios where deep learning frameworks perform model transfer.

[0167] In one possible implementation, the operators in the first computation graph and the second computation graph belong to multiple first partitions, the operators in the same first partition have the same type, and the tensors output by the operators have the same dimension; some operators in the first computation graph and the second computation graph belong to second partitions, the operators in the second partitions have different types and / or the tensors output by the operators have different dimensions.

[0168] Operators of the same type and whose output tensors have the same dimension can be called homogeneous operators; therefore, the first partition can also be called a homogeneous partition. In the second partition, operators of different types and / or whose output tensors have different dimensions are not homogeneous operators; therefore, the operators in the second partition are not homogeneous operators, and the second partition can also be called a mixed partition.

[0169] In one possible implementation, the processor can classify the operators of the first computation graph and the operators of the second computation graph according to the type of the operator and the dimension of the tensor output by the operator, grouping homogeneous operators into one category, with each category of homogeneous operators belonging to a first partition. Each first partition may include at least some of the operators of the first and / or second computation graphs.

[0170] Some operators in the first and second computational graphs may not have homogeneous operators. These operators can belong to the second partition (hybrid partition), which can include some operators from the first and / or second computational graphs. For operators in both the first and second computational graphs, all operators without homogeneous operators belong to the hybrid partition. For example, some operators in the computational graph are unmappable operator types, or there are no operators with the same output operand dimension; these operators do not have homogeneous operators. For instance, MS's `tuple_getitem` type operator and TF's `Identity` type operator cannot find a functionally equivalent corresponding operator type in each other's frameworks. For these operators, they can be assigned to the hybrid partition. When determining the mapping operator, mapping suggestions can be output based on the first footprint matrix when the stopping condition is met, the second footprint matrix, and the calculated similarity matrix.

[0171] In one possible implementation, there is a mapping relationship between the operators of the first computation graph and the operators of the second computation graph that belong to the same first partition and whose corresponding rows or columns in the first footprint matrix and the second footprint matrix are in the same position when the stopping condition is reached.

[0172] In one possible implementation, there is a mapping relationship between the operators of the first computation graph belonging to the second partition and multiple operators of the second computation graph belonging to the second partition. The multiple operators of the second computation graph belonging to the second partition are: the operators of the second computation graph corresponding to the largest values ​​among the elements corresponding to the operators of the first computation graph belonging to the second partition in the first matrix when the stopping condition is met. Based on the first footprint, the second matrix, and the first matrix when the stopping condition is met, mapping suggestions between the operators of the first computation graph in the mixed partition and multiple operators of the second computation graph can be output, or mapping suggestions between the operators of the second computation graph in the mixed partition and multiple operators of the first computation graph can be output. For example, for the operators of the first computation graph in the mixed partition, the multiple operators in the most similar second computation graph (top-k, where k is the number of operators suggested when outputting mapping suggestions, and k is a positive integer) are selected and output based on the similarity values ​​corresponding to the operator in the similarity matrix.

[0173] Because there is a higher probability of mapping relationships between homogeneous operators, and in the embodiments of this application, there is a mapping relationship between operators corresponding to the same row or column in the first footprint matrix and the second footprint matrix. Therefore, when determining the first footprint matrix and the second footprint matrix, the processor can arrange the operators of the first computation graph in the rows corresponding to the first footprint matrix and the operators of the second computation graph in the rows corresponding to the second footprint matrix according to the first partition, and arrange the operators of the first computation graph in the columns corresponding to the first footprint matrix and the operators of the second computation graph in the columns corresponding to the second footprint matrix according to the first partition.

[0174] Specifically, in one possible implementation, the operators corresponding to the same row positions in the first footprint matrix and the second footprint matrix are in the same first partition, and the operators corresponding to the same column positions are in the same first partition; furthermore, the operators of the first computation graph belonging to the same first partition are arranged consecutively in the corresponding rows or columns of the first footprint matrix, and the operators of the second computation graph belonging to the same first partition are arranged consecutively in the corresponding rows or columns of the second footprint matrix. That is, the row numbers of the operators of the first computation graph belonging to the same first partition in the first footprint matrix and the row numbers of the operators of the second computation graph belonging to the same first partition in the second footprint matrix have the same range, and the column numbers of the operators of the first computation graph belonging to the same first partition in the first footprint matrix and the column numbers of the operators of the second computation graph belonging to the same first partition in the second footprint matrix have the same range. For example, operators q(1), q(2), and q(3) of the first computation graph belonging to the same first partition are in rows 1-3 and columns 1-3 of the first footprint matrix. Operators r(1), r(2), and r(3) of the second computation graph belonging to the same first partition as operators q(1), q(2), and q(3) are in rows 1-3 and columns 1-3 of the second footprint matrix.

[0175] Figure 6 A flowchart illustrating the construction of a footprint matrix according to an embodiment of this application is shown. Figure 6 As shown, in one possible implementation, the process of determining the first footprint matrix of the first computational graph and the second footprint matrix of the second computational graph may include:

[0176] Step S3000: Based on the type of the operator and the dimension of the tensor output by the operator, the operators in the first computation graph and the second computation graph are partitioned to obtain the first partition; wherein, the operators located in the same first partition are homogeneous operators, and homogeneous operators have the same type and the same dimension of the tensor output by the operator.

[0177] Step S3002: Determine the first footprint matrix of the first computational graph and the second footprint matrix of the second computational graph according to the first partition.

[0178] In this embodiment, adjusting the order of rows and columns in the first footprint matrix according to the first matrix in step S300 may include: adjusting the order of rows and columns corresponding to operators located in the same first partition in the first footprint matrix according to the first matrix.

[0179] By homogeneously partitioning the operators in the first and second computational graphs, and adjusting the order of rows and columns in the first footprint matrix by adjusting the order of rows and columns corresponding to operators located in the same homogeneous partition, the iterative process can converge more quickly, accelerating the mapping process. This is because homogeneous operators have higher similarity, and there is a greater possibility of mapping relationships between homogeneous operators. Therefore, the homogeneous partitioning process can accelerate the convergence speed of step S300 and speed up the mapping process.

[0180] In step S3000, the processor can group operators of the same type and whose output tensors have the same dimension into the same homogeneous partition in both the first and second computation graphs. In other words, operators of the same type and whose output tensors have the same dimension are located in the same homogeneous partition in both the first and second computation graphs.

[0181] Figure 7 A schematic diagram of homogeneous partitioning according to an embodiment of this application is shown, such as... Figure 7 As shown, operators q(1), q(2), q(3), q(4) and r(1), r(2), r(3), r(4) are in the same homogeneous partition. The type of operator in this homogeneous partition is convolution, and the dimension of the output operands is 64×55×55. q(5), q(6), q(7) and r(5), r(6), r(7) are in the same homogeneous partition. The type of operator in this homogeneous partition is convolution, and the dimension of the output operands is 256×55×55. Operators q(n) and r(n) are in the same homogeneous partition. The type of operator in this homogeneous partition is pooling, and the dimension of the output operands is m×n×k. It should be noted that... Figure 7 The types of operators in homogeneous partitions, the dimensions of output operands, the number of operators in homogeneous partitions, and their numbering are merely examples in this application and do not limit this application in any way.

[0182] For step S3002, determining the first footprint matrix of the first computation graph and the second footprint matrix of the second computation graph according to the first partition is mainly to place operators located in the same homogeneous partition in adjacent rows and columns in the footprint matrix.

[0183] In the first and second footprint matrices, operators at the same position in the same row belong to the same first partition, and operators at the same position in the same column belong to the same first partition. Furthermore, operators of the first computation graph belonging to the same first partition are arranged consecutively in the corresponding rows or columns of the first footprint matrix, and operators of the second computation graph belonging to the same first partition are arranged consecutively in the corresponding rows or columns of the second footprint matrix. That is, when determining the first and second footprint matrices, for operators in the first and second computation graphs located in the same homogeneous partition, the row and column index range of the operator in the first computation graph in the first footprint matrix is ​​the same as the row and column index range of the operator in the second computation graph in the second footprint matrix. For example, still using... Figure 7 Taking the example shown, suppose the operators q(1), q(2), q(3), and q(4) of the first computation graph correspond to rows 1-4 and columns 1-4 in the first footprint matrix. Then, the operators r(1), r(2), r(3), and r(4) of the second computation graph correspond to rows 1-4 and columns 1-4 in the second footprint matrix. Thus, there is a greater likelihood that there is a mapping relationship between operators corresponding to the same rows and columns in the first and second footprint matrices. When adjusting the order of rows and columns in the first footprint matrix, by adjusting the order of rows and columns corresponding to operators located in the same homogeneous partition in the first footprint matrix, operators with mapping relationships can be quickly moved to the same rows and columns.

[0184] By using the above embodiments of this application to homogeneously partition the operators in the computation graph and determine the corresponding footprint matrix according to the homogeneous partitions, the operators with a higher probability of having a mapping relationship can be quickly identified, thus accelerating the mapping process.

[0185] In one possible implementation, adjusting the order of rows and columns corresponding to operators within the same first partition in the first footprint matrix according to the first matrix may include:

[0186] The mapping relationship between operators in the first computation graph and operators in the second computation graph located in the same first partition is determined based on the first matrix;

[0187] The order of rows and columns in the first footprint matrix corresponding to the operators in the first computation graph is adjusted according to the mapping relationship.

[0188] Specifically, the mapping relationship between operators in the first computation graph and operators in the second computation graph located in the same first partition is determined based on the first matrix. This may include two traversals. The following process is the process of determining the mapping relationship between operators in the same first partition.

[0189] During the first traversal: Starting from the operator in the first computation graph corresponding to the smallest element in a homogeneous partition of the similarity matrix, traverse all operators in that homogeneous partition. Determine the operator in the second computation graph that is most similar to the currently traversed operator based on the largest element in the row or column of the currently traversed operator in the similarity matrix. If the operator in the second computation graph that is most similar to the currently traversed operator has already been determined as the most similar operator of the traversed operator, then in the second traversal, re-determine the most similar operator for the traversed operator. That is, in the first traversal, if there is a conflict between the most similar operator determined by the later traversal and the most similar operator determined by the previous traversal, the result of the later traversal overwrites the result of the previous traversal.

[0190] During the second traversal: For operators in the homogeneous partition that have not yet found the most similar operator, start traversing the operators that have not yet found the most similar operator from the operator corresponding to the largest element in the homogeneous partition of the similarity matrix. Determine the operator in the second computation graph that is most similar to the operator currently being traversed based on the largest element in the row or column of the similarity matrix where the currently traversed operator is located. If the operator in the second computation graph that is most similar to the currently traversed operator has already been determined as the most similar operator of the traversed operator, then determine the operator in the second computation graph that is most similar to the currently traversed operator based on the second largest element value in the row or column of the similarity matrix where the currently traversed operator is located, until the most similar operator of the currently traversed operator is found.

[0191] Figure 5b A flowchart illustrating a method for adjusting the order of rows and columns of a footprint matrix according to an embodiment of this application is provided. In this example, it is assumed that the operators corresponding to the rows of the first footprint matrix are operators of the first computation graph, and the operators corresponding to the columns are operators of the second computation graph. The rows of the similarity matrix correspond to the operators of the first computation graph, and the columns correspond to the operators of the second computation graph. The process of adjusting the footprint matrix according to an embodiment of this application will be explained in conjunction with the similarity matrix in the example above.

[0192]

[0193] Figure 5b The diagram illustrates the process of determining the mapping relationship of operators for an operator on a homogeneous partition, as follows: Figure 5bAs shown, in step S500, the processor can first determine whether it is the first traversal or the second traversal. If it is the first traversal of the operators in the first computation graph, the processor can execute step S510 to determine whether all operators of the homogeneous partition have been traversed. If all operators of the homogeneous partition have been traversed, the processor can return to step S500 and continue to determine whether it is the first traversal or the second traversal. If the operators of the homogeneous partition have not been traversed, the processor can execute step S511 to determine the smallest element in the row corresponding to the untraversed operator in the similarity matrix, and determine the operator corresponding to the smallest element as the operator to be traversed. That is, in the first traversal: the traversal starts from the operator corresponding to the smallest element value in the similarity matrix.

[0194] After determining the operator being traversed, the processor can execute step S512, which determines the operator in the second computational graph that is most similar to the operator being traversed, based on the maximum element value in the row of the similarity matrix where the operator being traversed is located. For example, the operator in the second computational graph corresponding to the maximum element value in the row of the similarity matrix where the operator being traversed is located is determined as the operator most similar to the operator being traversed. In other words, since it has been assumed that the operators corresponding to the rows of the first footprint matrix are operators of the first computational graph, and the operators corresponding to the columns are operators of the second computational graph, and the rows of the similarity matrix correspond to operators of the first computational graph and the columns correspond to operators of the second computational graph, in the similarity matrix, the value of each row element represents the cosine similarity between an operator in the first computational graph and each operator in the second computational graph. The larger the value in this row element, the more similar the operators in the first and second computational graphs are. Therefore, the processor can determine the operator in the second computation graph corresponding to the largest element value in the row of the similarity matrix of the currently traversed operator as the operator most similar to the currently traversed operator.

[0195] After determining the most similar operator of the currently traversed operator, the processor can execute step S513 to determine whether the most similar operator has matched the previously traversed operator. If the most similar operator has not matched the previously traversed operator, the processor can execute step S515 to record the mapping relationship between the currently traversed operator and the most similar operator, mark the currently traversed operator as the traversed operator, and return to step S510 to continue to determine whether all operators of the homogeneous partition have been traversed.

[0196] If the processor determines that the most similar operator matches a previously traversed operator, the processor can execute step S514 to delete the mapping relationship between the previously traversed operator and the most similar operator, and add the previously traversed operator to the queue for the second traversal. Then, it continues to execute step S515. For steps S514 and S515, the processor's method of recording and deleting mapping relationships is merely an example of this application; this application is not limited to this. Other methods can also be used to ensure that when the most similar operator determined in the later traversal conflicts with the most similar operator determined in the earlier traversal, the result of the later traversal overwrites the result of the earlier traversal. Similarly, adding the previously traversed operator to the queue for the second traversal is only one way to implement the second traversal. The processor can also implement this process by adding special identifiers to the operators. This application does not limit the specific implementation method.

[0197] After the first traversal of the operators in the homogeneous partition according to the above process, the second traversal begins. The processor can execute step S520 to determine whether all operators in the homogeneous partition have been traversed, that is, whether the operators that have not yet been matched after the first traversal have been traversed. After determining that all operators in the homogeneous partition have been traversed, the processor can end the process of determining the mapping relationship. After determining that all operators in the homogeneous partition have not been traversed, the processor can execute step S521 to determine the largest element in the row corresponding to the untraversed operator in the similarity matrix, and take the operator corresponding to the largest element as the current traversal operator. That is, as mentioned above, in the second traversal process, for operators that have not yet matched the most similar operator, the traversal starts from the operator corresponding to the largest element in the similarity matrix.

[0198] After determining the operator being traversed, the processor can execute step S522 to determine the operator in the second computation graph that is most similar to the operator being traversed, based on the largest element value in the row where the operator being traversed is located in the similarity matrix. The specific determination method is detailed in step S512 and will not be repeated here. The processor can execute step S523 to determine whether the most similar operator has already matched a previously traversed operator. The specific process is detailed in step S513. The difference is that when the processor determines that the most similar operator matches a previously traversed operator, it no longer overwrites the results of the previous traversal. Instead, it executes step S524 to determine the operator in the second computation graph that is most similar to the operator being traversed, based on the second largest element value in the row where the operator being traversed is located in the similarity matrix. That is, it continues to search for other operators most similar to the currently traversed operator until it finds an operator that has not matched a previously traversed operator and is "most similar" to the currently traversed operator, and then executes step S525.

[0199] By performing the above two traversals on operators within all homogeneous partitions, the processor can more quickly determine the optimal local (stage) mapping relationship between operators in the first and second computational graphs, and adjust the order of rows and columns in the first footprint matrix corresponding to operators in the first computational graph according to the mapping relationship. Specifically, according to the arrangement order of operators corresponding to each row (column) of the second footprint matrix, the mapping operators of operators in the second computational graph in the first computational graph are determined according to the above mapping relationship, and the rows (columns) corresponding to the mapping operators in the first footprint matrix are adjusted to be the same as the rows (columns) of operators in the second computational graph in the second footprint matrix.

[0200] The method for adjusting the first footprint matrix provided in the above embodiments of this application can more quickly converge the calculation results to the stopping condition of the iteration and more quickly obtain the mapping relationship between operators, which is beneficial to accelerating the mapping process. It should be noted that the above example is only one example of adjusting the order of rows and columns of the first footprint matrix provided in this application, and this application is not limited thereto.

[0201] In one possible implementation, the homogeneity depth of operators corresponding to the same position in the first footprint matrix and the second footprint matrix is ​​the same, and the homogeneity depth of operators corresponding to the same position in the second footprint matrix is ​​the same. Furthermore, the rows or columns corresponding to operators of the first computation graph belonging to the same first partition are sorted in the first footprint matrix according to the homogeneity depth of the operators, and the rows or columns corresponding to operators of the second computation graph belonging to the same first partition are sorted in the second footprint matrix according to the homogeneity depth of the operators. The homogeneity depth is defined as the maximum number of operators of the same type and output tensor dimension as the operator in all branches to which the operator belongs in the computation graph.

[0202] In one possible implementation, the operator mapping method provided in this application may further include the following steps:

[0203] The homogeneity depth of an operator is determined based on its type, the dimension of the tensor output by the operator, and the order of the topological structure of the operator in the first or second computation graph.

[0204] The operators are partitioned to obtain homogeneous partitions. Within each homogeneous partition, the operators of the first and second computational graphs are sorted and numbered according to their homogeneity depth. In other words, within each homogeneous partition, the processor can sort and number the operators of the first computational graph according to their homogeneity depth, and sort and number the operators of the second computational graph according to their homogeneity depth. The specific method of sorting and numbering according to homogeneity depth is not limited. For example, it can be sorted from shallow to deep homogeneity depth and numbered from small to large, or it can be sorted from deep to shallow homogeneity depth and numbered from small to large.

[0205] In this way, when determining the first footprint matrix, the processor can determine it according to the order of operator numbers in the first computation graph within the first partition; when determining the second footprint matrix, the processor can determine it according to the order of operator numbers in the second computation graph within the first partition. This ensures that the homogeneous depth of operators corresponding to the same position in the first and second footprint matrices is the same, the homogeneous depth of operators corresponding to the same position in the columns is the same, and the rows or columns corresponding to operators of the first computation graph belonging to the same first partition in the first footprint matrix are sorted according to the homogeneous depth of operators, and the rows or columns corresponding to operators of the second computation graph belonging to the same first partition in the second footprint matrix are sorted according to the homogeneous depth of operators.

[0206] Distinguishing operators with the same attributes by homogeneous depth avoids the unreliability of distinguishing operators with the same attributes by their names. Furthermore, numbering and sorting homogeneous operators according to their homogeneous depth can accelerate the mapping process, since operators belonging to the same first region and with the same homogeneous depth are more likely to have a mapping relationship.

[0207] Because user naming of operators can be subjective or incomplete, distinguishing operators with the same attributes based on user-named operators presents two problems: it may not be able to distinguish them accurately (due to renaming or missing names) and it is unreliable; another problem is that it may not reflect the mapping relationship between homogeneous operators. The implementation method provided in this application, which distinguishes homogeneous operators with the same attributes based on homogeneity depth, can effectively solve these two problems.

[0208] Figure 8 and Figure 9 Schematic diagrams illustrating the determination of the homogeneity depth of a homogeneous operator according to an embodiment of this application are shown. Figure 8In the computational graph shown, there are four homogeneous operators Conv. The homogeneity depth of the first operator Conv is 1, and the homogeneity depth of the second operator Conv is 2. This is because, starting from the first homogeneous operator (the first Conv operator) and continuing to the second operator (the second Conv operator), the second Conv operator has two homogeneous operators (one is the first Conv operator, and the other is itself). Therefore, the homogeneity depth of the second Conv operator is 2. Similarly, it can be determined that... Figure 8 The homogeneous depth of other operators in the computation graph shown.

[0209] like Figure 9 As shown, the 4th Conv operator from top to bottom ( Figure 9 The operator (with a pattern filled in) belongs to two branches. The homogeneity depth of this operator differs between the two branches. For example, on the left branch, there is a Conv operator before the fourth Conv operator, and the homogeneity depth of the fourth Conv operator is 2. On the right branch, there are two Conv operators before the fourth Conv operator, and the homogeneity depth of the fourth Conv operator is 3. Therefore, the maximum number of homogeneous operators from the first homogeneous operator to this operator is 3. Thus, the homogeneity depth of the fourth Conv operator is 3.

[0210] After determining the homogeneous operators and their homogeneity depth in the first and second computation graphs, the homogeneous operators can be partitioned to obtain homogeneous partitions. Within these partitions, the operators in the two computation graphs are sorted and numbered according to their homogeneity depth.

[0211] In the above embodiments of this application, homogeneous depth is used as an example of the way to represent the topological structure of an operator in a computation graph. This application is not limited to this, and other methods can also be used to represent the topological structure of an operator in a computation graph.

[0212] The method of sorting and numbering homogeneous depth according to the above embodiments of this application can effectively distinguish different homogeneous operators and obtain the mapping relationship between homogeneous operators more quickly, which is beneficial to accelerating the mapping process.

[0213] In one possible implementation, if the number of operators in the first computation graph in the first partition is different from the number of operators in the second computation graph, the first partition also includes pseudo-operators; wherein, pseudo-operators are operators that do not have operator type and operands. If the number of operators in the first computation graph in the second partition is different from the number of operators in the second computation graph, the second partition also includes the pseudo-operators.

[0214] In one possible implementation, if the number of operators in the first computation graph in the first partition is less than the number of operators in the second computation graph in the first partition, the row or column corresponding to the pseudo-operator in the first footprint matrix is ​​located after the row or column corresponding to the operator in the first computation graph belonging to the first partition; if the number of operators in the second computation graph in the first partition is less than the number of operators in the first computation graph in the first partition, the row or column corresponding to the pseudo-operator in the second footprint matrix is ​​located after the row or column corresponding to the operator in the second computation graph belonging to the first partition.

[0215] In one possible implementation, if the number of operators in the first computation graph in the second partition is less than the number of operators in the second computation graph in the second partition, the rows and columns corresponding to the pseudo-operators in the first footprint matrix are randomly sorted with the rows and columns corresponding to the operators in the first computation graph belonging to the second partition, and the random sorting of rows and columns is the same; if the number of operators in the second computation graph in the second partition is less than the number of operators in the second computation graph in the second partition, the rows and columns corresponding to the pseudo-operators in the second footprint matrix are randomly sorted with the rows and columns corresponding to the operators in the second computation graph belonging to the second partition, and the random sorting of rows and columns is the same.

[0216] When transferring models between different deep learning frameworks, the number of operators in the computation graph corresponding to the model may differ. However, similarity calculation requires feature vectors of the same length. Therefore, when the dimensions of the first footprint matrix of the first computation graph and the second footprint matrix of the second computation graph are the same, the feature vectors of the operators in the first computation graph determined by the first footprint matrix and the feature vectors of the operators in the second computation graph determined by the second footprint matrix have the same length. To ensure that the footprints of the first footprint matrix of the first computation graph and the second footprint matrix of the second computation graph are the same, the embodiment of the operator mapping method provided in this application may further include:

[0217] If the number of operators in the first computation graph and the number of operators in the second computation graph are different in the homogeneous partition, then add pseudo-operators to the computation graph with fewer operators in the homogeneous partition so that the number of operators in the first computation graph and the second computation graph in the homogeneous partition are the same.

[0218] If the number of operators in the first computation graph and the number of operators in the second computation graph are different in the mixed partition, then a pseudo-operator is added to the computation graph with fewer operators in the mixed partition so that the number of operators in the first computation graph and the second computation graph in the mixed partition are the same.

[0219] When adding pseudo-operators in homogeneous and mixed partitions, the pseudo-operators can be randomly sorted and numbered. In one possible implementation, when adding pseudo-operators in homogeneous partitions, they can be placed after the operators in the computation graph. In mixed partitions, both pseudo-operators and computation graph operators can be randomly sorted and numbered according to their order.

[0220] Pseudo-operators are fictitious fillers, lacking any meaningful operator attributes or topological structure. Their purpose is to ensure that the total number of operators in the first and second computational graphs is equal, thereby constructing footprint matrices of the same size to allow subsequent matrix operations to proceed correctly. Thus, the operator mapping method provided in this application can be applied to scenarios involving the transfer of models between frameworks or systems employing different computational graph representations.

[0221] It should be noted that the above-described order of determining homogeneity depth, partitioning, adding pseudo-operators, sorting, and numbering is merely an example of this application, and this application is not limited to the above order. For example, the processor can also execute the above process in the following order: the processor can determine the homogeneity depth of operators in the first and second computation graphs; partition the operators in the first and second computation graphs to obtain homogeneous partitions and / or mixed partitions; simultaneously, the processor can sort the operators in the first and second computation graphs according to the homogeneity depth of the operators in the homogeneous partitions, and the sorting can be done in descending or ascending order of homogeneity depth, which is not limited in this application; the processor adds pseudo-operators according to the number of operators in the two computation graphs in the homogeneous partitions; the processor can set the homogeneity depth of all pseudo-operators to the maximum (total number of true operators + total number of pseudo-operators); for homogeneous partitions, the pseudo-operators can be sorted according to homogeneity depth; for mixed partitions, the pseudo-operators can be randomly sorted; after that, the processor can number all the operators.

[0222] Figure 10 An example of an insertion pseudo-operator according to an embodiment of this application is shown. For example... Figure 10 As shown, in the first homogeneous partition, two pseudo-operators q(3) and q(4) are added to the computation graph on the left. In the second homogeneous partition, one pseudo-operator r(7) is added to the computation graph on the right. In the mixed partition, two pseudo-operators are added to the computation graph on the left. After adding the pseudo-operators, the number of operators in the first and second computation graphs is the same.

[0223] When determining the first footprint matrix of the first computational graph and the second footprint matrix of the second computational graph, the rows and columns corresponding to the two pseudo-operators q(3) and q(4) in the first footprint matrix are after the rows and columns corresponding to the operators q(1) and q(2), and the rows and columns corresponding to the pseudo-operator r(7) in the second footprint matrix are after the rows and columns corresponding to the operator r(6). Furthermore, the pseudo-operators are unreachable from all other operators; that is, the shortest distance from the pseudo-operators to all other operators is infinite, and the reciprocal of the shortest distance is 0. In other words, the element values ​​corresponding to the pseudo-operators in the footprint matrix are all 0.

[0224] Application Examples

[0225] The operator mapping method for computational graphs provided in this application will be explained below with specific application scenarios and examples.

[0226] Application Scenario and Example Description: This example demonstrates the migration of a ResNet50 model from TensorFlow to MindSpore, assisting in locating problematic scripts during the migration process. The ResNet50 model has 5176 operators in its computation graph on the TensorFlow v1.14 platform and 2513 operators in its computation graph on the MindSpore v1.0.0 platform. In this example, the TensorFlow computation graph is represented as R, and the MindSpore computation graph is represented as Q. The operator mapping method of this application is explained below using this application scenario and example.

[0227] Figure 11 A flowchart illustrating an operator mapping method according to an embodiment of this application is shown. (In conjunction with...) Figure 2 The example shown exports the computation graph R from a model script in TensorFlow v1.14 and the computation graph Q from a model script in MindSpore v1.0.0, then inputs the computation graphs into the processor. The processor can then execute... Figure 11 The operator mapping method is shown.

[0228] The processor can compute homogeneous depths for operators in computation graphs Q and R.

[0229] Still with Figure 8 and Figure 9 For example, as shown in the example, Figure 8 As shown, assuming Figure 8 All operators output tensors with the same dimension. Starting from each operator, traverse upwards along the incoming edges, searching for the number of operators homogeneous (with the same properties) to the starting operator. Increasing this number by one gives the homogeneity depth of the starting operator. Figure 8 The last Conv operator shown is preceded by three Conv operators, and the homogeneity depth of the last Conv operator is 4. Figure 9 In the example shown, there are multiple branches, the fourth Conv operator from the top ( Figure 9 The operator (with a pattern filled in) belongs to two branches. The homogeneity depth of this operator differs between the two branches. For example, on the left branch, there is a Conv operator before the fourth Conv operator, and the homogeneity depth of the fourth Conv operator is 2. On the right branch, there are two Conv operators before the fourth Conv operator, and the homogeneity depth of the fourth Conv operator is 3. Therefore, the maximum number of homogeneous operators from the first homogeneous operator to this operator is 3. Thus, the homogeneity depth of the fourth Conv operator is 3.

[0230] After calculating the homogeneity depth of operators in the computation graph, the processor can perform homogeneous partitioning of the operators in computation graphs Q and R. An example of the processor performing homogeneous partitioning can be found in [reference needed]. Figure 10 .

[0231] The processor can group operators of the same type and with the same output operand dimension size in both computation graphs Q and R into the same homogeneous partition. For operators in both computation graphs Q and R that lack homogeneous operators, a mixed partition can be created. For example, MS's `tuple_getitem` operator and TF's `Identity` operator cannot find a corresponding operator type with the same functionality in each other's frameworks. Such operators can be grouped into a mixed partition.

[0232] After homogeneous partitioning, the two computation graphs have the same number of homogeneous partitions and one mixed partition. Each homogeneous partition differs from the other homogeneous partitions in operator type and / or output operand dimension. The order of the different homogeneous partitions within the same partition is irrelevant. If the number of operators in the two computation graphs is different within the same homogeneous partition, the processor can add pseudo-operators to the computation graph with fewer operators in the homogeneous partition, thereby making the number of operators in the two computation graphs in that homogeneous partition the same. Pseudo-operators are fictitious fillers, lacking any meaningful operator attributes or topological structure. The purpose of pseudo-operators is to make the total number of operators in computation graph Q and computation graph R equal, thereby constructing footprint matrices of the same size to enable subsequent matrix operations.

[0233] After homogeneous partitioning, the processor can number the operators according to a predetermined sorting. Within a homogeneous partition, the processor can sort and number the operators of the two computation graphs separately in ascending order of homogeneity depth. Pseudo-operators added in homogeneous partitions are placed after the real operators in the computation graph. In mixed partitions, pseudo-operators can be placed after the operators in the computation graph, or the pseudo-operators in the mixed partitions and the operators in the computation graph can be randomly sorted; this application does not impose any limitations on this.

[0234] After completing the homogeneous partitioning, the processor can construct the first footprint matrix of the first computational graph and the second footprint matrix of the second computational graph according to the operator numbers in the homogeneous partition. After completing the homogeneous partitioning, the total number of operators in both computational graphs Q and R is 5176. The processor can construct two 5176×5176 footprint matrices according to the operator numbers and initialize all elements of the two footprint matrices to 0.

[0235] Calculate the footprint matrix q of graph Q, initializing it to all zeros:

[0236] q1 q2 q3 … … q5176 q1 0 … … … … 0 q2 … … … … … … q3 … … … … … … … … … … … … … … … … … … … … q5176 0 … … … … 0

[0237] Calculate the footprint matrix r of graph Q, initializing it to 0:

[0238] r1 r2 r3 … … r5176 r1 0 … … … … 0 r2 … … … … … … r3 … … … … … … … … … … … … … … … … … … … … r5176 0 … … … … 0

[0239] The footprint matrices of computation graphs Q and R store the reciprocals of the shortest distances that can be reached between operators in the corresponding computation graphs. Since both computation graphs Q and R are directed graphs, some operators are unreachable, so the footprint matrices can be asymmetric.

[0240] The processor can traverse the operators in computation graphs Q and R separately, find the shortest distance from one operator to another, and determine the footprint matrix based on the shortest distance. Taking computation graph Q as an example, the processor can use each operator in computation graph Q as a starting point and perform a depth-first traversal of operators that can be visited repeatedly. Each time an operator is visited, a footprint value (the reciprocal of the visit distance) that decays with the visit distance is placed in the corresponding element of the column of the starting operator and the row of the visited operator in the footprint matrix. If the visited operator has already been visited in this traversal and has been assigned a larger or equal footprint value, the processor may not update the corresponding element in the footprint matrix and will not visit the direct downstream operator of the visited operator (which has already been visited). The relevant footprint values ​​for all pseudo-operators are 0.

[0241] Figure 12 A schematic diagram illustrating the determination of the shortest distance according to an embodiment of this application is shown. Figure 12As shown, starting from the starting operator, we traverse all other operators to determine the shortest distance from the starting operator to each other operator and the corresponding footprint value. The distance from the starting operator to the previous operator is infinite, therefore, the footprint value is 0. The shortest distance from the starting operator to the operators corresponding to the two connected child nodes is 1, and the corresponding footprint value is 1. Figure 12 The last operator in the sequence is a pseudo-operator; therefore, the footprint value from the starting operator to the pseudo-operator is 0. Based on the above process, each operator is determined sequentially. Figure 12 The footprint value from the starting point operator to other operators. Based on the footprint value, the following can be determined: Figure 12 The footprint matrix corresponding to the computational graph shown.

[0242] By homogeneously partitioning the operators of the first and second computational graphs and determining the homogeneity depth of the operators, the operators are sorted and numbered according to their homogeneity depth within the homogeneous partitions. The first footprint matrix of the first computational graph and the second footprint matrix of the second computational graph are then determined according to the order of the homogeneity depth of the operators within the homogeneous partitions. In this way, the operators corresponding to the same rows and columns in the first and second footprint matrices have the same or similar functions. Furthermore, the first footprint matrix indicates the topological relationship between the operators in the first computational graph and other operators in the first computational graph, while the second footprint matrix indicates the topological relationship between the operators in the second computational graph and other operators in the second computational graph.

[0243] After determining the footprint matrices corresponding to computational graphs Q and R, the processor can calculate the similarity matrix based on the footprint matrices corresponding to computational graphs Q and R.

[0244] In one example, the cosine similarity between operators in computational graphs Q and R can be calculated based on the footprint matrix q and the footprint matrix r to measure the similarity between two operators in the two computational graphs. Specifically, the process is as follows: determine the first eigenvector of the operator in computational graph Q based on the footprint matrix q, and determine the second eigenvector of the operator in computational graph R based on the footprint matrix r; then calculate the similarity matrix between the operators in computational graphs Q and R based on the first and second eigenvectors.

[0245] Taking an operator qk in graph Q as an example, where k is an integer between 1 and 5176, the eigenvector of operator qk can be expressed as:

[0246] Where col(qk) is the column vector corresponding to operator qk in the footprint matrix, row(qk) is the row vector corresponding to operator qk in the footprint matrix, and concat is the concatenation operator, which concatenates the row vector of the operator's row and the column vector of its column in the corresponding footprint matrix to obtain the eigenvector of the operator.

[0247] Based on the determined footprint matrix q and footprint matrix r, the eigenvector matrix of the computational graph Q and the eigenvector matrix of the computational graph R can be obtained.

[0248] The eigenvector matrix of computation graph Q is shown in the table below. The part above the thick solid line represents the column vectors corresponding to the operators in computation graph Q, and the part below the thick solid line represents the row vectors corresponding to the operators in computation graph Q concatenated to the column vectors. The eigenvector matrix of computation graph Q is 10352×5176.

[0249] q1 q2 q3 … … q5176 … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …

[0250] Similarly, the eigenvector matrix of computation graph R is shown in the table below. The part above the thick solid line can be the column vectors corresponding to the operators in computation graph R, and the part below the thick solid line can be the row vectors corresponding to the operators in computation graph R. The eigenvector matrix of computation graph R obtained by concatenating the row vectors below the column vectors is 10352×5176.

[0251] r1 r2 r3 … … r5176 … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …

[0252] Since the first footprint matrix indicates the topological relationship between the operator in the first computation graph and other operators in the first computation graph, and the second footprint matrix indicates the topological relationship between the operator in the second computation graph and other operators in the second computation graph, the eigenvector of the operator obtained by the method of extracting the eigenvector of the operator provided in this application can well reflect the topological relationship between an operator and other operators in the computation graph, that is, the "feature" of the operator in the computation graph.

[0253] The cosine similarity of any two non-zero eigenvectors is a real number between 0 and 1. The closer the value is to 1, the more similar the two eigenvectors are, indicating that the two operators corresponding to the eigenvectors are more similar. The similarity between a zero eigenvector and any eigenvector is defined as 0, meaning they are completely dissimilar.

[0254] Based on the eigenvectors of the operators determined above, the similarity between any two operators can be expressed as:

[0255] Where k and l are both integers between 1 and 5176. Let qk and rl represent the eigenvectors of operators qk and rl, respectively.

[0256] In the embodiments of this application, the row and column positions of the footprint matrix are equivalent to the mapping relationship of the operators. Therefore, when calculating the similarity matrix, the processor calculates the similarity between all operators of the computation graph Q and the computation R, and obtains the following similarity matrix:

[0257]

[0258] The objective function can be calculated based on the similarity matrix. value.

[0259] After that, the processor can... Figure 5b The process of updating the footprint matrix follows a similarity matrix calculation process. Since the row and column positions of operators in the footprint matrix are equivalent to the mapping relationships between operators, the footprint matrix can be updated after finding the optimal mapping relationship for the stage operators, and then the similarity matrix can be calculated. When updating the footprint matrix, the row and column positions of one of the two footprint matrices can be fixed while the row and column positions of the other footprint matrix are adjusted. The adjusted footprint matrix can be the footprint matrix of computation graph Q or the footprint matrix of computation graph R. In one possible implementation, the footprint matrix of the computation graph with fewer true operators can be selected for adjustment to reduce the number of adjustments. For example, in this example, the footprint matrix of computation graph Q can be selected for adjustment.

[0260] Figure 13 A schematic diagram illustrating the adjustment of the footprint matrix according to an embodiment of this application is shown. Figure 13 As shown, the processor can move row 7 to row 1, row 9 to row 2, row 2 to the last row, and so on. At the same time, it can move column 7 to column 1, column 9 to column 2, and column 2 to the last column.

[0261] Based on the adjusted footprint matrix of computational graph Q and the footprint matrix of computational graph R, the similarity matrix can be calculated, and the value of the objective function can be calculated.

[0262] Then, the processor can determine whether the stopping condition has been met. The stopping condition can be the condition satisfied by the similarity matrix (objective function value) obtained from multiple calculations, or it can be the number of iterations. This application does not limit this.

[0263] If the stopping condition is not met, the processor can continue updating the footprint matrix, calculating similarity based on the updated footprint matrix, and determining whether the stopping condition has been met, until the stopping condition is reached. The processor can determine the mapping relationship between operators based on the footprint matrix when the stopping condition is met.

[0264] For homogeneous partitions, one operator can be directly assigned as the mapping operator for the other among the two operators corresponding to the same rows (or columns) in the two footprint matrices. For example, when the stopping condition is met, the footprint matrix of the adjusted computational graph Q is:

[0265] q3 q2 q1 … … q5176 q3 0 … … … … 0 q2 … … … … … … q1 … … … … … … … … … … … … … … … … … … … … q5176 0 … … … … 0

[0266] The mapping operator for operator q3 is r1, the mapping operator for operator q2 is r2, the mapping operator for operator q1 is r3, and so on. In other words, the two operators corresponding to the same row (or column) of two footprint matrices are each other's mapping operators.

[0267] The processor can output a one-to-one mapping relationship between operators in a homogeneous partition based on two footprint matrices.

[0268] For operators in mixed partitions, there are no mapping operators. The processor can output mapping suggestions for operators in mixed partitions based on the similarity matrix obtained from previous iterations. Specifically, the processor sorts the mixed partition operators in the similarity matrix according to the magnitude of the cosine similarity between them and another operator, and determines one or more (e.g., 5) operators at the top of the sort as mapping suggestions for the mixed partition operators. The processor can output the corresponding mapping operator and the cosine similarity value as the mapping suggestion.

[0269] Taking the Identity class operator in TensorFlow as an example, during the process of adjusting the footprint matrix of the computation graph Q and calculating the similarity matrix, the cosine similarity between the Identity class operator and other operators in the mixing region is calculated. The calculated cosine similarities are sorted in descending order, and the top five operators and their corresponding cosine similarity values ​​are taken as the mapping suggestions for the Identity class operator and output.

[0270] In one possible implementation, this application also provides a parameter for measuring the quality of the mapping result: the L1 norm. The similarity between operators in two computational graphs can be measured by calculating the footprint difference matrix D based on the footprint matrix q and the footprint matrix r, and then calculating the L1 norm of the footprint difference matrix.

[0271] The footprint difference matrix can be calculated using the following formula:

[0272] D = qr, where D represents the footprint difference matrix;

[0273] L1 norm = ∑ i ∑ j |d ij The smaller the L1 norm, the closer the two footprint matrices are, and the more topologically similar the two computation graphs are.

[0274] The first footprint matrix is ​​adjusted by changing the order of rows and columns as in the previous example, and the L1 norm is calculated based on the adjusted first and second footprint matrices. By calculating the change in the L1 norm during the adjustment of the footprint matrix, it can be determined whether the obtained mapping relationship reflects the similarity between the two computational graphs.

[0275] Figure 14A schematic diagram illustrating the effect of iterative calculation according to an embodiment of this application is shown. Figure 14 As shown, with the increase of the number of iterations, the values ​​of the objective function and the L1 norm gradually converge. The value of the objective function gradually increases, and after reaching a certain value, the change is no longer significant. Similarly, the value of the L1 norm gradually decreases, and after reaching a certain value, the change is no longer significant.

[0276] Therefore, the stopping condition can be set by setting a threshold for the objective function or the L1 norm, or by adjusting the number of iterations. In the application example above, setting the number of iterations to 5 or 6 is sufficient to obtain a relatively good mapping relationship.

[0277] The following table shows some parameters of the mapping performance in an example of migrating a ResNet50 model from TensorFlow to MindSpore. In this example, the operator mapping method provided in this application can complete the computation within 90 seconds, completing one-to-one mapping of 99% of the backbone operators and providing many-to-many mapping suggestions, which can provide one-to-one mapping for 40% of all operators.

[0278] MS v1.0.0 Total number of operators 2513 1-to-1 mapping number 994(40％) Many-to-many mapping All MS operators have Top-K mapping suggestions. *Number of backbone operators 895 *One-to-one mapping number of backbone operators 888(99％) Mapping operation time ～90s

[0279] The operator mapping apparatus provided in this application constructs footprint matrices of two computation graphs, whereby the footprint matrices represent the topological relationships between operators in the computation graphs. Based on the footprint matrices, a first matrix indicating the topological relationships between the two computation graphs can be iteratively calculated. After iteration stops, one or more preferred mapping relationships can be determined based on the footprint matrices. When constructing the first and second footprint matrices, and adjusting the first footprint matrix, it is ensured that the operators corresponding to the same rows and columns in the first and second footprint matrices have the same type and the same dimension of the tensors output by the operators. In other words, the operators corresponding to the same rows and columns in the first and second footprint matrices are functionally identical. A similarity matrix is ​​used to measure the similarity of the topological relationships between operators. According to the operator mapping apparatus provided in this application, the mapping relationship between two operators can be reflected in terms of both function and topological relationship, thereby ensuring the correctness of the mapping result.

[0280] For operators that do not have homogeneous operators, the operator mapping method provided in this application can output mapping suggestions, supporting one-to-one, one-to-many, and many-to-many mappings. The operator mapping method provided in this application can support operator mapping between computation graphs with similar structures, and can be applied to scenarios where deep learning frameworks perform model transfer.

[0281] Application Scenario and Example Description: Migrating a ResNet50 model from MindSpore v0.5.0 to MindSpore v1.0.0. The ResNet50 model has 2514 operators in its computation graph on the MindSpore v0.5.0 platform and 2513 operators in its computation graph on the MindSpore v1.0.0 platform. In this example, the computation graph of MindSpore v0.5.0 is represented as R, and the computation graph of MindSpore v1.0.0 is represented as Q.

[0282] The process of calculating homogeneous depth is illustrated in the example above and will not be repeated here.

[0283] Homogeneous partitioning is performed. In this example, since the model is being migrated between different versions of the same deep learning framework, all operators are homogeneous, resulting in no mixed partitions. After homogeneous partitioning, the total number of operators in both computation graphs Q and R is 2514.

[0284] The processor can construct two 2514×2514 footprint matrices in the order of operator numbers and initialize all elements of both footprint matrices to 0. The process of determining the footprint matrices can also be found in the example above, and will not be repeated here.

[0285] The process of calculating similarity and outputting the mapping relationship is the same as in the example above, and will not be repeated here.

[0286] Figure 15 A schematic diagram illustrating the effect of iterative calculation according to an embodiment of this application is shown. Figure 15 As shown, with the increase of the number of iterations, the value of the objective function gradually converges, and the value of the objective function gradually increases. After reaching a certain value, the change is no longer significant. The value of the L1 norm also stops decreasing after the third iteration.

[0287] The following table shows some parameters of the mapping effect in an example of migrating a ResNet50 model from MindSpore v0.5.0 to MindSpore v1.0.0. In this example, the operator mapping method provided in this application can complete the calculation within 25 seconds, achieving 100% one-to-one mapping of operators, demonstrating fast mapping speed and high accuracy.

[0288] MS v1.0.0 Total number of operators 2513 1-to-1 mapping number 2513(100％) 1-to-1 mapping accuracy 83％ *Number of backbone operators 895 *One-to-one mapping number of backbone operators 895(100％) *Backbone operator 1-to-1 mapping accuracy 100％ Mapping operation time ～25s

[0289] The operator mapping method provided in this application can quickly find one-to-one, one-to-many, and many-to-many operator mapping relationships between two computation graphs without modifying the model script. This improves the comparison effect of computation graphs, thereby increasing the testing efficiency of operator developers and the development and migration speed of model networks. The mapping statistics table shows that, regardless of whether comparing computation graphs across frameworks or different versions within the same framework, the one-to-one mapping rate and accuracy of the backbone operators that directly play a role in locating model problems are very high, and the computation time is reasonable. With accurate backbone operator mapping relationships, developers can use them as a reference to quickly compare operator numerical files or analyze model scripts to locate problems.

[0290] As can be seen from the above examples, the operator mapping method provided by the embodiments of this application can quickly realize the mapping of operators in two computation graphs, and can obtain one or more preferred mapping relationships through iterative calculation. Therefore, it can support operator mapping between computation graphs with similar structures, and can also support one-to-many or many-to-many mapping, which is applicable to the scenario of model transfer in deep learning frameworks.

[0291] Embodiments of this application provide an operator mapping apparatus for computation graphs, used to perform operator mapping on operators in a first computation graph and a second computation graph. Figure 16 A block diagram of an operator mapping apparatus 90 for a computational graph according to an embodiment of this application is shown, as follows: Figure 16 As shown, the operator mapping device 90 may include:

[0292] Iteration module 91 is used to repeat the following process based on the first footprint matrix and the second footprint matrix until a stopping condition is met: calculate a first matrix based on the first footprint matrix and the second footprint matrix, adjust the order of rows and columns in the first footprint matrix based on the first matrix, and recalculate the first matrix based on the adjusted first footprint matrix and the second footprint matrix; wherein, the first footprint matrix indicates the topological relationship between operators of the first computation graph, the second footprint matrix indicates the topological relationship between operators of the second computation graph, and the first matrix indicates the similarity of the topological relationships between the first computation graph and the second computation graph;

[0293] The corresponding module 92 is used to obtain the corresponding operators of the multiple operators in the first computation graph in the second computation graph based on the first footprint matrix and the second footprint matrix when the stopping condition is reached.

[0294] In one possible implementation, there is a mapping relationship between the operators of the first computation graph and the operators of the second computation graph that belong to the same first partition and whose corresponding rows or columns are in the same position in the first footprint matrix and the second footprint matrix when the stopping condition is reached.

[0295] The operator mapping apparatus provided in this application constructs footprint matrices for two computation graphs, whereby the footprint matrix represents the topological relationship between operators in the computation graphs. Based on the footprint matrix, a first matrix indicating the similarity of the topological relationship between the two computation graphs can be iteratively calculated. After the iteration stops, one or more preferred mapping relationships can be determined based on the footprint matrix. Therefore, the operator mapping method provided in this application can support operator mapping between computation graphs with similar structures and is applicable to scenarios involving model transfer in deep learning frameworks.

[0296] In one possible implementation, the iteration module 91 includes:

[0297] The calculation unit is used to obtain the first matrix based on multiple first feature vectors and multiple second feature vectors, wherein each first feature vector corresponds to an operator in the first calculation graph, and the row vector and column vector corresponding to each first feature vector and the operator corresponding to the first feature vector in the first footprint matrix are related; each second feature vector corresponds to an operator in the second calculation graph, and the row vector and column vector corresponding to each second feature vector and the operator corresponding to the second feature vector in the second footprint matrix are related; the first feature vectors and the second feature vectors are used to calculate the similarity between the operator corresponding to the first feature vector and the operator corresponding to the second feature vector.

[0298] In one possible implementation, the value of an element in the first matrix represents the similarity between the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located, wherein the operator corresponding to the row where the element is located and the operator corresponding to the column where the element is located are respectively operators in the first computation graph and operators in the second computation graph.

[0299] The eigenvectors of operators are constructed using the row and column vectors corresponding to the footprint matrix of the operator in the computation graph, thereby extracting the topological relationship features of the operator in the computation graph. The similarity of the topological relationships of the operators is calculated based on their eigenvectors. A similarity matrix is ​​formed by comparing the similarity of the topological relationships of the operators in the first and second computation graphs. This similarity matrix is ​​used to measure the similarity of the topological relationships between the two computation graphs.

[0300] In one possible implementation, the operators corresponding to the same row positions in the first footprint matrix, the adjusted first footprint matrix, and the second footprint matrix are of the same type, and the tensors output by the operators have the same dimension. That is, the operators corresponding to the same row positions in the first footprint matrix and the second footprint matrix are of the same type, and the tensors output by the operators have the same dimension.

[0301] When constructing the first footprint matrix and the second footprint matrix, and adjusting the first footprint matrix, it is ensured that the operators corresponding to the same rows and columns in the first and second footprint matrices have the same type and the same dimension of the tensor output by the operators. In other words, the operators corresponding to the same rows and columns in the first and second footprint matrices are functionally identical. A similarity matrix is ​​used to measure the similarity of the topological relationships between the operators. According to the operator mapping method provided in this application, the mapping relationship between two operators can be reflected in terms of both function and topological relationship, thereby ensuring the correctness of the mapping result.

[0302] In one possible implementation, the operators in the first and second computation graphs belong to multiple first partitions, with operators of the same type and tensors of the same output dimension within the same first partition. Some operators in the first and second computation graphs belong to second partitions, where the operator types and / or the tensors of the output tensors differ. Operators at the same position in the first and second footprint matrices belong to the same first partition, and operators at the same position in the same column belong to the same first partition. Furthermore, operators in the first computation graph belonging to the same first partition are arranged consecutively in the corresponding rows or columns of the first footprint matrix, and operators in the second computation graph belonging to the same first partition are arranged consecutively in the corresponding rows or columns of the second footprint matrix.

[0303] Because there is a higher probability of mapping relationships between homogeneous operators, and in the embodiments of this application, there is a mapping relationship between operators corresponding to the same row or column in the first footprint matrix and the second footprint matrix. Therefore, according to the above embodiments provided by this application, by homogeneously partitioning the operators in the first and second computation graphs, and adjusting the order of rows and columns in the first footprint matrix by adjusting the order of rows and columns corresponding to operators located in the same homogeneous partition, the iteration process can converge more quickly, thus accelerating the mapping process.

[0304] For operators without homogeneous operators, the corresponding operator similarity is obtained through classification by setting mixed partitions and participating in iterative calculations. When determining the mapping operator, mapping suggestions can be output based on the calculated similarity. This supports operator mapping between computation graphs with similar structures, and also supports one-to-many or many-to-many mappings, making it suitable for scenarios involving model transfer in deep learning frameworks.

[0305] In one possible implementation, the homogeneity depth of operators corresponding to the same position in the first footprint matrix and the second footprint matrix is ​​the same, and the homogeneity depth of operators corresponding to the same position in the second footprint matrix is ​​the same. Furthermore, the rows or columns corresponding to operators of the first computation graph belonging to the same first partition are sorted in the first footprint matrix according to the homogeneity depth of the operators, and the rows or columns corresponding to operators of the second computation graph belonging to the same first partition are sorted in the second footprint matrix according to the homogeneity depth of the operators. The homogeneity depth is defined as the maximum number of operators of the same type and output tensor dimension as the operator in all branches to which the operator belongs in the computation graph.

[0306] Operators within homogeneous partitions are sorted according to their homogeneity depth. Based on the determined homogeneous partitions, the first footprint matrix of the first computational graph and the second footprint matrix of the second computational graph are determined respectively. Since operators with the same homogeneity depth are more likely to have mapping relationships, the process of calculating the similarity matrix according to the above embodiment of this application can converge quickly, determine the mapping relationships between operators more rapidly, and accelerate the mapping process. Furthermore, distinguishing operators with the same attributes by homogeneity depth avoids the unreliability of distinguishing operators with the same attributes by their names.

[0307] In one possible implementation, if the number of operators in the first computation graph in the first partition is different from the number of operators in the second computation graph, the first partition also includes pseudo-operators; wherein, pseudo-operators are operators without operator type and operands; if the number of operators in the first computation graph in the first partition is less than the number of operators in the second computation graph in the first partition, the row or column corresponding to the pseudo-operator in the first footprint matrix is ​​located after the row or column corresponding to the operator in the first computation graph belonging to the first partition; if the number of operators in the second computation graph in the first partition is less than the number of operators in the first computation graph in the first partition, the row or column corresponding to the pseudo-operator in the second footprint matrix is ​​located after the row or column corresponding to the operator in the second computation graph belonging to the first partition.

[0308] In one possible implementation, if the number of operators in the first computation graph in the second partition is different from the number of operators in the second computation graph, the second partition also includes the pseudo-operator; if the number of operators in the first computation graph in the second partition is less than the number of operators in the second computation graph in the second partition, the rows and columns corresponding to the pseudo-operator in the first footprint matrix are randomly sorted with the rows and columns corresponding to the operators in the first computation graph belonging to the second partition, and the random sorting of rows and columns is the same; if the number of operators in the second computation graph in the second partition is less than the number of operators in the second computation graph in the second partition, the rows and columns corresponding to the pseudo-operator in the second footprint matrix are randomly sorted with the rows and columns corresponding to the operators in the second computation graph belonging to the second partition, and the random sorting of rows and columns is the same.

[0309] When the number of operators in the first and second computational graphs differs, pseudo-operators are added to make the total number of operators in the first and second computational graphs equal, thereby constructing footprint matrices of the same size so that subsequent matrix operations can proceed normally. Thus, the operator mapping method provided in this application can be applied to scenarios involving the transfer of models between frameworks or systems employing different computational graph representations.

[0310] In one possible implementation, the iteration module 91 further includes:

[0311] The adjustment unit is used to adjust the order of rows and columns corresponding to operators located in the same first partition in the first footprint matrix according to the first matrix.

[0312] In one possible implementation, there is a mapping relationship between the operators of the first computation graph and the operators of the second computation graph that belong to the same first partition and have the same row or column position in the first footprint matrix and the second footprint matrix when the stopping condition is reached; there is a mapping relationship between the operators of the first computation graph belonging to the second partition and multiple operators of the second computation graph belonging to the second partition, wherein the multiple operators of the second computation graph belonging to the second partition are: the operators of the second computation graph corresponding to the largest multiple values ​​among the values ​​of the elements corresponding to the operators of the first computation graph belonging to the second partition in the first matrix when the stopping condition is reached.

[0313] According to the mapping method of the above embodiments of this application, operator mapping between computation graphs with similar structures can be supported, supporting one-to-one, one-to-many and many-to-many mapping, which can be applied to scenarios where deep learning frameworks perform model transfer.

[0314] In one possible implementation, the stopping condition is that the number of times the first matrix is ​​repeatedly calculated is greater than a first threshold.

[0315] By homogeneously partitioning the operators in the first and second computational graphs, and adjusting the order of rows and columns in the first footprint matrix by adjusting the order of rows and columns corresponding to operators located in the same homogeneous partition, the iterative process can converge more quickly, accelerating the mapping process. This is because homogeneous operators have higher similarity, and there is a greater possibility of mapping relationships between them. Therefore, the process of homogeneous partitioning can accelerate the speed of iterative convergence and the mapping process.

[0316] Embodiments of this application provide an operator mapping apparatus for a computation graph, including: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions.

[0317] Figure 17 A block diagram of an operator mapping apparatus according to an embodiment of this application is shown. Figure 17 As shown, the operator mapping device 1700 can vary considerably due to different configurations or performance, and may include one or more central processing units (CPUs) 1701 (e.g., one or more processors) and a memory 1705 that stores one or more application programs or data.

[0318] The memory 1705 can be volatile or persistent storage. The program stored in the memory 1705 can include one or more modules, each module including a series of instruction operations on the blockchain node. Furthermore, the central processing unit 1701 can be configured to communicate with the memory 1705 and execute the series of instruction operations in the memory 1705 on the operator mapping device 1700.

[0319] The operator mapping device 1700 may also include one or more power supplies 1702, one or more wired or wireless network interfaces 1703, one or more input / output interfaces 1704, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0320] In this embodiment, the process executed by the central processing unit 1701 in the operator mapping device 1700 is the same as described above. Figure 3 , Figure 5a , Figure 5b , Figure 6 or Figure 11 The method flow described in the illustrated embodiments is similar and will not be repeated here.

[0321] Embodiments of this application provide a non-volatile computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the above-described method.

[0322] Embodiments of this application provide a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0323] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), electrically programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital video disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing.

[0324] The computer-readable program instructions or code described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0325] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from computer-readable program instructions. These electronic circuits can execute computer-readable program instructions to implement various aspects of this application.

[0326] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0327] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0328] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0329] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0330] It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented using hardware (such as circuits or ASICs (Application Specific Integrated Circuits)) that performs the corresponding function or action, or using a combination of hardware and software, such as firmware.

[0331] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, disclosure, and appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0332] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method of operator mapping of a computational graph, the method comprising: Operator mapping is used to map operators to a first computation graph and a second computation graph, where the first and second computation graphs are computation graphs of the same neural network model. The operator mapping is used to locate problematic script positions during neural network model transfer between different deep learning frameworks. The operator mapping method includes: Based on the first footprint matrix and the second footprint matrix, repeat the following process until a stopping condition is met: calculate the first matrix based on the first footprint matrix and the second footprint matrix, adjust the order of rows and columns in the first footprint matrix according to the first matrix, and recalculate the first matrix based on the adjusted first footprint matrix and the second footprint matrix; wherein, the first footprint matrix indicates the topological relationship between operators of the first computation graph, the second footprint matrix indicates the topological relationship between operators of the second computation graph, and the first matrix indicates the similarity of the topological relationships between the first computation graph and the second computation graph; the first computation graph and the second computation graph are computation graphs generated by the script of the neural network model under different deep learning frameworks; Based on the first footprint matrix and the second footprint matrix when the stopping condition is met, the operators corresponding to each of the multiple operators in the first computation graph in the second computation graph are obtained. The operators corresponding to each of the multiple operators in the first computation graph in the second computation graph represent the operator mapping relationship between the first computation graph and the second computation graph. The operator mapping relationship is used to compare the numerical files of model scripts of different deep learning frameworks, so as to modify the model script of the transferred deep learning framework when the operands corresponding to the operators in the numerical files of the model scripts of different deep learning frameworks are inconsistent. The numerical files include the operands of the operators in the computation graph.

2. The method of claim 1, wherein, Calculating the first matrix based on the first footprint matrix and the second footprint matrix includes: The first matrix is ​​obtained based on multiple first eigenvectors and multiple second eigenvectors, wherein each first eigenvector corresponds to an operator in the first computation graph, and the row vector and column vector corresponding to each first eigenvector and the operator corresponding to the first eigenvector in the first footprint matrix are related. Each second eigenvector corresponds to an operator in the second computation graph, and the row vector and column vector corresponding to each second eigenvector and the operator corresponding to the second eigenvector in the second footprint matrix are related. The first eigenvectors and the second eigenvectors are used to calculate the similarity of the topological relationship between the operator corresponding to the first eigenvector and the operator corresponding to the second eigenvector.

3. The method of claim 2, wherein, The value of each element in the first matrix represents the similarity of the topological relationship between the operator corresponding to the row and the operator corresponding to the column of the element, wherein the operator corresponding to the row and the operator corresponding to the column of the element are respectively the operators in the first computation graph and the operators in the second computation graph.

4. The method according to any one of claims 1-3, characterized in that, The operators in the first computation graph and the second computation graph belong to multiple first partitions. The operators in the same first partition have the same type and the tensors output by the operators have the same dimension. The operators corresponding to the same row positions in the first footprint matrix and the second footprint matrix are in the same first partition, and the operators corresponding to the same column positions are in the same first partition; Furthermore, the operators of the first computation graph belonging to the same first partition are arranged consecutively in the corresponding rows or columns of the first footprint matrix, and the operators of the second computation graph belonging to the same first partition are arranged consecutively in the corresponding rows or columns of the second footprint matrix.

5. The method of claim 4, wherein, Some operators in the first computation graph and the second computation graph belong to the second partition, and the operator types in the second partition are different and / or the tensors output by the operators have different dimensions.

6. The method of claim 4, wherein, The operators corresponding to the same position in the first footprint matrix and the second footprint matrix have the same homogeneity depth, the operators corresponding to the same position in the second footprint matrix have the same homogeneity depth, and the rows or columns of the operators of the first computation graph belonging to the same first partition are sorted according to the homogeneity depth of the operators in the first footprint matrix, and the rows or columns of the operators of the second computation graph belonging to the same first partition are sorted according to the homogeneity depth of the operators in the second footprint matrix. The homogeneity depth is defined as the maximum number of operators of the same type and with the same output tensor dimension as the operator in all branches of the computation graph to which the operator belongs.

7. The method according to claim 5, characterized in that, If the number of operators in the first computation graph in the first partition is different from the number of operators in the second computation graph, the first partition also includes pseudo-operators; wherein, pseudo-operators are operators that do not have operator type and operands; If the number of operators in the first computation graph within the first partition is less than the number of operators in the second computation graph within the first partition, the row or column corresponding to the pseudo-operator in the first footprint matrix is ​​located after the row or column corresponding to the operator belonging to the first partition in the first computation graph. If the number of operators in the second computation graph in the first partition is less than the number of operators in the first computation graph in the first partition, the row or column corresponding to the pseudo-operator in the second footprint matrix is ​​located after the row or column corresponding to the operator in the first partition in the second computation graph.

8. The method according to claim 5 or 7, characterized in that, If the number of operators in the first computation graph in the second partition is different from the number of operators in the second computation graph, the second partition also includes pseudo-operators; If the number of operators in the first computation graph in the second partition is less than the number of operators in the second computation graph in the second partition, the rows and columns corresponding to the pseudo-operators in the first footprint matrix are randomly sorted with the rows and columns corresponding to the operators in the first computation graph belonging to the second partition, and the random sorting of rows and columns is the same. If the number of operators in the second computation graph in the second partition is less than the number of operators in the second computation graph in the second partition, the rows and columns corresponding to the pseudo-operators in the second footprint matrix are randomly sorted with the rows and columns corresponding to the operators in the second computation graph belonging to the second partition, and the random sorting of rows and columns is the same.

9. The method of claim 5, wherein, Adjusting the order of rows and columns in the first footprint matrix according to the first matrix includes: Adjust the order of rows and columns corresponding to operators located in the same first partition in the first footprint matrix according to the first matrix.

10. The method according to claim 5, characterized in that, There is a mapping relationship between the operators of the first computation graph and the operators of the second computation graph that belong to the same first partition and have the same row or column position in the first footprint matrix and the second footprint matrix when the stopping condition is reached; There is a mapping relationship between the operators of the first computation graph belonging to the second partition and multiple operators of the second computation graph belonging to the second partition. Wherein, the multiple operators of the second computation graph belonging to the second partition are: the operators of the second computation graph corresponding to the largest multiple values ​​among the values ​​of the elements of the first matrix that correspond to the operators of the first computation graph belonging to the second partition when the stopping condition is reached.

11. The method of claim 1, wherein, The stopping condition is that the number of times the first matrix is ​​repeatedly calculated is greater than a first threshold.

12. An apparatus for computing an operator map of a graph, the apparatus comprising: Operator mapping is used for mapping operators to a first computation graph and a second computation graph, where the first and second computation graphs are computation graphs of the same neural network model. The operator mapping is used to locate problematic script positions during neural network model transfer between different deep learning frameworks. The operator mapping device includes: An iterative module is used to repeat the following process based on a first footprint matrix and a second footprint matrix until a stopping condition is met: calculate a first matrix based on the first footprint matrix and the second footprint matrix, adjust the order of rows and columns in the first footprint matrix based on the first matrix, and recalculate the first matrix based on the adjusted first footprint matrix and the second footprint matrix; wherein, the first footprint matrix indicates the topological relationship between operators of the first computation graph, the second footprint matrix indicates the topological relationship between operators of the second computation graph, and the first matrix indicates the similarity of the topological relationships between the first computation graph and the second computation graph; the first computation graph and the second computation graph are computation graphs generated by the script of the neural network model under different deep learning frameworks; The corresponding module is used to obtain the corresponding operators in the second computation graph for each of the multiple operators in the first computation graph based on the first footprint matrix and the second footprint matrix when the stopping condition is reached. The operators corresponding to each of the multiple operators in the first computation graph in the second computation graph represent the operator mapping relationship between the first computation graph and the second computation graph. The operator mapping relationship is used to compare the numerical files of model scripts of different deep learning frameworks, so as to modify the model script of the transferred deep learning framework when the operands corresponding to the operators in the numerical files of the model scripts of different deep learning frameworks are inconsistent. The numerical files include the operands of the operators in the computation graph.

13. The apparatus of claim 12, wherein, The iteration module includes: The calculation unit is used to obtain the first matrix based on multiple first feature vectors and multiple second feature vectors, wherein each first feature vector corresponds to an operator in the first calculation graph, and the row vector and column vector corresponding to each first feature vector and the operator corresponding to the first feature vector in the first footprint matrix are related; each second feature vector corresponds to an operator in the second calculation graph, and the row vector and column vector corresponding to each second feature vector and the operator corresponding to the second feature vector in the second footprint matrix are related; the first feature vectors and the second feature vectors are used to calculate the similarity of the topological relationship between the operator corresponding to the first feature vector and the operator corresponding to the second feature vector.

14. The apparatus of claim 13, wherein, The value of each element in the first matrix represents the similarity of the topological relationship between the operator corresponding to the row and the operator corresponding to the column of the element, wherein the operator corresponding to the row and the operator corresponding to the column of the element are respectively the operators in the first computation graph and the operators in the second computation graph.

15. The apparatus of any of claims 12-14, wherein, The operators in the first computation graph and the second computation graph belong to multiple first partitions. The operators in the same first partition have the same type and the tensors output by the operators have the same dimension. The operators corresponding to the same row positions in the first footprint matrix and the second footprint matrix are in the same first partition, and the operators corresponding to the same column positions are in the same first partition; Furthermore, the operators of the first computation graph belonging to the same first partition are arranged consecutively in the corresponding rows or columns of the first footprint matrix, and the operators of the second computation graph belonging to the same first partition are arranged consecutively in the corresponding rows or columns of the second footprint matrix.

16. The apparatus of claim 15, wherein, Some operators in the first computation graph and the second computation graph belong to the second partition, and the operator types in the second partition are different and / or the tensors output by the operators have different dimensions.

17. The apparatus of claim 15, wherein, The operators corresponding to the same position in the first footprint matrix and the second footprint matrix have the same homogeneity depth, the operators corresponding to the same position in the second footprint matrix have the same homogeneity depth, and the rows or columns of the operators of the first computation graph belonging to the same first partition are sorted according to the homogeneity depth of the operators in the first footprint matrix, and the rows or columns of the operators of the second computation graph belonging to the same first partition are sorted according to the homogeneity depth of the operators in the second footprint matrix. The homogeneity depth is defined as the maximum number of operators of the same type and with the same output tensor dimension as the operator in all branches of the computation graph to which the operator belongs.

18. The apparatus according to claim 16, characterized in that, If the number of operators in the first computation graph in the first partition is different from the number of operators in the second computation graph, the first partition also includes pseudo-operators; wherein, pseudo-operators are operators that do not have operator type and operands; If the number of operators in the first computation graph within the first partition is less than the number of operators in the second computation graph within the first partition, the row or column corresponding to the pseudo-operator in the first footprint matrix is ​​located after the row or column corresponding to the operator belonging to the first partition in the first computation graph. If the number of operators in the second computation graph in the first partition is less than the number of operators in the first computation graph in the first partition, the row or column corresponding to the pseudo-operator in the second footprint matrix is ​​located after the row or column corresponding to the operator in the first partition in the second computation graph.

19. The apparatus of claim 16 or 18, wherein, If the number of operators in the first computation graph in the second partition is different from the number of operators in the second computation graph, the second partition also includes pseudo-operators; If the number of operators in the first computation graph in the second partition is less than the number of operators in the second computation graph in the second partition, the rows and columns corresponding to the pseudo-operators in the first footprint matrix are randomly sorted with the rows and columns corresponding to the operators in the first computation graph belonging to the second partition, and the random sorting of rows and columns is the same. If the number of operators in the second computation graph in the second partition is less than the number of operators in the second computation graph in the second partition, the rows and columns corresponding to the pseudo-operators in the second footprint matrix are randomly sorted with the rows and columns corresponding to the operators in the second computation graph belonging to the second partition, and the random sorting of rows and columns is the same.

20. The apparatus according to claim 16, characterized in that, The iteration module also includes: The adjustment unit is used to adjust the order of rows and columns corresponding to operators located in the same first partition in the first footprint matrix according to the first matrix.

21. The apparatus according to claim 16, characterized in that, There is a mapping relationship between the operators of the first computation graph and the operators of the second computation graph that belong to the same first partition and have the same row or column position in the first footprint matrix and the second footprint matrix when the stopping condition is reached; There is a mapping relationship between the operators of the first computation graph belonging to the second partition and multiple operators of the second computation graph belonging to the second partition. Wherein, the multiple operators of the second computation graph belonging to the second partition are: the operators of the second computation graph corresponding to the largest multiple values ​​among the values ​​of the elements of the first matrix that correspond to the operators of the first computation graph belonging to the second partition when the stopping condition is reached.

22. The apparatus of claim 12, wherein, The stopping condition is that the number of times the first matrix is ​​repeatedly calculated is greater than a first threshold.

23. An apparatus for computing an operator map of a graph, the apparatus comprising: include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method according to any one of claims 1-11 when executing the instructions.

24. A non-transitory computer readable storage medium having stored thereon computer program instructions, wherein, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1-11.

25. A computer program product comprising computer-readable code that, when executed in an electronic device, implements the method of any one of claims 1-11.

Images