Tensor processing method, tensor processing apparatus, electronic device, and storage medium

Abstract

A tensor processing method and apparatus, an electronic device, and a storage medium are provided.SOLUTION: Determining conversion function related information corresponding to each of one or more target input tensors of a first operator based on a computation logic of the first operator in a target computation graph and source split states of at least a part of source input tensors of the first operator, splitting each of the source input tensors of the first operator based on the conversion function related information corresponding to each of the target input tensors to obtain each of the target input tensors, and sending each of the target input tensors to a plurality of computing devices. The plurality of computing devices are configured to perform distributed parallel communication based on each of the target input tensors and the first operator to obtain an output tensor of the first operator.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to the field of computer technology, and in particular to technical fields such as deep learning and artificial intelligence. [Background technology]

[0002] In the field of deep learning, large-scale models can perform better than small-scale models, and distributed parallel training frameworks are a prerequisite for training large-scale models. Typically, the relatively high barrier to using distributed parallel training frameworks has led to the emergence of semi-automatic parallel frameworks, in which a user simply marks the logical partitioning states of some tensors in a computation graph. Based on the user's markings, the deep learning framework can convert the computation graph into a distributed parallel computation graph for parallel training. Computation graphs include two types: dynamic computation graphs and static computation graphs. The execution logic of dynamic computation graphs and static computation graphs differs. In deep learning frameworks, the execution logic and call stacks of dynamic computation graphs and static computation graphs are significantly different. This can result in different types of computation graphs consisting of the same operators, resulting in inconsistent results. This can lead to users repeatedly adjusting or executing the computation graphs, which can waste processing resources on multiple computing devices or reduce the resource utilization of multiple computing devices. Therefore, how to ensure consistency in execution results between different types of computation graphs consisting of the same operators is a problem that needs to be solved. Summary of the Invention [Means for solving the problem]

[0003] The present disclosure provides a tensor processing method, a tensor processing device, an electronic device, a storage medium, and a program.

[0004] According to one aspect of the present disclosure, there is provided a tensor processing method, the method comprising: Determine related information of a transformation function corresponding to each target input tensor among one or more target input tensors of the first operator based on a computation logic of the first operator in the target computation graph and a source split state of at least some source input tensors of the first operator, where when the target computation graphs are different types of computation graphs, related information of a transformation function corresponding to any target input tensor among the one or more target input tensors is the same; Performing division on each of the source input tensors of the first operator according to the associated information of the transformation function corresponding to each of the target input tensors to obtain each of the target input tensors; sending each of the target input tensors to a plurality of computing devices, the plurality of computing devices performing distributed parallel communication based on each of the target input tensors and the first operator to obtain an output tensor of the first operator.

[0005] According to one aspect of the present disclosure, there is provided a tensor processing apparatus, the apparatus comprising: a transformation function determination module for determining, based on a computation logic of a first operator in a target computation graph and a source split state of at least some source input tensors of the first operator, relevant information of a transformation function corresponding to each target input tensor among one or more target input tensors of the first operator, wherein when the target computation graphs are different types of computation graphs, relevant information of a transformation function corresponding to any target input tensor among the one or more target input tensors is the same; a tensor division module for performing division on each of the source input tensors of the first operator according to related information of a transformation function corresponding to each of the target input tensors to obtain each of the target input tensors; a communication module for transmitting each of the target input tensors to a plurality of computing devices, the plurality of computing devices performing distributed parallel communication based on each of the target input tensors and the first operator to obtain an output tensor of the first operator.

[0006] According to another aspect of the present disclosure, there is provided an electronic device, the device comprising: at least one processor; a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, cause the implementation of any one of the methods in the embodiments of the present disclosure.

[0007] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium having stored thereon computer instructions for causing a computer to perform any one of the methods in the embodiments of the present disclosure.

[0008] According to another aspect of the present disclosure, there is provided a program for implementing any one of the methods in the embodiments of the present disclosure when executed by a processor.

[0009] By adopting the above method, regardless of whether the target computation graph is a static computation graph or a dynamic computation graph, it is possible to determine the relevant information of the transformation function of the same target input tensor for the same operator. After dividing the source input tensor based on the transformation function corresponding to the target input tensor, regardless of whether the computation graph is statically executed or dynamically executed, it is possible to obtain the target input tensor in the same divided state. Ultimately, regardless of whether the computation graph is statically executed or dynamically executed, it is ensured that the target input tensor in the same divided state is sent to multiple computation devices to perform the same distributed parallel communication, and the consistency of the final result is guaranteed. In this way, whether in a dynamic computation graph or a static computation graph, the construction of a network model performed for the same operator can obtain consistent execution results, thereby avoiding the need for readjustment or re-execution of the computation graph, and thus avoiding the problems of wasting the processing resources of multiple computation devices or reducing the resource utilization rate of multiple computation devices caused by re-execution of the computation graph.

[0010] The accompanying drawings are for a better understanding of the solutions of the present disclosure, but are not intended to limit the present disclosure. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a flowchart of a tensor processing method according to one embodiment of the present disclosure. [Figure 2] FIG. 1 is a schematic diagram illustrating a transformation scenario of an input tensor of an operator according to an embodiment of the present disclosure. [Figure 3] FIG. 2 is a schematic diagram illustrating a processing scenario of a conversion function according to an embodiment of the present disclosure. [Figure 4] 10 is a flowchart of a tensor processing method according to another embodiment of the present disclosure. [Figure 5] FIG. 1 is a schematic diagram illustrating a configuration of a tensor processing device according to an embodiment of the present disclosure. [Figure 6] FIG. 10 is a schematic diagram illustrating the configuration of a tensor processing device according to another embodiment of the present disclosure. [Figure 7] FIG. 10 is a schematic diagram illustrating the configuration of a tensor processing device according to another embodiment of the present disclosure. [Figure 8] FIG. 1 is a block diagram of an electronic device for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings. Various details of the embodiments of the present disclosure are included herein for ease of understanding and should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, in the following description, descriptions of known functions and structures will be omitted for clarity and conciseness.

[0013] An embodiment of an aspect of the present disclosure provides a tensor processing method, as shown in FIG. 1, including:

[0014] In S101, based on the computation logic of a first operator in a target computation graph and the source splitting state of at least some source input tensors of the first operator, determine related information of a transformation function corresponding to each target input tensor among one or more target input tensors of the first operator, where when the target computation graphs are different types of computation graphs, the related information of the transformation function corresponding to any target input tensor among the one or more target input tensors is the same.

[0015] In S102, according to the related information of the transformation function corresponding to each of the target input tensors, division is performed on each of the source input tensors of the first operator to obtain each of the target input tensors.

[0016] In S103, each of the target input tensors is sent to a plurality of computing devices, where the plurality of computing devices perform distributed parallel communication based on each of the target input tensors and the first operator to obtain an output tensor of the first operator.

[0017] The tensor processing method provided in this embodiment can be applied to an electronic device, which may be a server or a computer, and further, the electronic device can be configured or run a deep learning framework.

[0018] In the embodiments of the present application, the target computation graph is a computation graph for constructing a network model, and the target computation graph can set or include one or more operators for constructing a network model, and can be used to determine or obtain the structure and / or function of the target model; in other words, the target computation graph can construct the structure of the target model, and the target model can be trained and / or finally obtained by executing the target computation graph, and the target model can be applied to various possible fields such as speech processing, image processing, data processing, text processing, etc., and the present disclosure does not limit or list the fields to which the target model can be applied.

[0019] There are two types of computation graphs: static computation graphs and dynamic computation graphs, and the type of the target computation graph may be either a static computation graph or a dynamic computation graph. Here, a dynamic computation graph (or simply called a dynamic graph) refers to a computation graph that can be immediately executed for each operator in the construction of a network model; that is, every time a user sets an operator in a dynamic graph, the operator is immediately executed by a call stack corresponding to the dynamic graph. A static computation graph (or simply called a static graph) can include all operators (multiple operators) in the construction of a network model; that is, all operators in the construction of a network model are recorded in a static graph, and when the static graph is executed, they can be scheduled and executed at the granularity of the entire static graph by a call stack corresponding to the static graph. The call stack corresponding to the dynamic graph is different from the call stack corresponding to the static graph.

[0020] A tensor may be a basic data structure in deep learning, and a tensor may include types of input data, model parameters, and output data, and the input data may include sample data, labels, etc. An input tensor according to an embodiment of the present disclosure may include at least one of sample data, labels, model parameters, etc.

[0021] In this way, regardless of whether the target computation graph is a static computation graph or a dynamic computation graph, it is possible to determine the relevant information of the transformation function of the same target input tensor for the same operator, and after dividing the source input tensor based on the transformation function corresponding to the target input tensor, regardless of whether the computation graph is statically executed or dynamically executed, it is possible to obtain the target input tensor in the same divided state. Ultimately, regardless of whether the computation graph is statically executed or dynamically executed, it is ensured that the target input tensor in the same divided state is sent to multiple computation devices to perform the same distributed parallel communication, and the consistency of the final result can be guaranteed. In this way, whether in a dynamic computation graph or a static computation graph, the construction of a network model performed for the same operator can obtain consistent execution results, thereby avoiding readjustment and re-execution of the computation graph and further avoiding problems such as wasting the processing resources of multiple computation devices or reducing the resource utilization rate of multiple computation devices caused by re-execution of the computation graph.

[0022] In some possible embodiments, before determining, based on the computation logic of a first operator in the target computation graph and the source split states of at least some of the source input tensors of the first operator, relevant information of a transformation function corresponding to each of the one or more target input tensors of the first operator, when the type of the target computation graph is a dynamic computation graph, in response to obtaining the distributed split mark set for the ith source input tensor of the first operator, obtaining a source split state of a tensor, where i is an integer greater than or equal to 1, and the i-th source input tensor is one of the at least some of the source input tensors; and when the type of the target computation graph is a dynamic computation graph, in response to setting the k-th output tensor of a third operator in the target computation graph to the i-th source input tensor of the first operator, setting the corresponding split state of the k-th output tensor of the third operator to the source split state of the i-th source input tensor of the first operator, where k is an integer greater than or equal to 1.

[0023] In this embodiment, the type of the target computation graph is a dynamic computation graph, and the target computation graph can alternatively be called a target dynamic computation graph or a target dynamic graph. In the embodiments of this application, the meanings of the target dynamic computation graph, the target dynamic graph, and the type of the target computation graph being a dynamic computation graph are all the same, and will not be repeated below.

[0024] The i-th source input tensor of the first operator may be any source input tensor of the first operator.

[0025] In one example, the distributed partition mark corresponding to the i-th source input tensor of the first operator may be set by a user in the target dynamic computation graph.

[0026] When a user constructs a network model (non-distributed) using the dynamic graph mode, a semi-automatic parallel API (Application Programming Interface) interface is used to mark some source input tensors in the model as distributed splits in the construction of the network model using the target dynamic graph.

[0027] Specifically, the user can set a current operator, which is the first operator in this embodiment, in the target dynamic graph, and can also set a distribution partition mark for each source input vector in at least some of the source input vectors of the current operator in the target dynamic graph, where the i-th source input tensor refers to any source input tensor of the current operator (i.e., the first operator) for which the user has set a distribution partition mark. Note that the user can set corresponding distribution partition marks for some or all of the source input tensors of the current operator (i.e., the first operator).

[0028] Take the case where the first operator is the MATMUL operator (an operator that performs a matrix multiplication operation) as an example, and assume that the MATMUL operator has two source input tensors, namely, source input tensor A and source input tensor B. Here, the user can only set the distribution partition mark of source input tensor A, and not set the distribution partition mark of source input tensor B.

[0029] Obtaining the source split state of the i-th source input tensor of the first operator based on the distributed split mark corresponding to the i-th source input tensor of the first operator can make the content of the distributed split mark corresponding to the i-th source input tensor of the first operator the source split state of the i-th source input tensor of the first operator.

[0030] The distributed partition mark corresponding to the i-th source input tensor can be configured according to actual needs. For example, the content of the distributed partition mark corresponding to the i-th source input tensor can include topology information of the distributed cluster and an indication of whether to partition the i-th source input tensor in multiple dimensions.

[0031] Here, a distributed cluster may include one or more computing devices (a computing device may simply be referred to as a device), and the topology information of the distributed cluster may be used to represent multi-dimensional topology information configured among one or more computing devices. For example, suppose a distributed cluster includes eight computing devices, each of which is Device 0 to Device 7. The group of eight computing devices has a two-dimensional topology, i.e., every four computing devices constitute one dimension (or one path). The topology information of the distributed cluster may include [0,1,2,3][4,5,6,7], i.e., Device 0 to Device 3 constitute one dimension, and Device 4 to Device 6 constitute one dimension.

[0032] The indication of whether to divide or not can be expressed by a corresponding indicator value, for example, a first indicator value can be used to indicate division, and a second indicator value can be used to indicate not to divide. The first indicator value and the second indicator value are different, and the specific values ​​of the first indicator value and the second indicator value can be configured according to the actual situation, for example, the first indicator value can be 0, and the second indicator value can be -1. It should be understood that this is only an illustrative description, and as long as the first indicator value and the second indicator value are different, they fall within the protection scope of this embodiment, and are not limited or enumerated here.

[0033] Furthermore, the dimensions of the i-th source input tensor can be set according to actual circumstances. For example, the i-th source input tensor can include two dimensions, where the first dimension represents the row and the second dimension represents the column. For example, if the indication of whether the i-th source input tensor is divided in two dimensions is [-1, 0], the i-th source input tensor is not divided in the first dimension but is divided in the second dimension, which indicates that the i-th source input tensor is not divided in the row but is divided in the column.

[0034] Alternatively, the instruction to split or not may be used to indicate whether to perform a numerical split. That is, such a split instruction is not used to indicate a dimensional (or shape) split, but rather to indicate a numerical split of elements. Assuming that the i-th source input tensor contains four elements [1, 2, 3, 4] in two dimensions and the instruction to split or not indicates that a numerical split is to be performed, the i-th source input tensor may be split into two split results of the same shape or dimension but different numerical values: [0, 1, 2, 3] and [1, 1, 1, 1].

[0035] In addition, when the user sets a distributed division mark corresponding to each of the multiple source input tensors of the current operator (i.e., the first operator) in the target dynamic computation graph, the processing or related explanation for each source input tensor is the same as that for the i-th source input tensor, so it will not be described in detail.

[0036] In one example, the user does not set a distributed partition mark corresponding to the i-th source input tensor of the first operator, and in the target dynamic graph, the user can set the first operator (i.e., the current operator) as a downstream operator of the executed third operator (the third operator is also referred to as one upstream operator of the current operator), and directly set the partition state of the k-th output tensor of the third operator as the source partition state of the i-th source input tensor of the first operator. The source partition state of the i-th source input tensor may include the same content as in the above example, and will not be described again. Note that the third operator can have one or more output tensors, and the k-th output tensor is any one of all output tensors of the third operator. Furthermore, the first operator can have not only one upstream operator, the third operator, but also one or more other upstream operators. If an output tensor of any other upstream operator is also a source input tensor of the first operator, the partition state of that output tensor can similarly be set as the source partition state of one source input tensor of the first operator, and this will not be listed or described again here.

[0037] In actual processing, the first operator may include one source input tensor or multiple source input tensors, and the method for determining the source splitting state of any of the source input tensors of the first operator is the same as any of the methods for determining the source splitting state of the i-th source input tensor described above, and will not be described again here. For example, if the first operator is a MATMUL operator, and the source input tensors include source input tensor A and source input tensor B, the user can set the distributed splitting mark of source input tensor A, and the first operator also has upstream operators, each of which has multiple output tensors, one of which, output tensor C, is the source input tensor B of the first operator. The splitting state of output tensor C is the source splitting state of source input tensor B.

[0038] In this way, when the target computation graph is a dynamic computation graph, the source splitting state of at least some source input tensors of the first operator can be determined based on the distributed splitting marks set for at least some source input tensors of the first operator, or the source splitting state of a source input tensor of the first operator can be determined based on the splitting state of a certain output tensor of an upstream operator of the first operator, thereby obtaining an accurate initial splitting state corresponding to the input tensors of the operators in the dynamic graph, and then providing accurate information for accurately obtaining the target splitting state of the input tensors.

[0039] In some possible embodiments, the type of the target computation graph is a static computation graph, and the method further includes, in response to obtaining a static transformation instruction under an original dynamic computation graph, generating the target computation graph based on the original dynamic computation graph, where the target computation graph includes the same plurality of operators as the original dynamic computation graph, distributed partition marks of one or more source input tensors of the plurality of operators, and the plurality of operators includes the first operator.

[0040] In this embodiment, the method for setting each operator in the original dynamic computation graph is the same as the current operator in the above embodiment, or the target dynamic graph in the above embodiment may be the original dynamic computation graph in this embodiment.

[0041] Furthermore, as described in the above embodiment, when setting a current operator in the original dynamic computation graph, the user may set distributed partition marks for one or more source input vectors of the current operator. In this embodiment, all operators set by the user in the original dynamic computation graph and the distributed partition marks of at least some of the source input vectors in all operators are converted into each operator or each input tensor in the target computation graph (referred to as a target static computation graph or a target static graph) and recorded.

[0042] The plurality of operators includes a first operator, and the distributed partition marks of one or more source input tensors of the plurality of operators may include the distributed partition marks of at least some of the source input tensors of the first operator, or may not include the distributed partition marks of the source input tensors of the first operator.

[0043] It should be noted that in the above embodiment, the user may not set the distributed split mark of the source input tensor for the current operator, and the split state of an output tensor of an upstream operator of the current operator may directly become the source split state of a source input tensor of the current operator, but the source split state of the source input tensor set by a non-user will not be converted into the target static computation graph.

[0044] In this way, in either a static computation graph or a dynamic computation graph, a distributed partition mark of multiple operators, one or multiple source input tensors of multiple operators can be obtained, and in either a static computation graph or a dynamic computation graph, if a static computation graph is converted from a corresponding original dynamic computation graph, it can be guaranteed that both the static computation graph and the corresponding original dynamic computation graph will use the same distributed partition mark for subsequent processing.

[0045] before determining, based on the computation logic of a first operator in the target computation graph and the source splitting states of at least some of the source input tensors of the first operator, related information of a transformation function corresponding to each of the one or more target input tensors of the first operator, the method includes at least one of: if the i-th source input tensor of the first operator has a corresponding distributed splitting mark, obtaining a source splitting state of the i-th source input tensor of the first operator based on the corresponding distributed splitting mark of the i-th source input tensor of the first operator, where i is an integer greater than or equal to 1, and the i-th source input tensor is one of the at least some of the source input tensors; and if the i-th source input tensor of the first operator in the target computation graph is a k-th output tensor of a third operator, setting the corresponding splitting state of the k-th output tensor of the third operator as the source splitting state of the i-th source input tensor of the first operator, where k is an integer greater than or equal to 1.

[0046] In this embodiment, the type of the target computation graph is a static computation graph, that is, the target computation graph can be a target static computation graph (or a target static computation graph). In the embodiment of the present application, the meanings of the target static computation graph, the target static graph, and the type of the target computation graph is a static computation graph are all the same, and will not be repeated below.

[0047] In one example, the corresponding distributed partition mark of the i-th source input tensor of the first operator may be the one recorded in the target static computation graph.

[0048] The number of all source input tensors of the first operator may be 1 or more. In this example, the i-th source input tensor refers to one source input tensor for which a distributed division mark is recorded, that is, among all source input tensors of the first operator, corresponding distributed division marks may be recorded for only some of the source input tensors.

[0049] The distributed division mark corresponding to the i-th source input tensor may include the same content as in the above embodiment, and therefore will not be described again. The process of obtaining the source division state of the i-th source input tensor of the first operator based on the distributed division mark corresponding to the i-th source input tensor of the first operator is also the same as in the above embodiment, and therefore will not be described again.

[0050] In addition, when the corresponding distributed division marks of the multiple source input tensors of the first operator are recorded, the processing or related explanation of each of the source input tensors is the same as that of the i-th source input tensor above, so it will not be repeated.

[0051] In one example, a first operator in a target static computation graph may have one or more upstream operators, and a third operator may be any one of the upstream operators of the first operator, and therefore the first operator may be one of one or more downstream operators of the third operator. If the kth output tensor of the third operator has a corresponding split state, the split state of the kth input tensor of the third operator can be directly used as the source split state of the i-th source input tensor of the first operator. The content that the source split state of the i-th source input tensor may contain is similar to that of the above embodiment, and will not be described again.

[0052] In some possible examples, the first operator may have source input tensors with distributed partition marks set and determined by the user, and / or the source partition marks of some source input tensors of the first operator may be the partition marks of an output tensor derived from an upstream operator.

[0053] In this way, when the target computation graph is a static computation graph, the source splitting state of at least some of the source input tensors of the first operator can be determined based on the distributed splitting marks recorded in at least some of the source input tensors of the first operator, or the source splitting state of a source input tensor of the first operator can be determined based on the splitting state of the output tensor of the upstream operator of the first operator. In this way, the accurate initial splitting state corresponding to the input tensor of the operator in the static computation graph can be obtained, and accurate information can be provided for accurately obtaining the target splitting state of the subsequent input tensor.

[0054] In some possible embodiments, determining related information of a transformation function corresponding to each of the target input tensors of one or more target input tensors of the first operator based on computation logic of the first operator in the target computation graph and source split states of at least some source input tensors of the first operator includes: deriving a target split state of each target input tensor of the one or more target input tensors of the first operator based on computation logic of the first operator in the target computation graph and source split states of at least some source input tensors of the first operator; and determining related information of a transformation function corresponding to each of the target input tensors of the first operator based on the target split states of each of the target input tensors of the first operator.

[0055] In this embodiment, the type of the target computation graph may be a dynamic computation graph or a static computation graph, that is, regardless of whether the target computation graph is a target dynamic computation graph or a target static computation graph, the processing provided by this embodiment can be performed in either case. The first operator can refer to the current operator set by the user in the target dynamic computation graph, or the operator that is currently deriving the split state among all operators in the target static computation graph.

[0056] In this way, for the input tensor of an operator in the target computation graph, the target split state of the target input tensor of the operator can be determined based on the computation logic of the operator and the source split state of the input tensor, and one or more transformation functions for transforming the target input tensor can be obtained, and the transformation function corresponding to the target input tensor is the same regardless of the type of the target computation graph being a static computation graph or a dynamic computation graph. In this way, whether the target computation graph is static or dynamic, the split state of the same target input tensor can be determined for the same operator, and it is guaranteed that the same transformation function will be finally determined, and it is guaranteed that the same result can be obtained whether the computation graph is executed statically or dynamically.

[0057] In some possible embodiments, deriving a target partitioning state for each target input tensor among the one or more target input tensors of the first operator based on a computation logic of the first operator in the target computation graph and a source partitioning state of at least some source input tensors of the first operator includes: deriving a target partitioning state for each target input tensor among the one or more target input tensors of the first operator based on a computation logic of the first operator in the target computation graph, a source partitioning state of at least some source input tensors of the first operator, and a preset partitioning constraint of the first operator, wherein the preset partitioning constraint of the first operator is the same when the target computation graphs are different types of computation graphs, and the preset partitioning constraint of the first operator includes at least one of: the target partitioning state of the one or more target input tensors satisfies the computation requirements of the first operator; and implementing a distributed parallel communication efficiency requirement based on the target partitioning state of the one or more target input tensors of the first operator.

[0058] Deriving a target split state for each target input tensor among one or more target input tensors of the first operator based on computation logic of the first operator in the target computation graph, source split states of at least some of the source input tensors of the first operator, and preset split constraints of the first operator may be deriving a target split state for each target input tensor among all target input tensors of the first operator based on computation logic of the first operator in the target computation graph, source split states of each source input tensor of all source input tensors of the first operator, and preset split constraints of the first operator.

[0059] The first operator may have one or more source input tensors for which a distributed partition mark is set or recorded, and / or may have source partition states for one or more source input tensors determined by the partition states of one or more output tensors of one or more upstream operators. In addition to the above two cases, the first operator may have source input tensors without remaining partition states, i.e., source input tensors for which a distributed partition mark is not set or recorded and which do not inherit the partition states of the output tensors of the upstream operator. When performing the process of deriving the target partition states of one or more target input tensors of the first operator provided in this embodiment, the default partition state can be the source partition state of the source input tensor without the partition state for that portion. Here, the default partition state can be set according to actual circumstances, for example, all dimensions are not partitioned by default, all dimensions are partitioned by default, or the first specified dimension is partitioned and the second specified dimension is not partitioned by default. This does not limit or list all possible examples of default partition states.

[0060] The preset partitioning constraints of the first operator specifically include partition correctness requirements of the first operator and / or computation efficiency requirements of the first operator.

[0061] Here, the splitting correctness requirement of the first operator is that the target split state of the one or more target input tensors satisfies the efficiency requirement of the first operator, i.e., after the target input tensors derived by the solution provided by the present application are input to the first operator, the first operator can accurately perform calculations, i.e., the splitting correctness requirement of the first operator can be guaranteed.

[0062] The computational efficiency requirement of the first operator is to perform a distributed computational efficiency requirement based on the target partition state of the one or more target input tensors of the first operator.

[0063] Specifically, executing a distributed computing efficiency requirement based on the target partitioning state of the one or more target input tensors of the first operator includes at least one of a requirement that there is no redundancy when executing distributed parallel communication based on the target partitioning state of the one or more target input tensors of the first operator, a requirement that the amount of communication when executing distributed parallel communication based on the target partitioning state of the one or more target input tensors of the first operator is minimized, and a requirement that memory space and / or processing resources occupied when executing distributed parallel communication based on the target partitioning state of the one or more target input tensors of the first operator are minimized. Here, the memory space may include at least one of a video memory, a memory, etc., and the processing resources may include at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc.

[0064] 2, for example, the first operator 200 is a MATMUL operator. When the MATMUL operator has two input tensors, it requires that the split state of the second dimension of the first input tensor be the same as the split state of the first dimension of the second input tensor. Suppose the source split state of the source input tensor A 211 of the MATMUL operator in the two dimensions is [-1,-1], and the source split state of the source input tensor B 221 in the two dimensions is [0,-1]. There are two possible processing methods that satisfy the splitting correctness requirement of the first operator: In processing method 1, the target division state in two dimensions of the target input tensor A' 212 of the MATMUL operator is [-1, 0], and the target division state in two dimensions of the target input tensor B' 222 is [0, -1]. In processing method 2, the target division state in two dimensions of the target input tensor A' of the MATMUL operator is [-1, -1], and the target division state in two dimensions of the target input tensor B' is [-1, -1].

[0065] However, of the above two processing methods, processing method 2 does not perform division on either of the two target input tensors, and redundant calculations exist when two or two sets of computing devices perform distributed parallel communication processing of the first operator based on the two same target input tensors, so processing method 2 meets the division correctness requirement of the first operator but does not meet the computational efficiency requirement of the first operator. Processing method 1 performs corresponding division on the two target input tensors, and redundant calculations are avoided when two or two sets of computing devices perform distributed computing processing of the first operator based on the two divided target input tensors, so processing method 1 meets the division correctness requirement of the first operator and the computational efficiency requirement of the first operator.

[0066] A method for deriving a partitioned state of an output tensor of a first operator can derive the partitioned state of the output tensor of the first operator based on the target partitioned states of one or more target input tensors of the first operator and the computation logic of the first operator. In any type of computation graph, the target partitioned state of the target input tensor is obtained using the same preset partitioning constraint, so the computation logic of the first operator is the same in different types of computation graphs, and the partitioned state of the output tensor of the same first operator is derived in different types of computation graphs.

[0067] The number of output tensors of the first operator may be one or more, and the split state of the output tensor of the first operator may refer to the split state of each output tensor of the first operator.

[0068] Still referring to FIG. 2 , for example, the target division state of the first operator 200 (i.e., the MATMUL operator) in two dimensions of the target input tensor A′ 212 is [−1, 0], and the target division state of the two dimensions of the target input tensor B′ 222 is [0, −1]. The MATMUL operator includes one output tensor C 231, which also includes two dimensions, and the division states of the two dimensions are {[−1, −1], Partial}, where Partial is used to indicate that the output tensor that the MATMUL operator can obtain is a partial value of each dimension in the two dimensions based on the target division state of the two dimensions of the target input tensor A being [−1, 0] and the target division state of the two dimensions of the target input tensor B being [0, −1].

[0069] For example, it is assumed that all tensors and operators in the target computation graph (regardless of static or dynamic type) require a determined partition state, and each device (process) needs to determine the communication and partition operations required during the execution of the current operator (i.e., the first operator) based on the source partition state of the current operator (i.e., the first operator) and the source input tensor's source partition state. By deriving the partition using the above process, the target partition state of the current operator (i.e., the first operator) (e.g., the partition state can be expressed as Tensor DistAttr (disseminate attribute)) can be derived based on the source input tensor's source partition state and the current operator's (i.e., the first operator) own operation logic. The partition state of the current operator (i.e., the first operator) determines the communication and partition operations required during the execution of the operator, and the output tensor is involved in deriving the partition of the next operator as an input tensor of the next operator in the target computation graph.

[0070] The above-mentioned partitioning state can be used to describe the cluster and partitioning in which a tensor (e.g., the aforementioned source input tensor) is located. For example, the source partitioning state of any one source input tensor can be expressed as {mesh = [[0,1,2,3], [4,5,6,7]], dims_mapping:[-1,0]}, where "mesh" represents the distributed cluster, [[0,1,2,3], [4,5,6,7]] represents Device 0 to Device 3 in the distributed cluster, and Device 4 to Device 3 represent one dimension, and "dims_mapping" represents the shape partitioning of the tensor, which can perform partitioning on dimensions. The specific meanings of "-1" and "0" above are the same as those in the above examples, and will not be repeated.

[0071] The target division state of the first operator (OperatorDistAttr) can be used to describe the distribution cluster in which the first operator is located, the target division state of the required target input tensor, and the division state of the output tensor. For example, the target division state of the first operator can be expressed as {mesh = [[0,1,2,3], [4,5,6,7]], X's dims_mapping:[-1,0], Y's dims_mapping:[0,-1], Out's dims_mapping:[-1,-1]}, where "mesh" has the same meaning as above, "X's" represents one target input tensor, "Y's" represents another target input tensor, "Out's" represents the output tensor, and the meaning of "dims_mapping" will not be repeated.

[0072] In this way, by setting the partitioning constraints corresponding to the operators in advance, the target partitioning state of the input tensor of the operators finally derived can satisfy the correctness requirements and / or efficiency requirements of the operators. Furthermore, since the partitioning constraints set in advance for the operators are the same in different types of computation graphs, the same target partitioning state can be determined for the same operator using the same rules in any type of computation graph, and the consistency of the execution results can be guaranteed in different computation graphs.

[0073] In some possible embodiments, determining related information of a transformation function corresponding to each target input tensor among one or more target input tensors of the first operator based on the computation logic of the first operator in the target computation graph and the source splitting state of at least some source input tensors of the first operator includes determining related information of a transformation function corresponding to the nth target input tensor of the first operator based on the splitting transformation rule of the first operator, the target splitting state of the nth target input tensor of the first operator, the source splitting state of the nth source input tensor, and the nth source input tensor, where n is an integer greater than or equal to 1, the nth target input tensor is one or more target input tensors, and the nth source input tensor corresponds to the nth target input tensor.

[0074] The relevant information of the transformation function corresponding to the nth target input tensor is the same when the target computation graph is a different type of computation graph, and the transformation function corresponding to the nth target input tensor is used to transform the nth source input tensor into the nth target input tensor.

[0075] The splitting transformation rules may include correspondences between each of the one or more transformation functions and the target split state, the source split state, and the source input tensor, and the splitting transformation rules may be set in advance. In any type of computation graph, the above splitting transformation rules are fixed and the same.

[0076] The number of transformation functions corresponding to the nth target input tensor may be one or more, and the related information of the transformation function may include at least one of the type of the transformation function, the name of the transformation function, and the input parameters of the transformation function, where the input parameters of the transformation function may include an input address, an input size, a communication group (which may be represented as a Communicator), and the communication group may be an abstraction of multiple computing devices, for example, the communication group may be used to indicate or determine one or more computing devices that perform the calculation or processing of the transformation function in all computing devices. In a possible example, the related information of the transformation function may also be called the function signature of the transformation function.

[0077] Therefore, the relevant information of the transformation function corresponding to the nth target input tensor may include at least one of the type of each transformation function among the one or more transformation functions corresponding to the nth target input tensor, the input parameters of each transformation function corresponding to the nth target input tensor, etc.

[0078] Referring to FIG. 3, the transformation function corresponding to the nth target input tensor is represented as Reshard 300, and Reshard functions to transform the target split state 301 (which can be represented as, for example, dst(destination, target)-Disattr-n) of the input nth target input tensor, the source split state 302 (which can be represented as, for example, src(source, source)-Disattr-n) of the nth source input tensor, and the nth source input tensor 303 (which can be represented as, for example, src-tensor-n) into the output nth target input tensor 304 (which can be represented as, for example, dst-tensor-n).

[0079] The nth target input tensor is any one of all target input tensors of the first operator. The process performed for each target input tensor is the same, so it will not be described again. That is, by performing the process provided by this embodiment, the target split states of each target input tensor derived as described above can be mapped and converted into actual executable communication operators. The target split states obtained by the above process (e.g., including distribution marks set by the user) are descriptions of data structures in an abstract programming language. Although the combinations of such abstract descriptions are diverse, the communication operators corresponding to the fixed descriptions for each set are fixed. By performing the method provided by this embodiment, the fixed split states can be combined with transformation functions in either the target static graph or the target dynamic graph, and the correct transformation functions can be inserted into the target computation graph.

[0080] In some possible embodiments, performing division on each of the source input tensors of the first operator based on associated information of a transformation function corresponding to each of the target input tensors to obtain each of the target input tensors includes: calling a transformation function corresponding to each of the target input tensors from a function library through a call stack corresponding to the target computation graph based on associated information of a transformation function corresponding to each of the target input tensors; and performing division on each of the source input tensors of the first operator based on the transformation function corresponding to each of the target input tensors to obtain each of the target input tensors.

[0081] The function library stores or pre-installs multiple candidate functions and associated information for each candidate function. Different candidate functions among the multiple candidate functions can be used to perform different processes. The underlying layer of each candidate function is a collective communication primitive, and each candidate function is a static function. The associated information for each candidate function may include at least one of the same content as in the above embodiment, such as type, input parameters, etc., and will not be described in detail. The associated information for each candidate function allows each candidate function to present the same interface to the outside (e.g., target dynamic graph or target static graph). In this way, a function library (which can be called a communication library or communication operator library) that can be called under both a dynamic graph and a static graph can be realized, and the candidate functions in the function library can be correctly called regardless of whether the target computation graph is a static graph or a dynamic graph.

[0082] For example, candidate functions include at least one of: allreduce() (i.e., allreduce function) for performing an aggregation operation across multiple processes or multiple computing devices and / or broadcasting the aggregation result to multiple processes or multiple computing devices; allgather() (i.e., allgather function) for collecting data from all processes or all computing devices and / or sending the collected data to all processes or all computing devices; send() (i.e., send function) for sending data; recv() (i.e., receive function) for receiving data; concat() (concatenate function) for concatenating or joining two or more arrays, strings, or other types of objects; split() (split function) for splitting a string by a specified delimiter and / or returning a list containing the split substrings; and reduce scatter() (i.e., reduce(aggregate) scatter function) for performing an aggregation operation on a set of input tensors or arrays and distributing the aggregated results among each output tensor or array.

[0083] It should be understood that the above is merely a schematic example of a number of candidate functions stored or pre-installed in a function library, and that in actual processing, some of the candidate function types exemplified above may be included, but are not limited to these, and that this embodiment does not limit or enumerate them. Furthermore, this embodiment does not limit the underlying implementation form or underlying implementation logic or code of each of the candidate functions.

[0084] Calling the transformation functions corresponding to each of the target input tensors from a function library by a call stack corresponding to the target computation graph based on the associated information of the transformation functions corresponding to each of the target input tensors may refer to calling each of the transformation functions that match the associated information of the transformation functions from a plurality of candidate functions stored in a function library by a call stack corresponding to the target computation graph based on the associated information of the transformation function corresponding to the nth target input tensor. The transformation functions corresponding to each of the target input tensors of the first operator are all called using the same method as for the nth target input tensor, and therefore will not be described in detail.

[0085] Through the above process, regardless of whether the target computation graph is a dynamic graph or a static graph, the associated information of the conversion function corresponding to the same target input tensor can be used to appropriately call the corresponding conversion function from the function library, and regardless of whether the target computation graph is a dynamic graph or a static graph, the uniformity of the subsequent execution of distributed parallel communication can be guaranteed.

[0086] In the following, one of two types of target computation graphs, static and dynamic, will be taken as an example and explained as an example.

[0087] In one embodiment, performing division on each of the source input tensors of the first operator based on a transformation function corresponding to each of the target input tensors to obtain each of the target input tensors includes, when the type of the target computation graph is a dynamic computation graph, obtaining a first transformation function corresponding to each of the target input tensors based on a transformation function corresponding to each of the target input tensors and a dynamic encapsulation parameter, and performing division on each of the source input tensors of the first operator based on the first transformation function corresponding to each of the target input tensors to obtain each of the target input tensors.

[0088] Here, the dynamic encapsulation parameters can be set according to actual situations, and the dynamic encapsulation parameters may be set so that the call stack corresponding to the target dynamic computation graph correctly executes or calls the corresponding function. In some possible examples, the dynamic encapsulation parameters may include device management parameters (e.g., may be represented as Device Manager), and this does not limit or list all the contents that can be included in the dynamic encapsulation parameters.

[0089] Obtaining a first transformation function corresponding to each of the target input tensors based on the transformation function corresponding to each of the target input tensors and a dynamic encapsulation parameter may involve encapsulating the transformation function corresponding to the nth target input tensor with the dynamic encapsulation parameter to obtain the first transformation function corresponding to the nth target input tensor.

[0090] Performing division on each of the source input tensors of the first operator based on the first transformation function to obtain each of the target input tensors may involve inputting the nth source input tensor into a first transformation function corresponding to the nth target input tensor, and performing division on the nth target input tensor using one or more transformation functions included in the first transformation function corresponding to the nth target input tensor to obtain the nth target input tensor. The nth target input tensor obtained through the above process is an input tensor that satisfies the corresponding target division state.

[0091] In one embodiment, performing division on each of the source input tensors of the first operator based on a transformation function corresponding to each of the target input tensors to obtain each of the target input tensors includes, when the type of the target computation graph is a static computation graph, obtaining a second transformation function corresponding to each of the target input tensors based on the transformation function corresponding to each of the target input tensors and a static encapsulation parameter, and performing division on each of the source input tensors of the first operator based on the second transformation function corresponding to each of the target input tensors to obtain each of the target input tensors.

[0092] Here, the static encapsulation parameters may be set according to actual circumstances, and the static encapsulation parameters may be set so that a call stack corresponding to a static computation graph can correctly execute or call a corresponding function. In some possible examples, the static encapsulation parameters may include a computation device context (e.g., which may be represented as Device Context). This does not limit or list all the contents that may be included in the static encapsulation parameters.

[0093] Obtaining a second transformation function corresponding to each of the target input tensors based on the transformation function corresponding to each of the target input tensors and a static encapsulation parameter may involve encapsulating the transformation function corresponding to the nth target input tensor with the static encapsulation parameter to obtain the second transformation function corresponding to the nth target input tensor.

[0094] Performing division on each of the source input tensors of the first operator based on a second transformation function corresponding to each of the target input tensors to obtain each of the target input tensors may include inputting an n-th source input tensor to a second transformation function corresponding to an n-th target input tensor, and performing division on the n-th target input tensor using one or more transformation functions included in the second transformation function corresponding to the n-th target input tensor to obtain the n-th target input tensor.

[0095] In the above two embodiments, the nth target input tensor may include multiple split input tensors in either the target static graph or the target dynamic graph. The process of sending each of the target input tensors to the multiple computing devices and subsequent processes may be the following process: after obtaining multiple split input tensors included in the nth target input tensor of the first operator, the electronic device sends each of the split input tensors included in the nth target input tensor to each of the multiple computing devices performing distributed parallel communication; when the electronic device receives at least some of the output tensors of the first operator sent from each of the computing devices, it merges (directly merges or calculates and merges) at least some of the output tensors corresponding to the first operator sent from each of the computing devices to obtain an output tensor corresponding to the first operator. Therefore, after receiving each of the multiple computing devices corresponding to the split input tensors, each of the computing devices inputs the received split input tensors to the first operator to calculate and obtain at least some of the output tensors corresponding to the respective computing devices; and then each computing device can send at least some of the output tensors corresponding to the first operator it obtained to the electronic device.

[0096] Alternatively, the process of sending each of the target input tensors to the plurality of computing devices and subsequent processes may be the following process: an electronic device sends the plurality of divided input tensors included in the n-th target input tensor to each of a plurality of computing devices performing distributed parallel communication; and each computing device receives its corresponding divided input tensor, inputs the received divided input tensor into a first operator to calculate and obtain at least a portion of the output tensor corresponding to each computing device; and each computing device can use at least a portion of the output tensor obtained by itself corresponding to the first operator to perform related calculations of downstream operators. Whether at least a portion of the output tensor of the first operator is merged is related to at least one of the related requirements of the downstream operator, the calculation logic of the downstream operator, the division state of the input tensor of the downstream operator, etc., which are not limited to or listed in this embodiment.

[0097] Furthermore, if the multiple computing devices include an electronic device, the electronic device may perform some of the calculations by itself, and the remaining multiple computing devices may perform other parts of the calculations, all of which fall within the scope of protection of this embodiment and are not limited or enumerated here.

[0098] Furthermore, regardless of whether the target computation graph is a target static computation graph or a target dynamic computation graph, both can input each of the target input tensors to the first operator to obtain an output tensor of the first operator, where the method for generating each target input tensor is the same as the method for generating the nth target input tensor described above, and will not be described again. Whether the target computation graph is a target static computation graph or a target dynamic computation graph, both perform the same processing using the same conversion function on the same target input tensor, and therefore, regardless of whether the target computation graph is a target static computation graph or a target dynamic computation graph, both can obtain the same output tensor of the first operator.

[0099] By adopting the above solution, when the target computation graph is of the type of static computation graph or dynamic computation graph, different static or dynamic parameters are added for encapsulation, but the conversion function of the base layer is still essentially the same. In this way, when the target computation graph is of the type of static computation graph or dynamic computation graph, different encapsulation conversions are performed so that the same base layer function can match the call stack of the static computation graph and the call stack of the dynamic computation graph, respectively. Thus, when the target computation graph is of the type of static computation graph or dynamic computation graph, the same conversion function can be used to obtain the same tensor division result, and finally, regardless of whether the target computation graph is a target static computation graph or a target dynamic computation graph, a unified result can be obtained.

[0100] In some possible embodiments, after deriving the split state of the output tensor of the first operator, the tensor processing method further includes: when the output tensor of the first operator is a j-th source input tensor among one or more source input tensors of a second operator in the target computation graph, setting the split state of the output tensor of the first operator as a source split state of the j-th source input tensor of the second operator, where j is an integer greater than or equal to 1.

[0101] According to different types of target computation graphs, we explain as follows.

[0102] In one example, the target computation graph is a target dynamic computation graph, in which case, after inputting each of the target input tensors to the first operator to obtain an output tensor of the first operator, when downstream operators of the first operator are set in the target dynamic computation graph and the output tensor of the first operator is set as the j-th source input tensor among one or more source input tensors of the current operator, the split state of the output tensor of the first operator can be set as the source split state of the j-th source input tensor of the second operator.

[0103] In one example, the target computation graph is a target static computation graph, in which case, after deriving the target split state of one or more target input tensors of the first operator and deriving the split state of the output tensor of the first operator, if the target static computation graph includes a downstream operator of the first operator and the output tensor of the first operator is the jth source input tensor of one or more source input tensors of the downstream operator, the downstream operator can be a second operator, and the split state of the output tensor of the first operator can be the source split state of the jth source input tensor of the second operator.

[0104] It should be noted that regardless of whether the target computation graph is static or dynamic, the number of output tensors of the first operator may be one or more. When the first operator has multiple output tensors, if the m-th output tensor of the first operator is the j-th source input tensor among one or more source input tensors of a second operator in the target computation graph, the split state of the m-th output tensor of the first operator can be the source split state of the j-th source input tensor of the second operator, where m is an integer greater than or equal to 1, and the m-th output tensor is one of the multiple output tensors of the first operator.

[0105] 2, the process by which the first operator 200 obtains the output tensor C231 has been described in detail in the above embodiment and will not be described again here. The output tensor C of the first operator 200 serves as one source input tensor C of the downstream second operator 240, and the split state of the output tensor C can be directly used as the source split state of one source input tensor C of the second operator 240. Furthermore, the second operator 240 further includes another source input tensor D241, and the source input tensor D may not have a distributed split mark set by the user (which can be set as the default split state). Similarly, the source input tensors of the second operator also determine corresponding target split states, and finally obtain target input tensors C'232 and D'242 through transformation, and the second operator can obtain the output tensor E250.

[0106] By adopting the above method, regardless of whether the target computation graph is a static computation graph or a dynamic computation graph, the source splitting state of any of the source input tensors of the downstream operators of the first operator can be determined based on the splitting state of the output tensor of the first operator. In this way, whether the target computation graph is a static computation graph or a dynamic computation graph, it is possible to ensure that the corresponding splitting states of each tensor are set in the same way.

[0107] The tensor processing method provided in this embodiment will be exemplarily described with reference to FIG.

[0108] In S411, the user can set a current operator (e.g., the first operator in the above embodiment) in the single-card network construction (network model construction) of the target dynamic graph and set the distributed partition marks of at least some source input tensors of the current operator. For example, based on the distributed partition marks of at least some source input tensors of the current operator, obtain the source partition states of at least some source input tensors of the current operator, and / or set one or more output tensors and their corresponding partition states of one or more upstream operators of the current operator as one or more source input tensors and their corresponding source partition states of the current operator.

[0109] In S412, a partition is derived in the target dynamic graph, for example, a target partition state of one or more target input tensors of the current operator is derived based on the computation logic of the current operator, the source partition state of at least some source input tensors of the current operator, and the partition rules 401 (e.g., the preset partition constraints of the current operator), and a partition state of an output tensor of the current operator is derived.

[0110] In S413, a split transformation is determined in the target dynamic graph, for example, based on the split transformation rule 402 and the target split state of one or more target input tensors of the current operator, the relevant information of the transformation function corresponding to each target input tensor of one or more target input tensors of the current operator is determined.

[0111] In S414, the target dynamic graph immediately executes the current operator. The target dynamic graph executes the current operator through its corresponding call stack, for example, calling a transformation function corresponding to each of the target input tensors of the current operator from the communication library 403 based on the associated information of the transformation function corresponding to each of the target input tensors, obtaining a first transformation function corresponding to each of the target input tensors of the current operator based on the transformation function corresponding to each of the target input tensors of the current operator and dynamic encapsulation parameters, performing division on each of the source input tensors of the current operator based on the first transformation function corresponding to each of the target input tensors of the current operator to obtain each of the target input tensors, and performing distributed parallel communication among multiple computing devices based on each of the target input tensors and the current operator to obtain an output tensor of the first operator.

[0112] Each time a new operator is set in the target dynamic graph, the above steps S411 to S414 can be repeatedly executed, and finally, a plurality of operators can be obtained in the target dynamic graph after the construction of the network model. Then, according to actual needs, the execution of the dynamic to static conversion process can be selected, and the relevant process flow includes:

[0113] In S421, a dynamic-to-static process is performed on the target dynamic graph to obtain a target static graph, which includes a distributed partition mark in the target dynamic graph, and which can include multiple operators in the target dynamic graph.

[0114] In S422, a partition is derived in the target static graph. For example, a target partition state of one or more target input tensors of each operator is derived, and a partition state of an output tensor of each operator is derived based on the calculation logic of each operator in the plurality of operators, the source partition state of at least some source input tensors of each operator, and the partition rules 401 (such as preset partition constraints of each operator).

[0115] In S423, a split transformation is determined in the target static graph, for example, based on the split transformation rules 402 and the target split states of one or more target input tensors of each of the operators in the plurality of operators, related information of a transformation function corresponding to each of the target input tensors in one or more target input tensors of each of the operators is determined.

[0116] In S424, the entire target static graph is recorded and executed. Specifically, the target static graph records the multiple operators included in the entire graph, the target split states of the target input tensors of each operator, and the transformation functions corresponding to the target input tensors of each operator. Then, the entire target static graph is executed. The target static graph executes the multiple operators included in the entire graph according to their corresponding call stacks. Note that the target static graph may execute each operator sequentially based on the dependency relationships between each operator in the entire graph. The process of executing any current operator is the same as that of S414 described above, and will not be described again.

[0117] 5 is a schematic diagram illustrating the configuration of a tensor processing device provided in one embodiment of the present disclosure. a transformation function determination module 501 for determining, based on the computation logic of a first operator in a target computation graph and the source split state of at least some source input tensors of the first operator, relevant information of a transformation function corresponding to each target input tensor among one or more target input tensors of the first operator, wherein when the target computation graphs are different types of computation graphs, relevant information of a transformation function corresponding to any target input tensor among the one or more target input tensors is the same; a tensor division module 502 for performing division on each of the source input tensors of the first operator according to the associated information of the transformation function corresponding to each of the target input tensors to obtain each of the target input tensors; a communication module 503 for transmitting each of the target input tensors to a plurality of computing devices, the plurality of computing devices performing distributed parallel communication based on each of the target input tensors and the first operator to obtain an output tensor of the first operator.

[0118] As shown in FIG. 6, the device a target split state derivation module 601 for deriving a target split state for each target input tensor among the one or more target input tensors of the first operator based on a computation logic of the first operator in the target computation graph and a source split state of at least some source input tensors of the first operator; The transformation function determination module is used for determining, according to the target division state of each of the target input tensors of the first operator, relevant information of a transformation function corresponding to each of the target input tensors of the first operator.

[0119] The target partitioning state derivation module is used for deriving a target partitioning state of each target input tensor among one or more target input tensors of the first operator based on the computation logic of the first operator in the target computation graph, the source partitioning states of at least some source input tensors of the first operator, and the preset partitioning constraints of the first operator, and deriving a partitioning state of an output tensor of the first operator, wherein the preset partitioning constraints of the first operator are the same when the target computation graphs are different types of computation graphs, and the preset partitioning constraints of the first operator include at least one of: the target partitioning states of the one or more target input tensors satisfy the computation requirements of the first operator, and implementing the efficiency requirements of distributed parallel communication based on the target partitioning states of the one or more target input tensors of the first operator.

[0120] The target split state derivation module is used to, when the output tensor of the first operator is the j-th source input tensor among one or more source input tensors of the second operator in the target computation graph, set the split state of the output tensor of the first operator as the source split state of the j-th source input tensor of the second operator, where j is an integer greater than or equal to 1.

[0121] The tensor splitting module is used for calling a transformation function corresponding to each of the target input tensors from a function library through a call stack corresponding to the target computation graph based on related information of the transformation function corresponding to each of the target input tensors, and performing splitting on each of the source input tensors of the first operator based on the transformation function corresponding to each of the target input tensors to obtain each of the target input tensors.

[0122] When the type of the target computation graph is a dynamic computation graph, the tensor division module is used for: obtaining a first transformation function corresponding to each of the target input tensors based on a transformation function corresponding to each of the target input tensors and a dynamic encapsulation parameter; and performing division on each of the source input tensors of the first operator based on the first transformation function corresponding to each of the target input tensors to obtain each of the target input tensors.

[0123] When the type of the target computation graph is a static computation graph, the tensor division module is used for: obtaining a second transformation function corresponding to each of the target input tensors based on a transformation function corresponding to each of the target input tensors and a static encapsulation parameter; and performing division on each of the source input tensors of the first operator based on the second transformation function corresponding to each of the target input tensors to obtain each of the target input tensors.

[0124] The target split state derivation module: When a type of the target computation graph is a dynamic computation graph, in response to obtaining a distributed partition mark set for an i-th source input tensor of the first operator, obtain a source partition state of the i-th source input tensor of the first operator based on the distributed partition mark corresponding to the i-th source input tensor of the first operator, where i is an integer greater than or equal to 1, and the i-th source input tensor is one of the at least some source input tensors; When the type of the target computation graph is the dynamic computation graph, in response to setting the k-th output tensor of a third operator in the target computation graph to the i-th source input tensor of the first operator, set the corresponding split state of the k-th output tensor of the third operator to the source split state of the i-th source input tensor of the first operator, where k is an integer greater than or equal to 1.

[0125] As shown in FIG. 7, the type of the target computation graph is a static computation graph, and the tensor processing unit: The dynamic-to-static transformation module 701 further includes: a dynamic-to-static transformation module 701 for generating the target computation graph based on the original dynamic computation graph in response to obtaining a static transformation instruction under the original dynamic computation graph, the target computation graph including the same multiple operators as the original dynamic computation graph, distributed partition marks of one or more source input tensors of the multiple operators, and the multiple operators including the first operator.

[0126] The target split state derivation module: If the i-th source input tensor of the first operator has a corresponding distributed partition mark, obtain a source partition state of the i-th source input tensor of the first operator based on the corresponding distributed partition mark of the i-th source input tensor of the first operator, where i is an integer greater than or equal to 1, and the i-th source input tensor is one of the at least some source input tensors; and when the i-th source input tensor of the first operator in the target computation graph is the k-th output tensor of a third operator, set the corresponding split state of the k-th output tensor of the third operator to the source split state of the i-th source input tensor of the first operator, where k is an integer greater than or equal to 1.

[0127] For specific functions and exemplary descriptions of each module and sub-module of the apparatus according to the embodiments of the present disclosure, please refer to the relevant descriptions of the corresponding steps in the above-mentioned method embodiments, and they will not be repeated here.

[0128] In the technical solution of the present disclosure, the acquisition, storage, and application of users' personal information comply with the provisions of relevant laws and regulations and do not violate public order and morals.

[0129] According to embodiments of the present disclosure, the present disclosure further provides an electronic device, a non-transitory computer-readable storage medium, and a program product.

[0130] 8 is a block diagram of an electronic device 800 for implementing an embodiment of the present disclosure. The electronic device refers to various types of digital computers, including, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device also refers to various types of mobile devices, including, for example, personal digital assistants, cellular phones, intelligent phones, wearable devices, and other similar computing devices. The components, their connections, and functions described in this disclosure are merely exemplary and do not limit the implementation of what is described and specified in this disclosure.

[0131] 8, device 800 includes a computing unit 801 that can perform various appropriate operations and processes based on computer program instructions stored in a read-only memory (ROM) 802 or loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 can further store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0132] Multiple components in device 800 are connected to an I / O interface 805, which includes an input unit 806 such as a keyboard or mouse, an output unit 807 such as various displays and speakers, a storage unit 808 such as a magnetic disk or optical disk, and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various carrier networks.

[0133] The computing unit 801 may be various general-purpose and / or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, computing units that execute various machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs each of the methods and processes described above, such as the thread stripping method. For example, in some embodiments, the thread stripping method may be implemented as a computer software program tangibly embodied in a machine-readable medium such as the storage unit 808. In some embodiments, some or all of the computer program may be loaded and / or installed into the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, it may perform one or more steps of the thread stripping method described above. Additionally, in other embodiments, the computing unit 801 may be configured to perform the thread stripping method in any other suitable manner (eg, firmware).

[0134] According to another aspect of the present disclosure, there is provided an autonomous vehicle including the electronic device described above.

[0135] Various embodiments of the systems or techniques described in this disclosure may be implemented using digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. Each of these embodiments may involve execution by one or more computer programs executed and / or interpreted by a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose programmable processor, capable of receiving data and instructions from, and transferring data and instructions to, a storage system, at least one input device, and at least one output device.

[0136] Program code for carrying out the methods of the present disclosure can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programming data processing apparatus, such that when the program code is executed by the processor or controller, it can perform the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on-site, partially on-site, partially on-site and partially on a remote site as a separate soft encapsulation, or entirely on a remote site or server.

[0137] In the context of this disclosure, a machine-readable medium may be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Further examples of machine-readable storage media include one or more hard-wired electrical connections, a portable computer disk cartridge, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any combination of the foregoing.

[0138] To provide for user interaction, the systems and techniques described herein can be implemented on a computer that includes a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor, etc.) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball, etc.) for the user to provide input to the computer. Other types of devices can also be used to provide for user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, haptic feedback, etc.), and input from the user can be received in any form (e.g., acoustic input, voice input, tactile input, etc.).

[0139] The systems and techniques described herein can be implemented in a computing system that includes background components (e.g., as a data server), middleware components (e.g., an application server), front-end components (e.g., a user computer having a graphical user interface or network browser through which a user can interact with embodiments of the systems and techniques described herein), or any combination of such background, middleware, or front-end components. Components of the system can be connected to each other via any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0140] The computer system may include a client and a server. Typically, the client and server are remote from each other and generally interact via a communication network. The client-server relationship is created by a computer program running on a corresponding computer. The server may be a cloud server, a server in a distributed system, or a server incorporating a blockchain.

[0141] It should be understood that steps can be newly ranked, added, or deleted using the various aspects of the flow shown above. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order. This disclosure is not limited thereto, as long as the technical solutions disclosed in this disclosure can achieve the desired results.

[0142] The above specific examples do not constitute limitations on the scope of protection of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions are possible depending on design considerations and other factors. Any modifications, equivalent replacements, improvements, etc. within the spirit and principles of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. 1. A tensor processing method comprising: Determine related information of a transformation function corresponding to each target input tensor among one or more target input tensors of the first operator based on a computation logic of the first operator in the target computation graph and a source split state of at least some source input tensors of the first operator, where when the target computation graphs are different types of computation graphs, related information of a transformation function corresponding to any target input tensor among the one or more target input tensors is the same; performing division on each of the source input tensors of the first operator according to the associated information of the transformation function corresponding to each of the target input tensors to obtain each of the target input tensors; transmitting each of the target input tensors to a plurality of computing devices, the plurality of computing devices performing distributed parallel communication based on each of the target input tensors and the first operator to obtain an output tensor of the first operator; Tensor processing methods.

2. Determining, based on a computation logic of a first operator in the target computation graph and a source split state of at least a portion of source input tensors of the first operator, relevant information of a transformation function corresponding to each target input tensor among one or more target input tensors of the first operator, includes: deriving a target split state for each target input tensor among the one or more target input tensors of the first operator based on a computation logic of the first operator in the target computation graph and a source split state of at least some source input tensors of the first operator; and determining, based on a target split state of each of the target input tensors of the first operator, relevant information of a transformation function corresponding to each of the target input tensors of the first operator. The tensor processing method of claim 1 .

3. Deriving a target split state for each target input tensor among the one or more target input tensors of the first operator based on computation logic of the first operator in the target computation graph and source split states of at least some source input tensors of the first operator includes: deriving a target partitioning state for each target input tensor among one or more target input tensors of the first operator and deriving a partitioning state for an output tensor of the first operator based on a computation logic of the first operator in the target computation graph, a source partitioning state of at least a part of source input tensors of the first operator, and a preset partitioning constraint of the first operator, wherein the preset partitioning constraint of the first operator is the same when the target computation graph is a different type of computation graph, and the preset partitioning constraint of the first operator includes at least one of: the target partitioning state of the one or more target input tensors satisfies a computation requirement of the first operator; and implementing a distributed parallel communication efficiency requirement based on the target partitioning state of the one or more target input tensors of the first operator. The tensor processing method of claim 2 .

4. After deriving the split state of the output tensor of the first operator, the tensor processing method comprises: When the output tensor of the first operator is a j-th source input tensor among one or more source input tensors of a second operator in the target computation graph, the split state of the output tensor of the first operator is a source split state of the j-th source input tensor of the second operator, where j is an integer greater than or equal to 1; 4. The tensor processing method of claim 3.

5. performing division on each of the source input tensors of the first operator based on associated information of a transformation function corresponding to each of the target input tensors to obtain each of the target input tensors, calling a transformation function corresponding to each of the target input tensors from a function library through a call stack corresponding to the target computation graph based on association information of the transformation function corresponding to each of the target input tensors; performing division on each of the source input tensors of the first operator based on a transformation function corresponding to each of the target input tensors to obtain each of the target input tensors; The tensor processing method of claim 1 .

6. performing division on each of the source input tensors of the first operator based on a transformation function corresponding to each of the target input tensors to obtain each of the target input tensors, If the type of the target computation graph is a dynamic computation graph, obtaining a first transformation function corresponding to each of the target input tensors based on a transformation function corresponding to each of the target input tensors and a dynamic encapsulation parameter; performing a division on each of the source input tensors of the first operator based on a first transformation function corresponding to each of the target input tensors to obtain each of the target input tensors; 6. The tensor processing method of claim 5.

7. performing division on each of the source input tensors of the first operator based on a transformation function corresponding to each of the target input tensors to obtain each of the target input tensors, If the type of the target computation graph is a static computation graph, obtaining a second transformation function corresponding to each of the target input tensors based on the transformation function corresponding to each of the target input tensors and a static encapsulation parameter; performing a division on each of the source input tensors of the first operator based on a second transformation function corresponding to each of the target input tensors to obtain each of the target input tensors; 6. The tensor processing method of claim 5.

8. Before determining, based on the computation logic of a first operator in the target computation graph and the source split state of at least a portion of the source input tensors of the first operator, relevant information of a transformation function corresponding to each target input tensor among one or more target input tensors of the first operator, the tensor processing method includes: When a type of the target computation graph is a dynamic computation graph, in response to obtaining a distributed partition mark set for an i-th source input tensor of the first operator, obtain a source partition state of the i-th source input tensor of the first operator based on the distributed partition mark corresponding to the i-th source input tensor of the first operator, where i is an integer greater than or equal to 1, and the i-th source input tensor is one of the at least some source input tensors; and when the type of the target computation graph is the dynamic computation graph, in response to setting a k-th output tensor of a third operator in the target computation graph to an i-th source input tensor of the first operator, setting a corresponding split state of the k-th output tensor of the third operator to a source split state of the i-th source input tensor of the first operator, where k is an integer greater than or equal to 1. The tensor processing method of claim 1 .

9. the type of the target computation graph is a static computation graph; The tensor processing method comprises: In response to obtaining a static transformation instruction under an original dynamic computation graph, generating the target computation graph based on the original dynamic computation graph, wherein the target computation graph includes the same multiple operators as the original dynamic computation graph, distributed partition marks of one or more source input tensors of the multiple operators, and the multiple operators include the first operator. The tensor processing method of claim 1 .

10. Before determining, based on the computation logic of a first operator in the target computation graph and the source split state of at least a portion of the source input tensors of the first operator, relevant information of a transformation function corresponding to each target input tensor among one or more target input tensors of the first operator, the tensor processing method includes: If the i-th source input tensor of the first operator has a corresponding distributed partition mark, obtain a source partition state of the i-th source input tensor of the first operator based on the corresponding distributed partition mark of the i-th source input tensor of the first operator, where i is an integer greater than or equal to 1, and the i-th source input tensor is one of the at least some source input tensors; and when the i-th source input tensor of the first operator in the target computation graph is the k-th output tensor of a third operator, a corresponding split state of the k-th output tensor of the third operator is set as a source split state of the i-th source input tensor of the first operator, where k is an integer greater than or equal to 1.

10. The tensor processing method of claim 9.

11. 1. A tensor processing device, comprising: a transformation function determination module for determining, based on a computation logic of a first operator in a target computation graph and a source split state of at least a part of source input tensors of the first operator, relevant information of a transformation function corresponding to each target input tensor among one or more target input tensors of the first operator, wherein when the target computation graphs are different types of computation graphs, relevant information of a transformation function corresponding to any target input tensor among the one or more target input tensors is the same; a tensor division module for performing division on each of the source input tensors of the first operator based on related information of a transformation function corresponding to each of the target input tensors to obtain each of the target input tensors; a communication module for transmitting each of the target input tensors to a plurality of computing devices, the plurality of computing devices performing distributed parallel communication based on each of the target input tensors and the first operator to obtain an output tensor of the first operator; Tensor processing unit.

12. The tensor processing device includes: a target split state derivation module for deriving a target split state for each target input tensor among the one or more target input tensors of the first operator based on a computation logic of the first operator in the target computation graph and a source split state of at least some source input tensors of the first operator; The transformation function determination module is used to determine the relevant information of the transformation function corresponding to each of the target input tensors of the first operator according to the target split state of each of the target input tensors of the first operator; The tensor processing device of claim 11 .

13. The target partitioning state derivation module derives a target partitioning state of each target input tensor among one or more target input tensors of the first operator, and derives a partitioning state of an output tensor of the first operator, based on the computation logic of the first operator in the target computation graph, a source partitioning state of at least a part of the source input tensors of the first operator, and a preset partitioning constraint of the first operator, wherein the preset partitioning constraint of the first operator is the same when the target computation graph is a different type of computation graph, and the preset partitioning constraint of the first operator includes at least one of: the target partitioning state of the one or more target input tensors satisfies the computation requirement of the first operator; and implementing an efficiency requirement of distributed parallel communication based on the target partitioning state of the one or more target input tensors of the first operator. The tensor processing device of claim 12.

14. The target split state deriving module is used to, when the output tensor of the first operator is the jth source input tensor among one or more source input tensors of the second operator in the target computation graph, set the split state of the output tensor of the first operator as the source split state of the jth source input tensor of the second operator, where j is an integer greater than or equal to 1; The tensor processing device of claim 13.

15. The tensor division module is used for: calling a transformation function corresponding to each of the target input tensors from a function library through a call stack corresponding to the target computation graph based on related information of a transformation function corresponding to each of the target input tensors; and performing division on each of the source input tensors of the first operator based on the transformation function corresponding to each of the target input tensors to obtain each of the target input tensors. The tensor processing device of any one of claims 11 to 14.

16. When the type of the target computation graph is a dynamic computation graph, the tensor division module is used for: obtaining a first transformation function corresponding to each of the target input tensors according to a transformation function corresponding to each of the target input tensors and a dynamic encapsulation parameter; and performing division on each of the source input tensors of the first operator according to the first transformation function corresponding to each of the target input tensors to obtain each of the target input tensors. The tensor processing device of claim 15.

17. When the type of the target computation graph is a static computation graph, the tensor division module is used for: obtaining a second transformation function corresponding to each of the target input tensors based on a transformation function corresponding to each of the target input tensors and a static encapsulation parameter; and performing division on each of the source input tensors of the first operator based on the second transformation function corresponding to each of the target input tensors to obtain each of the target input tensors. The tensor processing device of claim 15.

18. The target split state derivation module: When a type of the target computation graph is a dynamic computation graph, in response to obtaining a distributed partition mark set for an i-th source input tensor of the first operator, obtain a source partition state of the i-th source input tensor of the first operator based on the distributed partition mark corresponding to the i-th source input tensor of the first operator, where i is an integer greater than or equal to 1, and the i-th source input tensor is one of the at least some source input tensors; and when the type of the target computation graph is the dynamic computation graph, in response to setting the k-th output tensor of a third operator in the target computation graph to the i-th source input tensor of the first operator, setting a corresponding split state of the k-th output tensor of the third operator to a source split state of the i-th source input tensor of the first operator, where k is an integer greater than or equal to 1. The tensor processing device of any one of claims 11 to 14.

19. the type of the target computation graph is a static computation graph; The tensor processing device includes: and a dynamic-to-static transformation module for generating a target computation graph based on the original dynamic computation graph in response to obtaining a static transformation instruction under the original dynamic computation graph, the target computation graph including a plurality of operators identical to the original dynamic computation graph, a distributed partition mark of one or more source input tensors of the plurality of operators, and the plurality of operators including the first operator. The tensor processing device of any one of claims 11 to 14.

20. The target split state derivation module: If the i-th source input tensor of the first operator has a corresponding distributed partition mark, obtain a source partition state of the i-th source input tensor of the first operator based on the corresponding distributed partition mark of the i-th source input tensor of the first operator, where i is an integer greater than or equal to 1, and the i-th source input tensor is one of the at least some source input tensors; and when the i-th source input tensor of the first operator in the target computation graph is the k-th output tensor of a third operator, setting the corresponding split state of the k-th output tensor of the third operator as the source split state of the i-th source input tensor of the first operator, where k is an integer greater than or equal to 1. The tensor processing device of claim 19.

21. at least one processor; a memory communicatively coupled to the at least one processor; 11. An electronic device, wherein the memory stores instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform the tensor processing method of any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the tensor processing method of any one of claims 1 to 10.

23. A program for implementing the tensor processing method according to any one of claims 1 to 10 when executed by a processor in a computer.

Images