Method, device, computer equipment and medium for determining tensor segmentation mode

Abstract

The embodiment of the present invention discloses a method, apparatus, computer equipment and medium for determining a tensor splitting mode. The method includes obtaining a computational graph to be tensor split including multiple computational operators; determining a data entry set corresponding to the computational graph including multiple target computational operators and a target tensor splitting mode of the target computational operator according to an optional tensor splitting mode set corresponding to the computational operator; calculating the computational time corresponding to each data entry according to each target tensor splitting mode; generating multiple cross data entries according to multiple data entry pairs selected from the data entry set, and updating the data entry set according to the computational time of each cross data entry; obtaining the target data entry in the updated data entry set, thereby determining the tensor splitting mode of each computational operator in the computational graph. The technical solution of the embodiment of the present invention provides a method applicable to complex scenarios and capable of efficiently determining a better splitting mode for an operator.

Description

Technical Field

[0001] Embodiments of the present invention relate to artificial intelligence chip technology, and in particular to a method, apparatus, computer equipment, and medium for determining a tensor segmentation pattern. Background Art

[0002] Existing AI processors typically integrate multiple or multiple groups of parallel computing units on-chip to achieve parallel data processing, reducing neural network training or inference processing time. Therefore, chips require upper-level neural network graph compilers. One processing method is to split the input tensor data of all operators in the computation graph and distribute it to multiple computing units for execution.

[0003] Some specific problems faced by the above processing method are as follows: in some scenarios, not splitting may be more cost-effective than splitting; there are thousands of operators in the entire computational graph, and each operator usually supports multiple tensor splitting forms. Therefore, there are many permutations and combinations that need to be processed. Finding the optimal or better splitting method relatively quickly is a huge challenge; the network is ever-changing, and it is impossible to adapt each network separately. Universality and scalability are also issues that must be addressed.

[0004] Current mainstream computing frameworks such as Pytorch and TensorFlow do not support optimized processing for the above scenarios. Therefore, there is an urgent need to explore a method that can be applied to complex scenarios and efficiently determine the optimal operator segmentation mode to meet the tensor segmentation needs in various application scenarios. Summary of the Invention

[0005] Embodiments of the present invention provide a method, apparatus, computer equipment, and medium for determining a tensor splitting mode, so as to provide a method applicable to complex scenarios and capable of efficiently determining an optimal splitting mode for an operator.

[0006] In a first aspect, an embodiment of the present invention provides a method for determining a tensor segmentation mode, the method comprising:

[0007] Get the computational graph to be split, which includes multiple computational operators.

[0008] Determine a data entry set corresponding to the computation graph according to an optional tensor splitting mode set corresponding to the computation operator, wherein the data entry includes a plurality of target computation operators and target tensor splitting modes of the target computation operators;

[0009] Calculating the computation time corresponding to each data entry according to each target tensor segmentation mode included in each data entry;

[0010] generating a plurality of cross data entries according to a plurality of data entry pairs selected from the data entry set, and updating the data entry set according to a computation time of each cross data entry;

[0011] A target data entry is obtained from the updated data entry set, and a tensor splitting mode of each computing operator in the computation graph is determined based on the target data entry.

[0012] In a second aspect, an embodiment of the present invention further provides a device for determining a tensor splitting mode, the device comprising:

[0013] The computation graph acquisition module is used to obtain the computation graph to be split into tensors, which includes multiple computation operators.

[0014] A data entry set determination module is used to determine a data entry set corresponding to the computation graph based on an optional tensor splitting mode set corresponding to the computation operator, wherein the data entry includes multiple target computation operators and target tensor splitting modes of the target computation operators;

[0015] A computation time determination module, configured to calculate the computation time corresponding to each data entry according to each target tensor segmentation mode included in each data entry;

[0016] a data entry set updating module, configured to generate a plurality of cross data entries based on a plurality of data entry pairs selected from the data entry set, and update the data entry set based on a calculation time of each cross data entry;

[0017] The tensor splitting mode determination module is used to obtain the target data entry in the updated data entry set, and determine the tensor splitting mode of each computing operator in the calculation graph according to the target data entry.

[0018] In a third aspect, an embodiment of the present invention further provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for determining the tensor splitting pattern as described in any embodiment of the present invention is implemented.

[0019] In a fourth aspect, an embodiment of the present invention further provides a storage medium of computer-executable instructions, on which a computer program is stored. When the program is executed by a processor, the method for determining the tensor splitting pattern as described in any embodiment of the present invention is implemented.

[0020] The embodiment of the present invention obtains a computational graph including multiple computational operators to be tensor split; determines a data entry set including multiple target computational operators and target tensor splitting modes of the target computational operators in each data entry corresponding to the computational graph according to an optional tensor splitting mode set corresponding to the computational operators; calculates the computation time corresponding to each data entry according to each target tensor splitting mode included in each data entry; generates multiple cross data entries according to multiple data entry pairs selected from the data entry set, and updates the data entry set according to the computation time of each cross data entry; obtains the target data entry in the updated data entry set, and determines the tensor splitting mode of each computational operator in the computational graph according to the target data entry. The technical means solves the problem that the existing technology cannot efficiently determine the optimal splitting method of the operator in complex scenarios, and provides a method that is applicable to complex scenarios and can efficiently determine the optimal splitting mode of the operator. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flowchart of a method for determining a tensor segmentation mode provided in Example 1 of the present invention;

[0022] Figure 2 A flowchart of another method for determining a tensor segmentation mode provided in the second embodiment of the present invention;

[0023] Figure 3a A flowchart of another method for determining a tensor segmentation mode provided in the third embodiment of the present invention;

[0024] Figure 3b This is a specific application scenario for obtaining cross-data entries provided by the third embodiment of the present invention;

[0025] Figure 3c This is a specific application scenario of performing a random replacement operation on a pair provided by the third embodiment of the present invention;

[0026] Figure 4 A schematic structural diagram of a device for determining a tensor segmentation mode provided in a fourth embodiment of the present invention;

[0027] Figure 5 This is a structural diagram of a computer device provided in Example 5 of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0029] Example 1

[0030] Figure 1 This is a flowchart of a method for determining a tensor splitting mode provided in the first embodiment of the present invention. This embodiment is applicable to the case of performing tensor splitting on an operator. The method can be performed by a tensor splitting mode determination device, which can be implemented in software and / or hardware and generally integrated into an artificial intelligence processor. The method specifically includes the following steps:

[0031] S101. Obtain a computational graph for tensor segmentation, where the computational graph includes multiple computational operators.

[0032] Among them, a computation graph corresponds to a computation task. The computation graph is a graphical representation of the computation process of the computation task. A computation graph corresponds to one or more input tensor data and one or more output tensor data.

[0033] A computation graph includes multiple operators, which can be understood as symbols that implement a given computational operation, such as addition, subtraction, exponentiation, or convolution. Similarly, each operator corresponds to one or more input tensors and one or more output tensors. Multiple operators are combined according to a predefined topology to form the computation graph.

[0034] S102. Determine a data entry set corresponding to the computation graph based on a set of optional tensor splitting modes corresponding to the computation operators, where the data entries include multiple target computation operators and target tensor splitting modes of the target computation operators.

[0035] The set of optional tensor splitting modes may be a set of all applicable tensor splitting modes corresponding to each computing operator. A data entry may be a mapping relationship entry consisting of multiple computing operators and their corresponding tensor splitting modes. A data entry set may be a set of multiple data entries. A target computing operator may refer to a computing operator that can determine a tensor splitting mode. A target tensor splitting mode may refer to the tensor splitting mode corresponding to the computing operator included in each data entry.

[0036] In this embodiment, a plurality of data entries corresponding to the computation graph may be determined based on a set of optional tensor splitting modes corresponding to each computation operator in the computation graph to form a data entry set.

[0037] S103 : Calculate the computation time corresponding to each data entry according to each target tensor segmentation mode included in each data entry.

[0038] The computation time may refer to the time consumed by each computation operator and the target tensor segmentation mode of each computation operator in each data entry.

[0039] In this embodiment, the computation time corresponding to each data entry can be calculated based on each computation operator in each data entry and the target tensor segmentation mode corresponding to each computation operator.

[0040] S104 : Generate multiple cross data entries according to multiple data entry pairs selected from the data entry set, and update the data entry set according to the calculation time of each cross data entry.

[0041] The data entry pair may include two data entries, and the cross data entry may refer to an entry generated according to each data entry pair. The cross data entry may be obtained by combining a part of one data entry with a part of another data entry in the data entry pair.

[0042] Specifically, data entries can be randomly selected from the data entry set to generate multiple data entry pairs, thereby generating corresponding cross data entries according to each data entry pair, and updating the data entry set according to the calculation time of each cross data entry, that is, replacing the data entry in the data entry set with the longest calculation time.

[0043] S105 . Obtain a target data entry from the updated data entry set, and determine a tensor splitting mode for each computing operator in the computation graph based on the target data entry.

[0044] The target data entry may be a data entry obtained from the updated data entry set.

[0045] In this embodiment, the data entry with the shortest computation time can be obtained from the updated data entry set as the target data entry, and the tensor splitting mode corresponding to each computation operator in the computation graph can be determined based on the target data entry.

[0046] The technical solution of the embodiment of the present invention obtains a computational graph including multiple computational operators to be tensor split; determines a data entry set including multiple target computational operators and target tensor splitting modes of the target computational operators in each data entry corresponding to the computational graph according to an optional tensor splitting mode set corresponding to the computational operators; calculates the computational time corresponding to each data entry according to each target tensor splitting mode included in each data entry; generates multiple cross data entries according to multiple data entry pairs selected from the data entry set, and updates the data entry set according to the computational time of each cross data entry; obtains the target data entry in the updated data entry set, and determines the tensor splitting mode of each computational operator in the computational graph according to the target data entry. This technical solution solves the problem that the prior art cannot efficiently determine the optimal splitting method of the operator in complex scenarios, and provides a method that is applicable to complex scenarios and can efficiently determine the optimal splitting mode of the operator.

[0047] Example 2

[0048] Figure 2 This is a flowchart of another method for determining a tensor splitting mode provided in the second embodiment of the present invention. Based on the above embodiments, this embodiment preferably further refines the operation of determining a data entry set corresponding to a computation graph according to an optional tensor splitting mode set corresponding to a computation operator. Figure 2 , specifically including the following steps:

[0049] S201. Obtain a computational graph for tensor segmentation, where the computational graph includes multiple computational operators.

[0050] S202: Topologically sort the computation operators in the computation graph.

[0051] As mentioned above, the topological structure of multiple computing operators is defined in the computation graph. This topological structure represents the execution order of each computing operator in the entire computing task. By parsing the computation graph, the execution order of each computing operator can be obtained. Then, the computing operators can be topologically sorted in the order of execution from front to back.

[0052] S203: Create a data entry template, and obtain a current calculation operator in sequence according to the sorting order.

[0053] The data entry template may be a template containing at least two blank filling positions, for example, a calculation operator filling position and a tensor splitting pattern filling position.

[0054] In this embodiment, a data entry template containing multiple blank filling positions may be created, and calculation operators are selected in sequence as current calculation operators according to the topological sorting order.

[0055] S204: query the operator information library according to the operator type of the current computing operator, and when it is determined that the operator information library stores an optional tensor splitting mode set corresponding to the current computing operator, determine the current computing operator as the target computing operator.

[0056] The operator information library can be a pre-built information library. The operator information library can include a mapping relationship between operator types and a set of optional tensor splitting modes. Each optional tensor splitting mode set stores one or more optional tensor splitting modes. That is, a computing operator can correspond to one or more optional tensor splitting modes. Each optional tensor splitting mode includes: an input splitting state for each input terminal and a reference output splitting state for each output terminal.

[0057] For example, as shown in Table 1: a convolution type computing operator corresponds to two optional tensor splitting modes, and each optional tensor splitting mode corresponds to a matching input splitting state and a reference output splitting state.

[0058] Table 1

[0059] Operator Type Tensor splitting mode number Input 0 state Input 1 status Output 0 state Conv 0 FF FF FF 1 00 01 00

[0060] As shown in Table 1, the convolution type operator has two input terminals and one output terminal. The input 0 state can be understood as the input split state of the first input terminal of the operator, the input 1 state can be understood as the input split state of the second input terminal of the operator, and the output 0 state can be understood as the reference value of the output split state of the only output terminal of the operator.

[0061] The input split state can be understood as a specific split form of the input tensor data. For example, a split operation or a copy operation can be performed on a set dimension in the input tensor data. The reference output split state can be understood as a possible split form of the output tensor data. For example, a merge operation or a reduction operation can be performed on a set dimension in the output tensor data.

[0062] It should be noted that the reason why the above-mentioned segmentation states are divided into input segmentation states and reference output segmentation states is mainly because after the input segmentation state is determined, there are actually many possibilities for the corresponding output segmentation state, which may be different due to the different computing scenarios based on it. Therefore, it may not be possible to enumerate all possible output segmentation states only by constructing an information library. Furthermore, in the information library, only one reference output segmentation state that is adapted to general scenarios can be listed to assist in determining the target output segmentation state that best matches an input segmentation state. That is, the target output segmentation state is a specific segmentation form of the input tensor data corresponding to the input segmentation state.

[0063] In this embodiment, different expressions of tensor data segmentation forms may also be preset, for example, "FF", "00" or "01" as described in Table 1.

[0064] Specifically, based on the operator type of the current operator, the operator information library can be queried to determine whether there is an optional tensor splitting mode set corresponding to the current operator. If it is determined that it exists, indicating that the current operator can perform splitting operations (for example, the splitting, copying, merging, and reduction operations described above), then the current operator can be determined as the target operator. Correspondingly, if it is determined that the operator information library does not contain an optional tensor splitting mode set corresponding to the current operator, indicating that the current operator cannot perform splitting operations, then the current operator can be discarded, and the next operator can be obtained in sequence in sorted order as the current operator, and the above-mentioned operation of determining the target operator can be performed.

[0065] S205 . Determine a target tensor splitting mode of the target computing operator according to the optional tensor splitting mode set of the target computing operator, and add the target computing operator and the target tensor splitting mode to the data entry template.

[0066] Specifically, among multiple optional tensor splitting modes for the target computing operator, a specific optional tensor splitting mode is selected, and based on the optional tensor splitting mode, the target tensor splitting mode of the target computing operator is determined, thereby adding the target computing operator and the target tensor splitting mode of the target computing operator to the pre-created data entry template, that is, filling the corresponding blank segments of the data entry template. In an optional implementation of this embodiment, determining the target tensor splitting mode of the target computing operator based on the optional tensor splitting mode set of the target computing operator may include:

[0067] An alternative tensor splitting mode is selected from the optional tensor splitting mode set of the target computing operator, where the optional tensor splitting mode includes: an input splitting state of each input end and a reference output splitting state of each output end; based on the alternative input splitting state and the alternative reference output splitting state in the alternative tensor splitting mode, a target output splitting state corresponding to the alternative input splitting state is derived; it is verified whether the alternative input splitting state and the target output splitting state meet the input and output validity conditions corresponding to the target computing operator; if so, the alternative input splitting state and the target output splitting state are determined as the target tensor splitting mode of the target computing operator; otherwise, the operation of selecting the alternative tensor splitting mode from the optional tensor splitting mode set of the target computing operator is returned to be executed until the target tensor splitting mode of the target computing operator is successfully determined.

[0068] Among them, the alternative tensor splitting mode can be an optional tensor splitting mode selected from all optional tensor splitting modes in the optional tensor splitting mode set of the target computing operator. Specifically, the alternative tensor splitting mode can be obtained by random selection. For example, if the optional tensor splitting mode set of a target computing operator includes 5 optional tensor splitting modes, the above 5 optional tensor splitting modes are numbered sequentially from 0 to 4, and then a random number between [0, 4] can be generated, and according to the specific value of the random number, the alternative tensor splitting mode is determined among the 5 optional tensor splitting modes.

[0069] The input split state may refer to the state of a tensor attached to the input position of a computing operator. The reference output split state may refer to the state of a tensor attached to the output position of a computing operator for reference provided in the operator information library.

[0070] In an optional implementation of this embodiment, based on the alternative input split state and the alternative reference output split state in the alternative tensor split mode, the target output split state corresponding to the alternative input split state can be derived as follows:

[0071] The alternative tensor splitting mode and the computing environment that matches the computational graph (for example, the computing network adapted by the computational graph, the number of computing units, or the total dimension of the original input data, etc.) are input into a pre-built tensor state deduction function, and the corresponding output splitting state deduction result is obtained as the target output splitting state.

[0072] Optionally, matching tensor state deduction functions can be constructed according to different operator types. For example, if there are N types of computing operators, N different tensor state deduction functions can be constructed accordingly.

[0073] Specifically, a tensor state deduction function matching a set operator type can be constructed based on operator attributes of a calculation operator of a set operator type and parameters such as the operator calculation logic.

[0074] Among them, operator attributes may include: whether the calculation operator performs axis adjustment processing on the input tensor, etc. The operator calculation logic refers to the specific calculation method used by the calculation operator to calculate the input tensor.

[0075] In this embodiment, when calling a matching tensor state deduction function for the operator type of a computing operator to derive the output state, it is possible to first detect whether the alternative reference output split state in the alternative tensor split mode is a specific tensor split operation. For example, if the alternative reference output split state is to split the tensor along the X-axis into a specific tensor split operation, and if the alternative reference output split state is not to perform any splitting on the output tensor, it is not a specific tensor split operation.

[0076] If the detected alternative reference output split state is a specific tensor split operation, the alternative reference output split state can be directly output as the target output split state; if the detected alternative reference output split state is not a specific tensor split operation, the target output split state can be derived through the tensor state deduction function and the alternative input split state in the alternative tensor split mode.

[0077] Input and output validity conditions can be used to verify whether operators that cannot be split are incorrectly split. For example, some operators cannot be split in certain computing scenarios, otherwise the accuracy will be affected. Therefore, verification is required to remove illegal tensor splitting patterns.

[0078] The input and output validity conditions can be summarized and obtained by analyzing the computing performance of various types of computing operators in various computing scenarios.

[0079] The input split state and the output split state can be represented by a mask, as shown in Table 2 for example: different physical meanings represented by different input split states and output split states.

[0080] Table 2

[0081] Input split status Physical meaning Output segmentation status Physical meaning FF No operations are performed on the tensor FF Do nothing with the output tensor FC Tensor Copy FE Tensor Reduction 00 Split the tensor along axis 0 00 Merge tensors along axis 0 01 Split the tensor along the first axis 01 Merge tensors along the first axis 02 Split the tensor along the second axis 02 Merge tensors along the second axis 03 Split the tensor along the third axis 03 Merge tensors along the third axis

[0082] The specific mask can be freely defined, and its form is not limited to the specific symbols in the above table.

[0083] Specifically, an alternative tensor splitting mode can be randomly obtained from all optional tensor splitting modes retrieved from the operator information library by the target computing operator. Furthermore, based on the alternative input splitting states and the alternative reference output splitting states in the alternative tensor splitting mode, the target output splitting state corresponding to the alternative input splitting state can be derived. Typically, the target output splitting state can be derived using a tensor state derivation function.

[0084] After deriving the target output split state, it is possible to verify whether the alternative input split state and the derived target output split state satisfy the input and output validity conditions corresponding to the current target computing operator. If the input and output validity conditions are satisfied, the alternative input split state and the target output split state can be determined as the target tensor split mode of the target computing operator; otherwise, the operation of selecting the alternative tensor split mode from the optional tensor split mode set of the target computing operator can be returned until the target tensor split mode of the target computing operator is successfully determined.

[0085] The specific method of deriving the target output segmentation state based on the alternative reference output segmentation state can be referred to as follows:

[0086] If the alternative reference output split state is "FF", the derivation result of the tensor state derivation function can be directly used as the target output split state.

[0087] In another optional implementation of this embodiment, selecting an alternative tensor splitting mode from the optional tensor splitting mode set of the target computing operator may include:

[0088] Detect whether the target computing operator has a previous adjacent operator; if so, obtain the target output split state in the target tensor split mode determined for the previous adjacent operator; based on the target output split state of the previous adjacent operator, screen out the optional tensor split mode that satisfies the input-output connection relationship validity verification condition with the previous adjacent operator in the optional tensor split mode set of the target computing operator; select the alternative tensor split mode from each screened optional tensor split mode.

[0089] The previous adjacent operator refers to a computing operator that is located at the previous topological position of the currently processed target computing operator in the computing graph, and the computing operator is also the target computing operator.

[0090] The connection validity check condition can be used to check whether the output segmentation state of the previous operator and the input segmentation state of the next operator can be effectively connected. Similarly, this can be obtained by analyzing the computational performance of different operators in different computing scenarios and summarizing them.

[0091] In an optional implementation of this embodiment, a query dictionary can be constructed to obtain the validity verification conditions of the input-output connection relationship. The query dictionary may include multiple key-value pairs, the key name in each key-value pair is the output segmentation state of the previous calculation operator, and the key value in each key-value pair is at least one optional input segmentation state of the next calculation operator under the output segmentation state of the previous calculation operator corresponding to the key name.

[0092] Specifically, when verifying the validity check condition of the input-output connection relationship for the output split state A of the previous calculation operator and the input split state B of the next calculation operator, you can first query the target key-value pair whose key name matches the output split state A in the query dictionary based on the output split state A, and then determine whether the input split state B is a data item in the key value of the target key-value pair. If so, it is determined that the input split state B and the output split state A meet the validity check condition of the input-output connection relationship; otherwise, it is determined that the input split state B and the output split state A do not meet the validity check condition of the input-output connection relationship.

[0093] In this embodiment, an alternative tensor splitting mode is selected from the set of optional tensor splitting modes of the target computing operator, and can be obtained by screening based on input and output validity. Specifically, it can be detected whether the target computing operator has a previous adjacent computing operator. If so, a target tensor splitting mode can be determined for the previous adjacent operator, and the target output splitting state can be obtained in the target tensor splitting mode. Then, from the set of optional tensor splitting modes of the target computing operator, an optional tensor splitting mode that satisfies the input and output connection relationship validity verification condition with the target output splitting state of the previous adjacent operator is screened out as the alternative tensor splitting mode.

[0094] It should be noted that there are multiple optional tensor splitting modes corresponding to each target computing operator. Therefore, in the process of generating a data entry, it is necessary to randomly select one of them to determine the target tensor splitting mode of the target computing operator.

[0095] S206 , returning to execute the operation of obtaining one current calculation operator in sequence according to the sorting order until all calculation operators in the calculation graph are processed to generate a data entry.

[0096] Optionally, for the computational operators initially selected in the sorted order, after adding the computational operators and the corresponding target tensor splitting patterns to the data entry template, the next computational operator in the sorted order can be obtained as the current computational operator, and the corresponding target tensor splitting patterns and the addition of the data entry template can be performed. The computational operators are sequentially obtained in the sorted order, and the aforementioned addition of the data entry template is repeated until all computational operators in the computational graph are processed and a data entry is generated.

[0097] It should be noted that the various computational operators included in the computational graph to be tensor split may not require tensor splitting. Such computational operators may not be added to the data entry. Therefore, the computational operators included in the data entry may be a subset of the computational operators in the computational graph.

[0098] S207. Repeat the process from creating a data entry template to generating data entries until a set number of data entries are generated to form the data entry set, wherein the data entries include multiple target computing operators and target tensor segmentation patterns of the target computing operators.

[0099] In this embodiment, each target computing operator has multiple candidate tensor splitting modes, and one of them needs to be randomly selected to determine the target tensor splitting mode of the target computing operator. Therefore, the process from creating a data entry template to generating a data entry needs to be repeated. During each repeated execution of the operation, a target tensor splitting mode can be determined for each target computing operator. Then, according to the target tensor splitting mode corresponding to each target computing operator, a data entry can be formed by combining them until a set number of data entries are generated, thereby forming a data entry set based on multiple data entries.

[0100] It is understandable that the number of target computing operators included in each data entry in the data entry set is the same, but the target tensor segmentation modes corresponding to each target computing operator are not completely the same.

[0101] S208 . Calculate the computation time corresponding to each data entry according to each target tensor segmentation mode included in each data entry.

[0102] In an optional implementation of this embodiment, calculating the computation time corresponding to each data entry according to each target tensor splitting mode included in each data entry may include:

[0103] Obtain the current processing entry from all data entries, and obtain the original operator time consumption graph corresponding to the calculation graph, where the original operator time consumption graph has the same operator topology as that in the calculation graph; in the original operator time consumption graph, add the splitting operators corresponding to the target tensor splitting modes of each target operator in the current processing entry to generate the target operator time consumption graph corresponding to the current processing entry; query the pre-built time consumption model library and calculate the computational time corresponding to the target operator time consumption graph.

[0104] Among them, the original operator time consumption graph may refer to a time consumption graph obtained based on the initial computational graph containing multiple computational operators to be split for the tensor, and the original operator time consumption graph is the same as the operator topology structure in the computational graph. The operator topology structure may be a topology structure obtained based on multiple computational operators in the computational graph to be split for the tensor. The splitting operator may refer to an operator used to split tensor data, which is a new operator different from the computational operators in the computational graph. The splitting operator may correspond to the target tensor splitting mode. The target operator time consumption graph may refer to a time consumption graph corresponding to the data entry obtained after the splitting operator is added to the original operator time consumption graph and the corresponding processing operation is completed.

[0105] The corresponding processing operations may include operations such as merging and canceling of various segmentation operators. The pre-built time-consuming model library may include corresponding relationships between calculation operators and segmentation operators and calculation time consumption.

[0106] Specifically, one data entry can be sequentially obtained from all data entries in the data entry set as the current processing entry, and an original operator timing graph with the same operator topology as that in the computation graph can be obtained based on the computation graph. Splitting operators corresponding to the target tensor splitting patterns of each target operator in the current processing entry are then added to the original operator timing graph to generate a target operator timing graph corresponding to the current processing entry. By querying a pre-built timing model library, multiple computational times corresponding to each operator (including computational operators and splitting operators) in the target operator timing graph can be obtained. By summing these multiple computational times, the computational time corresponding to the target operator timing graph can be obtained.

[0107] Optionally, adding the splitting operators corresponding to the target tensor splitting modes of each target operator in the current processing entry to the original operator timing graph to generate the target operator timing graph corresponding to the current processing entry may include:

[0108] In the original operator time-consuming graph, add the splitting operators corresponding to the target tensor splitting modes of each target operator in the current processing entry; merge and / or cancel each of the added splitting operators to generate the target operator time-consuming graph corresponding to the current processing entry.

[0109] Specifically, in the original operator time-consuming graph, add the splitting operators corresponding to the target tensor splitting modes of each target operator in the current processing entry to detect whether these splitting operators can be merged and / or canceled. If so, the corresponding operations can be performed to obtain the target operator time-consuming graph. If not, the target operator time-consuming graph can be obtained directly.

[0110] In a specific example, if the operator timing graph includes a splitting operator A and a splitting operator B, the splitting operator A is used to split the second axis of the output tensor data of the calculation operator No. 1 once, and obtain N tensor data; and the splitting operator B is used to split the second axis of the input tensor data of the calculation operator No. 2 once, and obtain N tensor data. The above-mentioned calculation operator No. 1 and calculation operator No. 2 are connected sequentially. Then, the above-mentioned splitting operator A and splitting operator B can be merged, and the above-mentioned two splitting operators can be merged into a new calculation operator C, which is used to split the second axis of the output tensor data of the calculation operator No. 1 once, and obtain 2N tensor data.

[0111] In another specific example, if the operator timing graph includes a splitting operator D and a splitting operator E, the splitting operator D is used to merge the output tensor data of the calculation operator No. 3 on the second axis; and the splitting operator E is used to split the second axis of the input tensor data of the calculation operator No. 4, and the above-mentioned calculation operators No. 3 and No. 4 are connected sequentially, and then, the above-mentioned splitting operator C and splitting operator D can be canceled to remove the splitting operator D and splitting operator E included in the operator timing graph.

[0112] The advantage of this setting is that it can eliminate redundant merging and splitting operations and shorten the calculation time of data entries.

[0113] S209 : Generate multiple cross data entries according to multiple data entry pairs selected from the data entry set, and update the data entry set according to the calculation time of each cross data entry.

[0114] In this embodiment, a processing logic of selecting the best from the best is introduced. Assuming that the data entries in the data entry set contain better data entries, theoretically, it is possible to obtain better data entries by effectively combining the target tensor segmentation patterns of the target computing operators included in the above data entries. Based on the above theory, it is possible to construct data entries with lower computational cost by combining two data entries in the data entry set to obtain data entry pairs, and using each data entry pair to generate matching cross-data entries. Furthermore, when constructing the above-mentioned data entry with lower computational cost, the data entry can be used to replace the data entry with the maximum computational cost in the data entry set.

[0115] S210 . Obtain a target data entry from the updated data entry set, and determine a tensor splitting mode for each computing operator in the computation graph based on the target data entry.

[0116] The technical solution of the embodiment of the present invention is to topologically sort the various computing operators in the computing graph to be split into tensors; create a data entry template, and obtain the current computing operators in sequence according to the sorting order, query the operator information library, determine the target computing operator, and thus determine the target tensor splitting mode of the target computing operator, add the target computing operator and the target tensor splitting mode to the data entry template, return to execute the operation of obtaining the current computing operators in sequence until the processing of all computing operators in the computing graph is completed, and a data entry is generated; repeat the process from creating the data entry template to generating the data entry until the target tensor splitting mode is generated. A certain number of data entries are selected to form a data entry set; the computation time corresponding to each data entry is calculated; multiple cross data entries are generated according to multiple data entry pairs selected from the data entry set, and the data entry set is updated according to the computation time of each cross data entry; the target data entry is obtained from the updated data entry set, and the tensor splitting mode of each computing operator in the computation graph is determined according to the target data entry. This technical means solves the problem that the existing technology cannot efficiently determine the optimal splitting mode of the operator in complex scenarios, and provides a method that can be applied to complex scenarios and can efficiently determine the optimal splitting mode of the operator.

[0117] Based on the above technical solution, after completing the processing of all computing operators in the computation graph to generate a data entry, the following steps may also be included:

[0118] Based on the generated data entries and the topological sorting of each operator in the calculation graph, verify whether the two adjacent operators meet the input-output connection relationship validity verification conditions; if it is determined that any two adjacent operators do not meet the input-output connection relationship validity verification conditions, discard the data entry.

[0119] In this embodiment, after each data entry is generated, the validity of the current data entry can be verified based on the data entry to see whether any two adjacent operators in the computation graph meet the input-output connection validity verification conditions, thereby ensuring the validity of the current data entry. Specifically, based on the generated data entry and the topological sorting of the operators in the computation graph, it can be verified whether any two adjacent operators meet the input-output connection validity verification conditions. If it is determined that any group of two adjacent operators does not meet the input-output connection validity verification conditions, the currently verified data entry can be discarded. If all any two adjacent operators in the data entry meet the input-output connection validity verification conditions, it indicates that the current data entry is valid and can continue to be used.

[0120] Example 3

[0121] Figure 3aA flowchart of another method for determining a tensor splitting mode is provided in the third embodiment of the present invention. Based on the above embodiments, this embodiment preferably combines multiple data entry pairs selected from the data entry set to obtain multiple cross data entries, and further refines the operation of updating the data entry set based on the calculation time of each cross data entry. Referring to FIG3 , this method specifically includes the following steps:

[0122] S301. Obtain a computational graph for tensor segmentation, where the computational graph includes multiple computational operators.

[0123] S302. Determine a data entry set corresponding to the computation graph based on a set of optional tensor splitting modes corresponding to the computation operators, where the data entries include multiple target computation operators and target tensor splitting modes of the target computation operators.

[0124] S303 : Calculate the computation time corresponding to each data entry according to each target tensor segmentation mode included in each data entry.

[0125] S304: Select a first data entry and a second data entry from the data entry set.

[0126] In an optional implementation of this embodiment, selecting the first data entry and the second data entry from the data entry set may include:

[0127] According to the calculation time of each data entry in the data entry set, the total calculation time corresponding to the data entry set is calculated; according to the total calculation time, the time consumption proportion corresponding to each data entry is determined; according to the time consumption proportion of each data entry and the arrangement order of each data entry in the data entry set, the cumulative probability range corresponding to each data entry is calculated; two random numbers between [0, 1] are randomly generated, and when it is determined that each random number falls into a different cumulative probability range, the first data entry and the second data entry corresponding to the cumulative probability range that falls into are obtained.

[0128] The total computation time may be obtained by summing the computation time of each data item. The computation time proportion may be the proportion of the computation time of each data item to the total computation time. The arrangement order may be the order in which the data items are generated. The cumulative probability range may be calculated based on the computation time proportion of each data item.

[0129] Specifically, the computation time of each data entry can be summed to obtain the total computation time of the data entry set, thereby calculating the proportion of the computation time of each data entry in the total computation time. Based on the proportion and the order of each data entry in the data entry set, the cumulative probability range of each data entry is calculated, and two random numbers between [0, 1] are generated. When it is determined that the two random numbers fall into different cumulative probability ranges, the first data entry and the second data entry corresponding to the cumulative probability ranges are obtained. Correspondingly, if the two generated random numbers fall into the same cumulative probability range, a new random number can be generated to obtain the first data entry and the second data entry.

[0130] For example, there are 3 data items in the data item set, and the order of arrangement is data item 1, data item 2 and data item 3. The calculation time of data item 1 is 1 minute, and the calculation time of data item 2 and data item 3 are both 2 minutes. The total calculation time of the data item set is 5 minutes. Then the time consumption proportion of data item 1 is The time consumption ratios of data entry 2 and data entry 3 are both , therefore, the cumulative probability range corresponding to data entry 1 is [ , ], the cumulative probability range corresponding to data entry 2 is ( , ], the cumulative probability range corresponding to data entry 3 is ( , The two random numbers generated are 0.2 and 0.7, which fall in [ , ]and( , ], the first data entry and the second data entry may be data entry 1 and data entry 3, respectively.

[0131] S305 : Combining the first data entry and the second data entry to obtain a cross data entry that satisfies a validation condition for an input-output connection relationship.

[0132] In an optional implementation of this embodiment, combining the first data entry and the second data entry to obtain a cross data entry that satisfies the input-output connection relationship validity check condition may include:

[0133] A crossover operator is selected from each target computing operator included in a data entry; a first entry portion and a second entry portion are obtained from a first data entry and a second data entry respectively according to the position of the crossover operator in the data entry; the crossover operator is the last operator in the first entry portion or the first operator in the second entry portion; the first entry portion and the second entry portion are combined to obtain an alternative crossover entry, and the crossover operator is obtained from the alternative crossover entry; an adjacent operator matching the crossover operator is obtained according to the topological sorting of each computing operator in a computation graph; it is verified whether the crossover operator and the matched adjacent operator meet the input-output connection relationship validity check condition; if so, the alternative crossover entry is determined as the crossover data entry; otherwise, the operation of selecting the crossover operator from each target computing operator included in the data entry is returned to be executed until an end selection condition is met; when the end selection condition is met, the first data entry or the second data entry is determined as the crossover data entry.

[0134] The first entry portion may be the portion of the first data entry from the first target calculation to the crossover operator. The second entry portion may be the portion of the second data entry from the crossover operator to the last target calculation operator. An alternative crossover entry may be an entry obtained by combining the first and second entry portions, where the validity of the input-output connection relationship has not been verified.

[0135] In this embodiment, a crossover operator can be selected from each target operator included in the data entry. Based on the position of the crossover operator in the data entry, the portion of the first data entry from the first target operator to the crossover operator is used as the first entry portion, and the portion of the second data entry from the crossover operator to the last target operator is used as the second entry portion. The first entry portion and the second entry portion are then combined to obtain a candidate crossover entry. Since the candidate crossover entry is obtained by splicing and combining, the validity of the input-output connection relationship between the two adjacent operators at the splicing point can be verified. The crossover operator can be obtained from the candidate crossover entry, and based on the topological sorting of the operators in the computation graph, the adjacent operators that match the crossover operator are obtained to perform input-output connection validity verification. If this verification condition is met, the candidate crossover entry can be determined as the crossover data entry; otherwise, the operation of selecting the crossover operator from each target operator included in the data entry can be returned until the end selection condition is met. When the end selection condition is met, the first data entry or the second data entry is determined as the crossover data entry.

[0136] For example, Figure 3b This is a specific application scenario for obtaining cross-data entries provided by the third embodiment of the present invention.

[0137] 1) Assuming that the number of data entries in the data entry set is N, calculate the time consumption proportion of each data entry based on the computation time corresponding to each data entry and the total computation time of the data entry set. And calculate the cumulative probability range of each data entry according to the order of each data entry in the data entry set.

[0138] 2) Generate two random numbers p1 and p2 between [0, 1], see in which cumulative probability range p1 and p2 fall respectively, and take out the data entries corresponding to p1 and p2 respectively, abbreviated as data entry m and data entry n.

[0139] 3) When p1 and p2 fall into different cumulative probability ranges, that is, data entry m and data entry _n are different, obtain the target number of calculation operators in the data entry, assuming it is M, and generate a random integer k in [0, M) as the crossover position.

[0140] 4) Combine the portion of the target calculation operators from the 0th to the kth in data entry m and the portion of the target calculation operators from the kth to the last in data entry n into a new data entry k, and generate the corresponding operator timing graph.

[0141] 5) Perform a validity check on the input-output connection relationship of the operator at the intersection of data entry k. If valid, continue with the subsequent processing; otherwise, re-intersect until it is legal.

[0142] S306: Calculate the computation time corresponding to the intersection data entry, and update the data entry set according to the computation time of each data entry in the data entry set and the intersection data entry.

[0143] In an optional implementation of this embodiment, before calculating the computation time corresponding to the intersection data entry, the following steps may be further included:

[0144] When it is detected that the random replacement condition is met, a replacement operator is selected from the cross data entry; and the target tensor segmentation pattern of the replacement operator is replaced to meet the input and output validity conditions and the input and output connection relationship validity verification conditions.

[0145] The replacement operator may be a randomly selected target computing operator that replaces the target tensor segmentation pattern that has been determined.

[0146] Detecting that the random replacement condition is met may include: randomly generating a target random number, and comparing the target random number with a preset variation threshold; if it is determined that the target random number is greater than or equal to the variation threshold, determining that the random replacement condition is met.

[0147] The target random number may be a value between (0, 1). The preset mutation threshold may be a value between (0, 1). If the generated target random number is greater than or equal to the mutation threshold, then the random replacement condition is determined to be satisfied.

[0148] Optionally, replacing the target tensor splitting mode of the replacement operator to satisfy input and output validity conditions and input and output connection relationship validity verification conditions may include:

[0149] Obtain an optional tensor splitting mode corresponding to the target tensor splitting mode of the replacement operator as a comparison tensor splitting mode; select a replacement tensor splitting mode that is different from the comparison tensor splitting mode from the set of optional tensor splitting modes corresponding to the replacement operator stored in the operator information library; derive a replacement output splitting state corresponding to the replacement input splitting state based on the replacement input splitting state and the replacement reference output splitting state in the replacement tensor splitting mode; verify whether the replacement input splitting state and the replacement output splitting state meet the input and output validity conditions corresponding to the replacement operator; if so, obtain an adjacent operator that matches the replacement operator based on the topological sorting of each computing operator in the computational graph; verify whether the replacement operator and the matched adjacent operator meet the input and output connection relationship validity verification conditions based on the replacement input splitting state and the replacement output splitting state; if so, use the replacement input splitting state and the replacement output splitting state to replace the target tensor splitting mode of the replacement operator.

[0150] The comparison tensor splitting mode is an optional tensor splitting mode corresponding to the target tensor splitting mode. That is, it is obtained from the set of optional tensor splitting modes corresponding to the replacement operator and is used to finally calculate the target tensor splitting mode. In other words, the input tensor splitting state in the comparison tensor splitting mode is consistent with the input tensor splitting state in the target tensor splitting mode.

[0151] The replacement tensor splitting mode can be obtained from the optional tensor splitting mode set of the replacement operator, which is different from the comparison tensor splitting mode, a tensor splitting mode.

[0152] Exemplarily, a target computing operator is selected as a replacement operator, and the optional tensor splitting mode set stored in the operator information library for the replacement operator includes tensor splitting mode 1, tensor splitting mode 2, tensor splitting mode 3, and tensor splitting mode 4. If the optional tensor splitting mode corresponding to the target tensor splitting mode of the replacement operator is tensor splitting mode 1 (i.e., the comparison tensor splitting mode), then the replacement tensor splitting mode corresponding to the replacement operator is any one of splitting mode 2, splitting mode 3, and splitting mode 4.

[0153] For example, Figure 3cThis is a specific application scenario of performing a random replacement operation on provided by the third embodiment of the present invention.

[0154] 1) Given a mutation threshold G, the value of G is between (0, 1).

[0155] 2) Generate a random number p between 0 and 1 and compare it with G; if p is higher than G, proceed to the next step; otherwise, skip this step.

[0156] 3) Get the target number of operators in the data entry, assuming it is K. Generate a random integer j in [0, K) as the mutation position.

[0157] 4) Extract the target operator corresponding to j as the replacement operator and query all optional tensor splitting modes for that operator in the operator information database. Assume that the total number of optional tensor splitting modes is N. Randomly generate a random number between [0, N) as a new replacement splitting mode. This replacement splitting mode is different from the comparison tensor splitting mode currently corresponding to the target operator.

[0158] 5) According to the replacement input tensor splitting mode in the replacement splitting mode in (4) and the replacement reference output splitting state, the tensor state derivation function is called to derive the replacement output splitting state of the replacement operator.

[0159] 6) Based on the topological sorting of the operators in the computation graph, the input-output connection relationship is verified to check the validity of the random replacement result. If it is not legal, the random replacement process is terminated. If it is legal, the replacement operator information at position j in the data entry is refreshed, and the operator time consumption graph of the cross-data entry after the replacement operator is replaced is updated.

[0160] S307 : Return to executing the operation of selecting the first data entry and the second data entry in the data entry set until the update end condition is met.

[0161] S308. Obtain a target data entry from the updated data entry set, and determine a tensor splitting mode for each computing operator in the computation graph based on the target data entry.

[0162] The technical solution of the embodiment of the present invention obtains a computational graph to be tensor split including multiple computational operators, determines a data entry set corresponding to the computational graph according to a set of optional tensor splitting modes corresponding to the computational operators, calculates the computational time corresponding to each data entry, selects a first data entry and a second data entry from the data entry set, combines the first data entry and the second data entry to obtain a full cross-data entry, calculates the computational time corresponding to the cross-data entry, and updates the data entry set according to the computational time of each data entry and the cross-data entry in the data entry set, returns to execute the operation of selecting the first data entry and the second data entry in the data entry set until the update end condition is met, obtains the target data entry in the updated data entry set, and determines the tensor splitting mode of each computational operator in the computational graph according to the target data entry. The technical solution solves the problem that the prior art cannot efficiently determine the optimal splitting method of the operator in complex scenarios, and provides a method that is applicable to complex scenarios and can efficiently determine the optimal splitting mode of the operator.

[0163] Example 4

[0164] Figure 4 This is a schematic diagram of the structure of a device for determining a tensor splitting mode provided in the fourth embodiment of the present invention. The device can be used to perform the method for determining a tensor splitting mode provided in any embodiment of the present invention. The device includes: a computation graph acquisition module 401, a data entry set determination module 402, a computation time determination module 403, a data entry set update module 404, and a tensor splitting mode determination module 405. Among them:

[0165] A computation graph acquisition module 401 is used to acquire a computation graph to be subjected to tensor segmentation, wherein the computation graph includes multiple computation operators.

[0166] A data entry set determination module 402 is configured to determine a data entry set corresponding to the computation graph based on a set of optional tensor splitting modes corresponding to the computation operators, wherein the data entry includes a plurality of target computation operators and target tensor splitting modes for the target computation operators;

[0167] A computation time determination module 403 is configured to calculate the computation time corresponding to each data entry according to each target tensor segmentation mode included in each data entry;

[0168] A data entry set updating module 404 is configured to generate a plurality of cross data entries based on a plurality of data entry pairs selected from the data entry set, and update the data entry set based on a calculation time of each cross data entry;

[0169] The tensor splitting mode determination module 405 is used to obtain a target data entry from the updated data entry set, and determine the tensor splitting mode of each computing operator in the computation graph according to the target data entry.

[0170] The technical solution of the embodiment of the present invention obtains a computational graph including multiple computational operators to be tensor split; determines a data entry set including multiple target computational operators and target tensor splitting modes of the target computational operators in each data entry corresponding to the computational graph according to an optional tensor splitting mode set corresponding to the computational operators; calculates the computational time corresponding to each data entry according to each target tensor splitting mode included in each data entry; generates multiple cross data entries according to multiple data entry pairs selected from the data entry set, and updates the data entry set according to the computational time of each cross data entry; obtains the target data entry in the updated data entry set, and determines the tensor splitting mode of each computational operator in the computational graph according to the target data entry. This technical solution solves the problem that the prior art cannot efficiently determine the optimal splitting method of the operator in complex scenarios, and provides a method that is applicable to complex scenarios and can efficiently determine the optimal splitting mode of the operator.

[0171] In the above apparatus, optionally, the data entry set determination module 402 may include:

[0172] The computing operator topological sorting unit is used to perform topological sorting on each computing operator in the computing graph;

[0173] The current calculation operator acquisition unit is used to create a data entry template and obtain a current calculation operator in sequence according to the sorting order;

[0174] a target computing operator determining unit, configured to query an operator information library according to an operator type of the current computing operator, and when determining that an optional tensor splitting mode set corresponding to the current computing operator is stored in the operator information library, determine the current computing operator as the target computing operator;

[0175] a data entry template adding unit, configured to determine a target tensor splitting mode of a target computing operator according to an optional tensor splitting mode set of the target computing operator, and add the target computing operator and the target tensor splitting mode to the data entry template;

[0176] A data entry generation unit is used to return and execute the operation of sequentially obtaining a current calculation operator in the sorted order until all calculation operators in the calculation graph are processed to generate a data entry;

[0177] The data entry set forming unit is used to repeatedly execute the process from creating a data entry template to generating data entries until a set number of data entries are generated to form the data entry set.

[0178] In the above device, optionally, the data entry template adding unit may include:

[0179] The alternative tensor splitting mode selection subunit is used to select an alternative tensor splitting mode from the optional tensor splitting mode set of the target computing operator. The optional tensor splitting mode includes: the input splitting state of each input terminal and the reference output splitting state of each output terminal;

[0180] a target output split state derivation subunit, configured to derive a target output split state corresponding to the alternative input split state according to the alternative input split state in the alternative tensor split mode and the alternative reference output split state;

[0181] The segmentation state verification subunit is used to verify whether the candidate input segmentation state and the target output segmentation state meet the input and output validity conditions corresponding to the target calculation operator;

[0182] a target tensor splitting mode determining subunit, configured to, if yes, determine the candidate input splitting state and the target output splitting state as the target tensor splitting mode of the target computing operator;

[0183] The target tensor splitting mode is successfully determined by the sub-unit, and otherwise, returns to execute the operation of selecting an alternative tensor splitting mode from the optional tensor splitting mode set of the target computing operator until the target tensor splitting mode of the target computing operator is successfully determined.

[0184] In the above device, optionally, the alternative tensor splitting mode selection subunit can be specifically used to:

[0185] Detecting whether the target calculation operator has a previous adjacent operator;

[0186] If so, obtain the target output split state in the target tensor split mode determined for the previous adjacent operator;

[0187] According to the target output splitting state of the previous adjacent operator, in the optional tensor splitting mode set of the target computing operator, the optional tensor splitting mode that meets the input-output connection relationship validity verification condition with the previous adjacent operator is screened out; among the selected optional tensor splitting modes, the alternative tensor splitting mode is selected.

[0188] The above apparatus may optionally further include a data entry discarding unit configured to: after completing processing of all computing operators in the computation graph to generate a data entry;

[0189] Based on the generated data entries and the topological sorting of the operators in the computation graph, verify whether any two adjacent operators meet the input-output connection validity check conditions;

[0190] If it is determined that any two adjacent computing operators do not meet the input-output connection relationship validity verification condition, the data entry is discarded.

[0191] In the above device, optionally, the calculation time determination module 403 may include:

[0192] The original operator time consumption graph acquisition unit is used to obtain the current processing entry from all data entries and obtain the original operator time consumption graph corresponding to the calculation graph. The original operator time consumption graph has the same topological structure as the operator in the calculation graph.

[0193] A target operator time consumption graph generating unit is used to add the splitting operators corresponding to the target tensor splitting modes of each target operator in the current processing entry to the original operator time consumption graph, so as to generate the target operator time consumption graph corresponding to the current processing entry;

[0194] The computation time calculation unit is used to query a pre-built time consumption model library and calculate the computation time corresponding to the target operator time consumption graph.

[0195] In the above device, optionally, the target operator time consumption graph generating unit may be specifically used to:

[0196] In the original operator time-consuming graph, add the splitting operators corresponding to the target tensor splitting modes of each target operator in the current processing entry;

[0197] The added split operators are merged and / or cancelled to generate a target operator time consumption graph corresponding to the current processing entry.

[0198] In the above apparatus, optionally, the data entry set updating module 404 may include:

[0199] a first and a second data entry selection unit, configured to select a first data entry and a second data entry from the data entry set;

[0200] a cross data entry obtaining unit, configured to obtain, based on the first data entry and the second data entry, a cross data entry that satisfies a validity check condition for an input-output connection relationship;

[0201] a data entry set updating unit, configured to calculate a computation time corresponding to the intersection data entry, and update the data entry set according to the computation time of each data entry in the data entry set and the intersection data entry;

[0202] The update end condition satisfaction unit is used to return to executing the operation of selecting the first data entry and the second data entry in the data entry set until the update end condition is satisfied.

[0203] In the above device, optionally, the first and second data item selection units may be specifically configured to:

[0204] Calculating a total computation time corresponding to the data entry set based on the computation time of each data entry in the data entry set;

[0205] Determining the time consumption proportion corresponding to each of the data items according to the total calculation time consumption;

[0206] Calculating a cumulative probability range corresponding to each data entry based on a time consumption ratio of each data entry and an arrangement order of each data entry in the data entry set;

[0207] Two random numbers between [0, 1] are randomly generated, and when it is determined that each random number falls into a different cumulative probability range, a first data entry and a second data entry corresponding to the cumulative probability range that falls into are obtained.

[0208] In the above device, optionally, the cross data entry obtaining unit may be specifically configured to:

[0209] Selecting a crossover operator from among target calculation operators included in the data entry;

[0210] Obtaining a first entry portion and a second entry portion in the first data entry and the second data entry respectively according to the position of the crossover operator in the data entry; the crossover operator is the last operator in the first entry portion or the first operator in the second entry portion;

[0211] Combining the first entry portion and the second entry portion to obtain a candidate crossover entry, and obtaining a crossover operator from the candidate crossover entry;

[0212] According to the topological sorting of each computing operator in the computation graph, obtain the adjacent operators that match the crossover operator;

[0213] Verify whether the crossover operator and the matched adjacent operator meet the input-output connection relationship validity verification conditions;

[0214] If so, the candidate cross entry is determined as the cross data entry; otherwise, the operation of selecting a cross operator from each target calculation operator included in the data entry is returned to be executed until the end selection condition is met;

[0215] When the end selection condition is met, the first data entry or the second data entry is determined as the intersection data entry.

[0216] The above device may optionally further include a target tensor splitting mode replacing unit, configured to: before calculating the computation time corresponding to the cross data entry:

[0217] a replacement operator selection subunit, configured to select a replacement operator from the cross data entry when it is detected that a random replacement condition is met;

[0218] The target tensor splitting mode replacement subunit is used to replace the target tensor splitting mode of the replacement operator to meet the input and output validity conditions and the input and output connection relationship validity verification conditions.

[0219] In the above device, optionally, detecting that the random replacement condition is satisfied includes:

[0220] Randomly generate a target random number, and compare the target random number with a preset variation threshold;

[0221] If it is determined that the target random number is greater than or equal to the variation threshold, it is determined that the random replacement condition is met.

[0222] In the above device, optionally, the target tensor splitting mode replacement subunit may be specifically used to:

[0223] Get the optional tensor splitting mode corresponding to the target tensor splitting mode of the replacement operator as the comparison tensor splitting mode;

[0224] Selecting a replacement tensor splitting pattern that is different from and compared with the tensor splitting pattern from a set of optional tensor splitting patterns corresponding to the replacement operator stored in the operator information library;

[0225] According to the replacement input split state in the replacement tensor split mode and the replacement reference output split state, a replacement output split state corresponding to the replacement input split state is derived;

[0226] Verify whether the replacement input segmentation state and replacement output segmentation state meet the input and output validity conditions corresponding to the replacement operator;

[0227] If so, obtain the adjacent operator that matches the replacement operator based on the topological sorting of the operators in the computation graph;

[0228] Verifying, based on the replacement input segmentation state and the replacement output segmentation state, whether the replacement operator and the matched adjacent operator satisfy the input-output connection relationship validity verification condition;

[0229] If so, the target tensor splitting mode of the replacement operator is replaced using the replacement input splitting state and the replacement output splitting state.

[0230] The apparatus for determining a tensor splitting mode provided in an embodiment of the present invention can execute the method for determining a tensor splitting mode provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0231] Example 5

[0232] Figure 5 A schematic diagram of the structure of a computer device provided in the fifth embodiment of the present invention is shown in FIG. Figure 5 As shown, the computer device includes a processor 501, a memory 502, an input device 503 and an output device 504; the number of processors 501 in the computer device can be one or more. Figure 5 In the figure, a processor 501 is taken as an example; the processor 501, memory 502, input device 503 and output device 504 in the computer device can be connected by a bus or other means. Figure 5 The bus connection is taken as an example.

[0233] Memory 502, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the method for determining a tensor splitting mode in embodiments of the present invention (e.g., the computation graph acquisition module 401, the data entry set determination module 402, the computation time determination module 403, the data entry set update module 404, and the tensor splitting mode determination module 405 in the apparatus for determining a tensor splitting mode). Processor 501 executes the software programs, instructions, and modules stored in memory 502 to execute various functional applications and data processing of the computer device, thereby implementing the aforementioned method for determining a tensor splitting mode. This method may include:

[0234] Get the computational graph to be split, which includes multiple computational operators.

[0235] Determine a data entry set corresponding to the computation graph according to an optional tensor splitting mode set corresponding to the computation operator, wherein the data entry includes a plurality of target computation operators and target tensor splitting modes of the target computation operators;

[0236] Calculating the computation time corresponding to each data entry according to each target tensor segmentation mode included in each data entry;

[0237] generating a plurality of cross data entries according to a plurality of data entry pairs selected from the data entry set, and updating the data entry set according to a computation time of each cross data entry;

[0238] A target data entry is obtained from the updated data entry set, and a tensor splitting mode of each computing operator in the computation graph is determined based on the target data entry.

[0239] The memory 502 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the terminal, etc. Furthermore, the memory 502 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some instances, the memory 502 may further include memory remotely located relative to the processor 501, and these remote memories may be connected to the computer device via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0240] The input device 503 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the computer device. The output device 504 may include a display device such as a display screen.

[0241] Example 6

[0242] Embodiment 6 of the present invention further provides a storage medium containing computer-executable instructions. When the computer-executable instructions are executed by a computer processor, the computer-executable instructions are used to perform a method for determining a tensor splitting mode. The method includes:

[0243] Get the computational graph to be split, which includes multiple computational operators.

[0244] Determine a data entry set corresponding to the computation graph according to an optional tensor splitting mode set corresponding to the computation operator, wherein the data entry includes a plurality of target computation operators and target tensor splitting modes of the target computation operators;

[0245] Calculating the computation time corresponding to each data entry according to each target tensor segmentation mode included in each data entry;

[0246] generating a plurality of cross data entries according to a plurality of data entry pairs selected from the data entry set, and updating the data entry set according to a computation time of each cross data entry;

[0247] A target data entry is obtained from the updated data entry set, and a tensor splitting mode of each computing operator in the computation graph is determined based on the target data entry.

[0248] Of course, the storage medium containing computer-executable instructions provided in an embodiment of the present invention is not limited to the method operations described above, and can also execute related operations in the method for determining the tensor splitting mode provided in any embodiment of the present invention.

[0249] Through the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented with the help of software and necessary general-purpose hardware. Of course, it can also be implemented with hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the existing technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0250] It is worth noting that in the embodiment of the above-mentioned device for determining the tensor splitting pattern, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0251] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for determining a tensor segmentation mode, characterized in that: include: Get the computational graph to be split, which includes multiple computational operators. Determine a data entry set corresponding to the computation graph according to an optional tensor splitting mode set corresponding to the computation operator, wherein the data entry includes a plurality of target computation operators and target tensor splitting modes of the target computation operators; Calculating the computation time corresponding to each data entry according to each target tensor segmentation mode included in each data entry; generating a plurality of cross data entries according to a plurality of data entry pairs selected from the data entry set, and updating the data entry set according to a computation time of each cross data entry; A target data entry is obtained from the updated data entry set, and a tensor splitting mode of each computing operator in the computation graph is determined based on the target data entry.

2. The method according to claim 1, characterized in that According to the optional tensor splitting mode set corresponding to the computation operator, the data entry set corresponding to the computation graph is determined, including: Perform topological sorting on each operator in the computation graph; Create a data entry template and obtain a current calculation operator in sequence according to the sort order; According to the operator type of the current computing operator, the operator information library is queried, and when it is determined that the operator information library stores an optional tensor splitting mode set corresponding to the current computing operator, the current computing operator is determined as the target computing operator; Determine a target tensor splitting mode of the target computing operator according to the optional tensor splitting mode set of the target computing operator, and add the target computing operator and the target tensor splitting mode to the data entry template; Return and execute the operation of obtaining one current operator in sequence according to the sort order until all operators in the computation graph are processed to generate a data entry; The process from creating a data entry template to generating data entries is repeatedly executed until a set number of data entries are generated to form the data entry set.

3. The method according to claim 2, characterized in that Determine the target tensor splitting mode of the target computing operator based on the optional tensor splitting mode set of the target computing operator, including: Selecting an alternative tensor splitting mode from an optional tensor splitting mode set of a target computing operator, wherein the optional tensor splitting mode includes: an input splitting state of each input terminal and a reference output splitting state of each output terminal; According to the alternative input split state in the alternative tensor split mode and the alternative reference output split state, a target output split state corresponding to the alternative input split state is derived; Verify whether the candidate input segmentation status and target output segmentation status meet the input and output validity conditions corresponding to the target calculation operator; If so, the candidate input split state and the target output split state are determined as the target tensor split mode of the target computing operator; Otherwise, the operation of selecting an alternative tensor splitting mode from the optional tensor splitting mode set of the target computing operator is returned to execution until the target tensor splitting mode of the target computing operator is successfully determined.

4. The method according to claim 3, characterized in that Select an alternative tensor splitting mode from the optional tensor splitting mode set of the target computing operator, including: Detecting whether the target calculation operator has a previous adjacent operator; If so, obtain the target output split state in the target tensor split mode determined for the previous adjacent operator; According to the target output splitting state of the previous adjacent operator, in the optional tensor splitting mode set of the target computing operator, the optional tensor splitting mode that meets the input-output connection relationship validity verification condition with the previous adjacent operator is screened out; among the selected optional tensor splitting modes, the alternative tensor splitting mode is selected.

5. The method according to claim 2, characterized in that After completing the processing of all computational operators in the computational graph to generate a data entry, it also includes: Based on the generated data entries and the topological sorting of the operators in the computation graph, verify whether any two adjacent operators meet the input-output connection validity check conditions; If it is determined that any two adjacent computing operators do not meet the input-output connection relationship validity verification condition, the data entry is discarded.

6. The method according to claim 1, characterized in that Calculating the computation time corresponding to each data entry according to each target tensor segmentation mode included in each data entry includes: Get the current processing entry from all data entries, and get the original operator time consumption graph corresponding to the computation graph. The original operator time consumption graph has the same topological structure as the operator in the computation graph. In the original operator time-consuming graph, the splitting operators corresponding to the target tensor splitting modes of each target operator in the current processing entry are added to generate the target operator time-consuming graph corresponding to the current processing entry; A pre-built time-consuming model library is queried to calculate the computation time corresponding to the target operator time-consuming graph.

7. The method according to claim 6, characterized in that In the original operator time-consuming graph, add the splitting operators corresponding to the target tensor splitting modes of each target operator in the current processing entry to generate the target operator time-consuming graph corresponding to the current processing entry, including: In the original operator time-consuming graph, add the splitting operators corresponding to the target tensor splitting modes of each target operator in the current processing entry; The added split operators are merged and / or cancelled to generate a target operator time consumption graph corresponding to the current processing entry.

8. The method according to claim 1, characterized in that Combining a plurality of data entry pairs selected from a data entry set to obtain a plurality of cross data entries, and updating the data entry set according to a calculation time consumption of each cross data entry, including: selecting a first data entry and a second data entry from the set of data entries; Combining the first data entry and the second data entry to obtain a cross data entry that satisfies a validation condition for an input-output connection relationship; Calculating the computation time corresponding to the intersecting data entry, and updating the data entry set according to the computation time of each data entry in the data entry set and the intersecting data entry; Return to executing the operation of selecting the first data entry and the second data entry in the data entry set until the update end condition is met.

9. The method according to claim 8, characterized in that Selecting a first data entry and a second data entry from the data entry set includes: Calculating a total computation time corresponding to the data entry set based on the computation time of each data entry in the data entry set; Determining the time consumption proportion corresponding to each of the data items according to the total calculation time consumption; Calculating a cumulative probability range corresponding to each data entry based on a time consumption ratio of each data entry and an arrangement order of each data entry in the data entry set; Two random numbers between [0, 1] are randomly generated, and when it is determined that each random number falls into a different cumulative probability range, a first data entry and a second data entry corresponding to the cumulative probability range that falls into are obtained.

10. The method according to claim 8, characterized in that Combining the first data entry and the second data entry to obtain a cross data entry that satisfies a validation condition for an input-output connection relationship includes: Selecting a crossover operator from among target calculation operators included in the data entry; Obtaining a first entry portion and a second entry portion from the first data entry and the second data entry respectively according to the position of the crossover operator in the data entry; the crossover operator is the last operator in the first entry portion or the first operator in the second entry portion; Combining the first entry portion and the second entry portion to obtain a candidate crossover entry, and obtaining a crossover operator from the candidate crossover entry; According to the topological sorting of each computing operator in the computation graph, obtain the adjacent operators that match the crossover operator; Verify whether the crossover operator and the matched adjacent operator meet the input-output connection relationship validity verification conditions; If so, the candidate cross entry is determined as the cross data entry; otherwise, the operation of selecting a cross operator from each target calculation operator included in the data entry is returned to be executed until the end selection condition is met; When the end selection condition is met, the first data entry or the second data entry is determined as the intersection data entry.

11. The method according to claim 8, characterized in that Before calculating the computation time corresponding to the intersection data entry, the method further includes: When it is detected that the random replacement condition is met, selecting a replacement operator in the cross data entry; The target tensor segmentation mode of the replacement operator is replaced to meet the input and output validity conditions and the input and output connection relationship validity verification conditions.

12. The method according to claim 11, characterized in that The random replacement conditions are detected to be met including: Randomly generate a target random number, and compare the target random number with a preset variation threshold; If it is determined that the target random number is greater than or equal to the variation threshold, it is determined that the random replacement condition is met.

13. The method according to claim 11, characterized in that Replacing the target tensor splitting mode of the replacement operator to meet input and output validity conditions and input and output connection relationship validity verification conditions includes: Get the optional tensor splitting mode corresponding to the target tensor splitting mode of the replacement operator as the comparison tensor splitting mode; Selecting a replacement tensor splitting pattern that is different from and compared with the tensor splitting pattern from a set of optional tensor splitting patterns corresponding to the replacement operator stored in the operator information library; According to the replacement input split state in the replacement tensor split mode and the replacement reference output split state, a replacement output split state corresponding to the replacement input split state is derived; Verify whether the replacement input segmentation state and replacement output segmentation state meet the input and output validity conditions corresponding to the replacement operator; If so, obtain the adjacent operator that matches the replacement operator based on the topological sorting of the operators in the computation graph; Verifying, based on the replacement input segmentation state and the replacement output segmentation state, whether the replacement operator and the matched adjacent operator satisfy the input-output connection relationship validity verification condition; If so, the target tensor splitting mode of the replacement operator is replaced using the replacement input splitting state and the replacement output splitting state.

14. A device for determining a tensor segmentation mode, characterized in that: The device comprises: The computation graph acquisition module is used to obtain the computation graph to be split into tensors, which includes multiple computation operators. A data entry set determination module is used to determine a data entry set corresponding to the computation graph based on an optional tensor splitting mode set corresponding to the computation operator, wherein the data entry includes multiple target computation operators and target tensor splitting modes of the target computation operators; A computation time determination module, configured to calculate the computation time corresponding to each data entry according to each target tensor segmentation mode included in each data entry; a data entry set updating module, configured to generate a plurality of cross data entries based on a plurality of data entry pairs selected from the data entry set, and update the data entry set based on a calculation time of each cross data entry; The tensor splitting mode determination module is used to obtain the target data entry in the updated data entry set, and determine the tensor splitting mode of each computing operator in the calculation graph according to the target data entry.

15. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for determining the tensor segmentation mode according to any one of claims 1 to 13 is implemented.

16. A computer executable instruction storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for determining a tensor segmentation mode as described in any one of claims 1 to 13 is implemented.

Images