Tensor parallel processing method and apparatus
Patent Information
- Application Number
- CN202510484174.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-16
- Publication Date
- 2026-10-23
Smart Images
Figure CN122965645A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and more specifically, to a tensor parallel processing method and apparatus. Background Technology
[0002] Tensor parallelism is a technique for accelerating the training and inference of deep learning models by distributing large tensors across multiple computing devices. Currently, existing techniques mainly generate segmentation strategies by designing matching patterns for models with fixed structures and specifying the segmentation coefficients of each operator, or by searching for the optimal segmentation strategy using a breadth-first search algorithm. However, the former requires redesigning patterns for models with different structures, resulting in poor generality, while the latter requires searching and storing all segmentation strategies, leading to significant search time and space overhead, making it unsuitable for scenarios with a large number of operator segmentation strategies. Summary of the Invention
[0003] In view of this, the purpose of the present invention is to provide a tensor parallel processing method and apparatus to control the search space size of the partitioning strategy, thereby improving the search speed while ensuring a certain degree of search width and versatility.
[0004] In a first aspect, embodiments of the present invention provide a tensor parallel processing method, the method comprising:
[0005] In response to the existence of an operator to be searched in the computation graph, the current operator to be searched is determined, wherein the computation graph includes multiple operators and the connection relationships between the operators;
[0006] Determine at least one segmentation method for the current operator to be searched;
[0007] In response to the existence of operators to be searched in the computation graph, a corresponding number of target candidate strategies are determined from each of the segmentation strategies based on the propagation state of each segmentation method and the revenue parameters of the segmentation strategies formed by each segmentation method, and the next operator to be searched is determined. The corresponding number is less than or equal to a predetermined number. The segmentation strategy is determined based on the combination of each candidate strategy corresponding to the historical operators to be searched determined before the current operator to be searched and each of the segmentation methods.
[0008] In response to the completion of operator segmentation in the computation graph, a target strategy is determined from each of the target candidate strategies based on the payoff parameters corresponding to each target candidate strategy.
[0009] The tensor is segmented according to the target strategy and distributed to various computing devices for parallel processing.
[0010] Optionally, the revenue parameters corresponding to the partitioning strategy include a combination of one or more of the following: the revenue from hard weight occupancy of each computing device under the partitioning strategy, the revenue from ultra-large weight occupancy, the revenue from soft weight occupancy, and the revenue from inter-device communication.
[0011] The revenue from the space occupied by hard weights is determined based on a predetermined upper limit for space occupied and the total space occupied by weights after segmentation. The revenue from the ultra-large space weights is determined based on the number of weights exceeding the lower limit of the ultra-large weight space. The revenue from the space occupied by soft weights is determined based on the difference between the predetermined upper limit for space occupied and the total space occupied by weights after segmentation. The revenue from inter-device communication is determined based on the total communication volume of operators that need to perform device communication.
[0012] Optionally, determining a corresponding number of target candidate strategies from each of the segmentation strategies based on the propagation state of each segmentation method and the revenue parameters of the segmentation strategies formed by each segmentation method includes:
[0013] Traverse each of the aforementioned segmentation methods, propagate each of the aforementioned segmentation methods starting from the current operator to be searched based on the connection relationship, and obtain the propagation status of each of the aforementioned segmentation methods. The propagation status is used to indicate whether the segmentation method has completed propagation.
[0014] The segmentation methods that have completed propagation are combined with the candidate strategies corresponding to the historical search operators determined before the current search operator to obtain the segmentation strategy formed by the segmentation methods.
[0015] Obtain the revenue parameters for each of the aforementioned segmentation strategies;
[0016] Based on a predetermined search algorithm and the payoff parameters of each of the segmentation strategies, a corresponding number of target candidate strategies are determined from the segmentation strategies. The predetermined search algorithm is a beam search algorithm, a heuristic search algorithm, or a random beam search algorithm.
[0017] Optionally, the segmentation method includes a segmentation coefficient and a segmentation arrangement order, and the method further includes:
[0018] After the propagation of the segmentation method is completed, the segmentation coefficients and segmentation order of each operator in the current propagation path are summed using a hash algorithm according to the propagation order of the operators in the current propagation path to obtain the hash value corresponding to the segmentation method.
[0019] The splitting method filters out duplicates based on hash values.
[0020] Optionally, the method further includes:
[0021] When the propagation of the segmentation method is performed, if the existing segmentation method of the target tensor conflicts with the current segmentation method and there is no available propagation method, a rollback operation is performed. Here, the target tensor is the tensor of the segmentation method to be propagated in the current propagation process.
[0022] Optionally, the method further includes:
[0023] The segmentation strategy and its corresponding cost are input into a neural network model to obtain the revenue parameters corresponding to the segmentation strategy. The cost includes the memory, computing resources, and communication resources required by each computing device to execute the segmentation strategy.
[0024] In a second aspect, embodiments of the present invention provide a tensor parallel processing apparatus, the apparatus comprising:
[0025] An operator determination unit is configured to determine the current operator to be searched in response to the existence of an operator to be searched in a computation graph, wherein the computation graph includes multiple operators and the connection relationships between the operators;
[0026] The segmentation unit is configured to determine at least one segmentation method for the current operator to be searched;
[0027] The strategy filtering unit is configured to, in response to the existence of operators to be searched in the computation graph, determine a corresponding number of target candidate strategies from each of the segmentation strategies based on the propagation state of each segmentation method and the revenue parameters of the segmentation strategies formed by each segmentation method, and determine the next operator to be searched, wherein the corresponding number is less than or equal to a predetermined number, and the segmentation strategy is determined based on the combination of each candidate strategy corresponding to the historical operators to be searched determined before the current operator to be searched and each of the segmentation methods;
[0028] The strategy determination unit is configured to determine a target strategy from each of the target candidate strategies based on the payoff parameters corresponding to each of the target candidate strategies in response to the completion of operator segmentation in the computation graph.
[0029] The processing unit is configured to segment the tensor according to the target strategy and distribute it to each computing device for parallel processing.
[0030] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in the first aspect of the present invention.
[0031] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method described in the first aspect of the present invention.
[0032] Fifthly, embodiments of the present invention provide a computer program product that, when run on a computer, causes the computer to perform the method described in the first aspect of the present invention.
[0033] In this embodiment of the invention, by identifying the current operator to be searched in a computation graph containing operators to be searched, and identifying at least one segmentation method corresponding to the current operator to be searched, and in response to the existence of other operators to be searched in the computation graph, a corresponding number of target candidate strategies are determined from each segmentation strategy based on the propagation state of each segmentation strategy and the revenue parameters of the segmentation strategies formed by each segmentation strategy, and the next operator to be searched is determined. In response to the completion of operator segmentation in the computation graph, a target strategy is determined from each target candidate strategy based on the revenue parameters corresponding to each target candidate strategy. The tensor is segmented according to the target strategy and allocated to each computing device for parallel processing. Thus, this embodiment of the invention can control the search space size of the segmentation strategy, improving the search speed while ensuring a certain degree of search width and versatility. Attached Figure Description
[0034] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:
[0035] Figure 1 This is a flowchart of the tensor parallel processing method according to an embodiment of the present invention;
[0036] Figure 2 This is a flowchart of the rollback operation method according to an embodiment of the present invention;
[0037] Figure 3 This is a flowchart of a method for determining target candidate strategies according to an embodiment of the present invention;
[0038] Figure 4 This is a flowchart of the filtering and segmentation strategy method according to an embodiment of the present invention;
[0039] Figure 5 This is a flowchart of the method for searching a segmentation strategy based on the beam search algorithm according to an embodiment of the present invention;
[0040] Figure 6 This is a schematic diagram of the calculation diagram according to an embodiment of the present invention;
[0041] Figure 7 This is a schematic diagram of the beam search process according to an embodiment of the present invention;
[0042] Figure 8 This is a schematic diagram of a tensor parallel processing device according to an embodiment of the present invention;
[0043] Figure 9This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation
[0044] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the substance of the present application, well-known methods, processes, flows, elements, and circuits are not described in detail.
[0045] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.
[0046] Unless the context explicitly requires it, words such as "including" or "contains" throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".
[0047] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.
[0048] Tensors, or multidimensional arrays, are fundamental data structures in deep learning models, commonly used to represent parameters such as weights and activation values in neural network models. As the scale of deep learning models increases, a single computing device can no longer meet the computational demands. Tensor parallelism is needed to distribute large tensors across multiple computing devices with parallel processing capabilities to improve computational performance and efficiency. However, existing technologies suffer from poor versatility and high computational overhead, making them unsuitable for scenarios with numerous operator partitioning strategies. Therefore, this invention provides a tensor parallel processing method and apparatus that introduces payoff calculation to control the search width based on payoff, thereby improving search efficiency.
[0049] In this embodiment of the invention, deep learning models are mainly used as an application scenario for illustration. It should be understood that the method of this embodiment can be applied to any scenario that requires the use of tensor parallel technology, such as signal and image processing, data analysis, data modeling, etc., and this embodiment does not limit this. Optionally, the computing device can be a graphics processor, tensor processor, neural network processor, or other device with parallel processing capabilities.
[0050] Figure 1 This is a flowchart of a tensor parallel processing method according to an embodiment of the present invention. Figure 1 As shown, the tensor parallel processing method of this invention includes the following steps:
[0051] Step S110: In response to the existence of a searchable operator in the computation graph, determine the current searchable operator. Here, an operator is a function or transformation that maps elements in one set to another. Specifically, in the field of deep learning, an operator is the smallest executable unit in a neural network, and a tensor is the input and output object of an operator. Operators implement computational logic through operations on tensors. A computation graph is a directed acyclic graph, represented by nodes and edges. Specifically, nodes represent individual operators, and edges represent the connections between operators, indicating the transfer of tensors between operators. That is, a computation graph includes multiple operators and the connections between them.
[0052] In one optional implementation, determining the current operator to be searched includes selecting a divisible and undivided operator in the computation graph as the operator to be searched. Optionally, the operator to be searched can be defined by the user; for example, it can be a weight operator, or other available operators can be selected according to the model design. Here, a weight operator is an operator representing a constant, used to represent weights. When selecting the operator to be searched, the operator with the greatest impact on the computation graph can be selected based on the connection relationships between operators in the computation graph. For example, the operator with the longest or most numerous connection chains can be prioritized as the operator to be searched. Alternatively, based on dependencies, operators that generate large tensors or key dependencies in the computation graph can be selected as the starting point. Or, selection can be based on memory optimization, communication optimization, etc., the specific selection methods are not elaborated here. It should be understood that the aforementioned divisible and undivided refers to the entire computation graph; undivided means that the operator has not been divided in all previous division strategies.
[0053] Step S120: Determine at least one segmentation method corresponding to the current operator to be searched.
[0054] Specifically, at least one segmentation method corresponding to the operator to be searched can be determined as follows: Based on the shape of the current operator to be searched and the number of computing devices, generate all non-repeating factor permutations and combinations. These factor permutations and combinations constitute the segmentation method of the current operator to be searched. The length of each factor permutation and combination does not exceed the order of the operator to be searched, and the factor corresponding to the shape position is divisible by the size of the corresponding shape position. For example, if the shape of the operator to be searched is (2, 8) and the number of devices is 4, then the order of the operator to be searched is 2, and the length of each factor permutation and combination does not exceed 2. The factors that divide 2 are 1 and 2, and the factors that divide 8 are 1, 2, 4 and 8. There are a total of 8 permutations: (1,1), (1,2), (1,4), (1,8), (2,1), (2,2), (2,4), and (2,8). Among them, the product of the factors (1,8), (2,4), and (2,8) exceeds the number of devices. Therefore, there are a total of 5 permutations of factors: (1,1), (1,2), (1,4), (2,1), and (2,2). These 5 permutations can be divided in five ways.
[0055] Step S130: In response to the existence of operators to be searched in the computation graph, a corresponding number of target candidate strategies are determined from each segmentation strategy based on the propagation state of each segmentation method and the revenue parameters of the segmentation strategies formed by each segmentation method, and the next operator to be searched is determined. The corresponding number is less than or equal to a predetermined number, and the segmentation strategy is determined based on the combination of candidate strategies and segmentation methods corresponding to the operators to be searched determined before the current operator to be searched. As mentioned above, the next operator to be searched is determined based on the entire computation graph, rather than as a subsequent step of a specific segmentation strategy.
[0056] In one optional implementation, the number of target candidate strategies is determined from each segmentation strategy based on the propagation state of each segmentation method and the revenue parameters of the segmentation strategies formed by each segmentation method. This can be achieved by first traversing each segmentation method and completing the propagation of each segmentation method, then selecting the segmentation strategies formed by each segmentation method that can complete the propagation in the propagation path corresponding to the current search operator, and then determining the number of target candidate strategies based on the revenue parameters corresponding to each selected segmentation strategy.
[0057] In another optional implementation, the number of target candidate strategies is determined from each segmentation strategy based on the propagation state of each segmentation method and the revenue parameters of the segmentation strategies formed by each segmentation method. Alternatively, each segmentation strategy can be sorted based on its revenue parameters, and the segmentation methods in each segmentation strategy can be propagated one by one according to the sorting. The segmentation strategies that have completed propagation are taken as target candidate strategies until the number of target candidate strategies reaches the corresponding number (e.g., the number of target candidate strategies reaches a predetermined number, or the number of target candidate strategies has not reached the predetermined number, but there are no more usable segmentation strategies).
[0058] The traversal of each segmentation method can specifically include: traversing each segmentation method sequentially according to a default order, or randomly selecting segmentation methods for traversal. The at least one segmentation method corresponding to the operator to be searched can be all segmentation methods executable by that operator or a subset of segmentation methods filtered based on predetermined conditions. These predetermined conditions can be flexibly adjusted according to actual needs, such as a predetermined number of segments.
[0059] The propagation methods of the segmentation can be divided into two types: intra-operator propagation and inter-operator propagation.
[0060] Inter-operator propagation refers to the transmission of the same tensor between two operators with a producer-consumer relationship. The relationship between the axes of this tensor remains unchanged during the transmission. The direction of propagation of the splitting method is determined by the dependency relationship between the operators. It can propagate from the input of the consumer operator to the output of the producer operator, i.e., forward propagation between operators, or from the output of the producer operator to the input of the consumer operator, i.e., backward propagation between operators.
[0061] Intra-operator propagation refers to the ability of a tensor's partitioning strategy to propagate within an operator to the axes of other tensors with corresponding axial relationships, according to a predefined method. Intra-operator propagation includes four types: input-to-input, input-to-output, output-to-input, and output-to-output. Specifically, input-to-input propagation refers to the requirement for axis alignment between multiple input tensors within the same operator. For example, in matrix multiplication A*B=C, the column axis of A must be aligned with the row axis of B. Input-to-output and output-to-input propagation are similar to inter-operator propagation and will not be elaborated further. Output-to-output propagation refers to the requirement for consistency between output tensors when multiple outputs exist within an operator, to facilitate subsequent operations. When defining a new operator, you can define the intra-operator propagation method yourself or have it automatically derived based on an AxisMap.
[0062] It should be understood that operators include two attributes: AxisMap and AxisType. AxisMap describes the positional relationships between the axes of the input and output tensors within an operator; axes with the same relationship are marked here. For example, in matrix multiplication A*B=C, where A has a shape of (m, k), B has a shape of (k, n), and C has a shape of (m, n), AxisMap stores the mapping relationships between the m-axis of A and the m-axis of C, between the k-axis of A and the k-axis of B, and between the n-axis of B and the n-axis of C. Based on AxisMap, we can know that the m-axis of A and the m-axis of C have the same relationship, and the n-axis of B and the n-axis of C have the same relationship. AxisType is used to describe the type of axis for each input and each output within an operator. In the example above, the m-axis and n-axis are parallel axes, which can be directly computed in parallel after partitioning. The k-axis is a reduce_sum axis, which cannot be directly computed in parallel after partitioning and requires a reduce_sum operation.
[0063] In one optional implementation, when the partitioning axis propagates to an axis with AxisType set to reduce, since parallel computation cannot be performed directly after partitioning, the operator needs to be marked to indicate that a corresponding device reduce operation needs to be inserted after the operator to maintain the correctness of the result. Axis types with AxisType set to reduce include axes such as reduce_sum, reduce_max (maximum value reduction), reduce_min (minimum value reduction), and reduce_product (cumulative product reduction), which belong to the reduction operation category.
[0064] The above propagation method allows the segmentation method to be propagated from a tensor with an existing segmentation method to a target tensor that does not yet have a segmentation method. The target tensor is the tensor whose segmentation method is to be propagated during the current propagation process. During this process, if the target tensor already has a segmentation method, it is determined whether there is a conflict between the target tensor's segmentation method and the current segmentation method. If, during the propagation of the segmentation method, there is a conflict between the target tensor's segmentation method and the current segmentation method, and no other available propagation method exists, a rollback operation is performed. Specifically, if there is no conflict, the current propagation stops, and subsequent tensor propagation operations are performed. If there is a conflict and other feasible propagation methods exist, other feasible propagation methods are tried. If there is a conflict and no other feasible propagation methods exist, a propagation error is considered to have occurred, the current propagation is terminated, and a rollback operation is performed. In the absence of a propagation error, the propagation process will continue until all possible propagations have been completed. Whether there is a conflict between the target tensor's segmentation method and the current segmentation method includes: inconsistencies in the segmentation coefficients and their arrangement between the segmentation axes, or the operator being unable to implement the current segmentation method.
[0065] Figure 2 This is a flowchart of the rollback operation method according to an embodiment of the present invention. Figure 2 As shown above, in response to a conflict between the existing segmentation method and the current segmentation method of the target tensor in the propagation path, a rollback operation is performed. The rollback operation includes the following steps:
[0066] Step S210: Backtrack each operator of the current propagation path to the state where the segmentation method is not determined.
[0067] Step S220: In response to the existence of available segmentation methods for the current operator to be searched, the next segmentation method of the current operator to be searched is selected as the target segmentation method for propagation.
[0068] Step S230: In response to the absence of an available segmentation method for the operator to be searched or the absence of a conflict with the target segmentation method, the rollback operation ends.
[0069] This invention, by executing a rollback mechanism when a conflict occurs during propagation, backtracks each operator on the current propagation path to a state where the segmentation method is undetermined, and reselects a segmentation method for propagation. This enables rapid handling of errors during propagation and enhances the robustness of the system.
[0070] In one optional implementation, the benefit parameters corresponding to each partitioning strategy include a combination of one or more of the following: the hard weight space occupancy benefit, the ultra-large space weight benefit, the soft weight space occupancy benefit, and the inter-device communication benefit of each computing device under the partitioning strategy.
[0071] Specifically, in order to calculate the above-mentioned revenue parameters, the following parameters need to be defined:
[0072] TilingMemUsage represents the total space occupied by the weights after partitioning, in bytes. The larger the value of TilingMemUsage, the lower the processing efficiency of the current partitioning strategy.
[0073] HardMemUpperBound (predefined upper limit of space usage) represents the maximum space limit that the partitioning strategy cannot exceed, in bytes. Optionally, its specific value can be determined by the system based on the hardware limitations of the computing device, or it can be configured by the user according to actual needs.
[0074] BigConstSize represents the lower bound of the space for super-large weights, in bytes. A weight exceeding BigConstSize indicates that the weight is too large, and there may be an unreasonable partitioning.
[0075] Among the above revenue parameters, the revenue from hard weight space occupation is determined based on the predetermined upper limit of space occupation and the sum of the space occupation of the weights after partitioning, and its calculation formula is -max(HardMemUpperBound, TilingMemUsage). The revenue from super-large space weight is determined based on the number of weights exceeding the lower limit of super-large weight space, and its calculation formula is the negative of the number of weights exceeding BigConstSize. The revenue from soft weight space occupation is determined based on the difference between the predetermined upper limit of space occupation and the sum of the space occupation of the weights after partitioning, and its calculation formula is max(HardMemUpperBound – TilingMemUsage, 0). The revenue from inter-device communication is determined based on the sum of the communication volume of the operators that need to perform device communication, and its calculation formula is the negative of the sum of the communication volume of the operators that need to perform device communication, where the unit of communication volume is bytes.
[0076] In another optional implementation, the revenue parameters corresponding to each segmentation strategy can be determined by a pre-trained neural network model. Specifically, this involves inputting the segmentation strategy and its corresponding cost into the neural network model to obtain the revenue parameters. The cost of each segmentation strategy includes the memory, computing resources, and communication resources required by each computing device to execute the segmentation strategy.
[0077] In another alternative implementation, the revenue parameters for each segmentation strategy can be determined based on the relative revenue differences between it and other segmentation strategies. For example, the revenue parameters for one segmentation strategy can be determined first, and then the revenue parameters for each segmentation strategy can be determined based on the revenue parameters of that segmentation strategy and the relative revenue differences between it and other segmentation strategies.
[0078] Optionally, the revenue parameter can be an array, such as (hard weight space occupation revenue, super-large weight revenue, inter-device communication revenue, soft weight space occupation revenue), where the order of each element in the array represents the sorting priority of the revenue.
[0079] Figure 3 This is a flowchart of a method for determining target candidate strategies according to an embodiment of the present invention. Figure 3 As shown, in one optional implementation, a corresponding number of target candidate strategies are determined from each segmentation strategy based on the propagation state of each segmentation method and the revenue parameters of the segmentation strategies formed by each segmentation method. Specifically, this may include the following steps:
[0080] Step S310: Traverse each segmentation method, propagate each segmentation method from the current operator to be searched based on the connection relationship, and obtain the propagation status of each segmentation method. The propagation status is used to indicate whether the segmentation method has completed propagation.
[0081] Step S320: Combine the segmentation methods that have completed propagation with the candidate strategies corresponding to the historical search operators determined before the current search operator to obtain the segmentation strategy formed by each segmentation method.
[0082] Step S330: Obtain the profit parameters of each splitting strategy.
[0083] Step S340: Based on the predetermined search algorithm and the reward parameters of each segmentation strategy, determine a corresponding number of target candidate strategies from the segmentation strategies. Optionally, the predetermined search algorithm can be a beam search algorithm, a heuristic search algorithm, a random beam search algorithm, or other algorithms capable of controlling the search width; this embodiment does not limit this. The following description mainly uses the beam search algorithm as an example. It should be understood that those skilled in the art are capable of making detailed modifications to the following method to make it applicable to other algorithms such as heuristic search algorithms.
[0084] The beam search algorithm evaluates the searched segmentation strategies in each propagation path, selects the N strategies with the highest returns, and determines the highest-return segmentation strategy as the target strategy after all search operators have been searched. N is the beam width of the beam search algorithm, and the value of N is greater than or equal to 1. The specific value of N can be preset on the device executing the method, or it can be set by technical personnel according to actual business needs. Specifically, a smaller N value can be set in scenarios where high search speed or reduced search resource consumption is required, or a larger N value can be set in scenarios where high search quality is required. By customizing the N value, the space occupied and search range during the search process can be controlled, achieving a balance between search speed and search width. When the predetermined search algorithm is the beam search algorithm, the N value is the predetermined number in step S130.
[0085] When the predetermined search algorithm is the beam search algorithm, the corresponding number of target candidate strategies can be determined from the beam search algorithm and the benefit parameters of each beam search strategy. This can include: sorting each beam search strategy based on a predetermined benefit ranking priority and the benefit parameters of each beam search strategy, and then using the beam search algorithm to determine the target candidate strategies from the sorted beam search strategies.
[0086] Specifically, taking the array (Hard weight space occupation benefit, Super-large weight benefit, Inter-device communication benefit, Soft weight space occupation benefit) as an example, the pre-determined benefit ranking priority can be as follows: First, compare the first element in each array, i.e., the size of the hard weight space occupation benefit. The segmentation strategy with a larger hard weight space occupation benefit will have a higher benefit and will be placed in a higher position in the ranking. For segmentation strategies with the same hard weight space occupation benefit, compare the second array element, i.e., the Super-large weight benefit, and so on. If all array elements are the same, the two segmentation strategies are considered to have the same benefit. It should be understood that the priority of different benefits can be set by the technical personnel.
[0087] In another optional implementation, determining a corresponding number of target candidate strategies from each segmentation strategy based on the propagation state of each segmentation method and the revenue parameters of the segmentation strategies formed by each segmentation method may further include: combining each segmentation method with the candidate strategies corresponding to historical search operators determined before the current search operator to obtain the segmentation strategies formed by each segmentation method; obtaining the revenue parameters of each segmentation strategy; sorting each segmentation strategy based on a predetermined search algorithm and the revenue parameters of each segmentation strategy; and propagating the segmentation methods in each segmentation strategy sequentially according to the sorting until a corresponding number of target candidate strategies are obtained. The corresponding number is less than or equal to a predetermined number, which is determined according to a predetermined search algorithm.
[0088] Step S140: In response to the completion of operator segmentation in the computation graph, a target strategy is determined from each target candidate strategy based on the payoff parameters corresponding to each target candidate strategy.
[0089] Step S150: The tensor is segmented according to the target strategy and distributed to each computing device for parallel processing.
[0090] This embodiment determines the current operator to be searched in a computation graph containing operators to be searched, identifies at least one segmentation method corresponding to the current operator, and, in response to the existence of other operators to be searched in the computation graph, determines a corresponding number of target candidate strategies from each segmentation strategy based on the propagation state of each segmentation strategy and the revenue parameters of the segmentation strategies formed by each segmentation strategy. It then determines the next operator to be searched in the computation graph. In response to the completion of operator segmentation in the computation graph, it determines the target strategy from each target candidate strategy based on the revenue parameters corresponding to each target candidate strategy. Finally, it segments the tensor according to the target strategy and allocates it to each computing device for parallel processing. Therefore, this embodiment of the invention can control the search space size of the segmentation strategy, improving search speed while ensuring a certain degree of search width and versatility.
[0091] It should be understood that a segmentation method in a propagation path consists of the segmentation methods of each operator along the path. Each operator's segmentation method can include a factor (segmentation coefficient, represented by an integer) and parts (segmentation order, represented by an integer array, used to indicate the ID of the specified computing device to which the operator is segmented). That is, if all operators along the propagation path have the same factor and parts, then the two segmentation methods can be considered duplicated. Specifically, during the search for segmentation strategies, since different factor permutations may result in the same number of factor products after segmentation using different methods, i.e., the same segmentation coefficient, such as segmenting an operator of shape (2, 8) in (1, 2) or (2, 1), the segmentation coefficient is 2 in both cases. Furthermore, the same allocation method may exist when distributing the segmented tensors to different computing devices, i.e., the same segmentation order. Therefore, it is necessary to filter duplicate segmentation strategies.
[0092] In one alternative implementation, the tensor parallel processing method also includes a strategy of filtering duplicate partitions using a hash algorithm. Figure 4 This is a flowchart of the filtering and segmentation strategy method according to an embodiment of the present invention. Figure 4 As shown, the strategy for filtering duplicate partitions using a hash algorithm can specifically include the following steps:
[0093] Step S410: After the propagation of the segmentation method is completed, the segmentation coefficients and segmentation order of each operator in the propagation path are summed by a hash algorithm according to the propagation order of the operators in the propagation path to obtain the hash value corresponding to each segmentation method.
[0094] For ease of understanding, the hash value generation process is given in pseudocode form below:
[0095] total_hash = 0 / / Initialize the hash value corresponding to the current splitting method
[0096] For op in ops: / / Traverse each operator in the propagation path
[0097] hash_value = hash(op.factor) / / Calculate the hash value corresponding to the splitting factor of the current operator.
[0098] For part in op.parts: / / Iterate through the computing device IDs corresponding to the current operator's segmentation order.
[0099] hash_value = hash_combine(hash_value, part) / / Perform a hash algorithm to sum the hash values of the current splitting factor and the current computing device ID.
[0100] total_hash = hash_combine(total_hash, hash_value) / / Calculate the hash value corresponding to the current splitting method
[0101] return total_hash / / Returns the hash value corresponding to the current splitting method.
[0102] In this context, “total_hash”, “op”, “ops”, “hash_value”, and “part” refer to “the hash value corresponding to the current segmentation method”, “the operator”, “the set of operators on the current propagation path”, “the hash value corresponding to the segmentation method of the current operator”, and “the current computing device ID”, respectively.
[0103] Step S420: Filter out duplicate splitting methods based on hash values.
[0104] This embodiment improves the search efficiency of the segmentation strategy by summing the segmentation coefficients and segmentation order of each operator in the propagation path using a hash algorithm, obtaining the hash value corresponding to each segmentation method, and filtering out duplicate segmentation methods based on the hash value.
[0105] Figure 5 This is a flowchart illustrating the method for searching a segmentation strategy based on a beam search algorithm according to an embodiment of the present invention. Figure 5 As shown, the beam search algorithm's segmentation strategy specifically includes the following steps:
[0106] Step S510: Initialize the beam search node and determine the beam width as N.
[0107] Step S520: Determine whether there is an operator to be searched in the computation graph. If yes, proceed to step S530; otherwise, proceed to step S590.
[0108] Step S530: Determine the current operator to be searched and generate at least one segmentation method corresponding to the current operator to be searched.
[0109] Step S540: Select a segmentation method for propagation.
[0110] Step S550: Determine whether a conflict has occurred during propagation. If yes, proceed to step S570; otherwise, proceed to step S560.
[0111] Step S560: Calculate the revenue parameters of the current splitting method and add the current splitting method to the bundle search as a new node.
[0112] Step S570: Determine whether there are available segmentation methods for the current operator to be searched. Available segmentation methods refer to the uncalculated segmentation methods among all the segmentation methods corresponding to the current operator to be searched.
[0113] Step S580: Retain the N splitting strategies with the highest returns among multiple nodes as target candidate strategies.
[0114] Step S590: Determine the target candidate strategy with the highest return as the target strategy, and divide the tensor according to the target strategy and distribute it to each computing device for parallel processing.
[0115] Figure 6 This is a schematic diagram of the calculation graph according to an embodiment of the present invention. Figure 7 This is a schematic diagram of the beam search process according to an embodiment of the present invention. Figure 6 , 7 As shown, in an optional embodiment, the computation graph includes eight operators: a, b, c, d, e, f, g, and h. Among them, ab, bc, cf, de, ef, and gh all have unidirectional connections. Figure 7 In the diagram, AG represents each node in the beam search algorithm. The following explanation uses a beam width of 2 as an example.
[0116] In one alternative implementation, firstly, all operators in the computation graph are divisible and have not yet been divisible. Any operator can be selected as the operator to be searched. Since operator 'a' has a long connection chain in the computation graph, we can take operator 'a' as an example. We determine the current operator to be searched as 'a' and proceed with subsequent propagation. In the first round of search corresponding to the propagation path abcf, three divisible methods (A, B, and C) that have completed propagation are found. In the first round of search, since there are no historical operators to be searched before the current operator to be searched, the resulting divisible method is the divisible strategy. Since the bundle width is 2, only the two divisible strategies with higher returns, A and C, are retained as target candidate strategies.
[0117] In the second round of search, the first search is performed using the partitioning method A as a candidate strategy. In response to the existence of operators d, e, g, and h in the computation graph, the current operator d is determined.
[0118] Iterate through the segmentation methods H, D, and E corresponding to the current operator d to be searched. Starting from the operator d to be searched, propagate the segmentation method H along the propagation path of def. When the segmentation method H propagates to f, it is found that the operator f already has a segmentation method A obtained in the propagation path abcf, and there is a conflict between segmentation method A and segmentation method H. At the same time, there are no other feasible propagation methods in the current propagation path, so a rollback operation is performed.
[0119] In the rollback operation, operators d and e that have completed propagation in the propagation path def are backtracked to the state where the segmentation method is undetermined. If a usable segmentation method D exists for the current operator to be searched, segmentation method D is selected and propagated again. If segmentation method D does not conflict with segmentation method A, segmentation method D is retained after filtering as a new node in the bundle search algorithm, and the rollback operation ends. If segmentation method D still conflicts with segmentation method A, if a usable segmentation method E exists for the current operator to be searched, segmentation method E is selected as the target segmentation method and propagated again until the target segmentation method does not conflict or no usable segmentation method exists for the current operator to be searched, at which point the rollback operation ends.
[0120] The following explanation uses the example of a segmentation method D that does not conflict with segmentation method A.
[0121] After the current search is completed, the segmentation strategy AD is obtained. Then, segmentation method A is used as a candidate strategy, and segmentation method E is propagated to obtain segmentation strategy AE. The payoff parameters of segmentation strategies AD and AE are calculated. Next, the above search process is repeated with segmentation method C as a candidate strategy to obtain segmentation strategies CD and CE and their corresponding payoff parameters. Based on the payoff parameters of each segmentation strategy, the two segmentation strategies AE and CE with higher payoffs are retained as target candidate strategies.
[0122] In response to the existence of operators g and h in the computation graph, the current operator to be searched is determined to be g. Searches are performed based on AE and CE respectively, and two segmentation methods, F and G, are obtained. These are combined with the two previously retained candidate segmentation strategies to form four segmentation strategies: AEF, AEG, CEF, and CEG. After benefit evaluation, the two target candidate strategies with higher benefits, CEF and CEG, are retained.
[0123] In response to the absence of a search operator in the computation graph, the search ends. From the two remaining target candidate strategies CEF and CEG, the splitting strategy with the highest return is selected as the target strategy. The tensor is split according to the target strategy and distributed to each computing device for parallel processing.
[0124] In another optional implementation, three segmentation methods, A, B, and C, are found in the first round of search. Since there are no previous search operators before the current one to be searched, the obtained segmentation method is the segmentation strategy. Because the number of segmentation strategies exceeds the bundle width of 2, the payoff parameters corresponding to the three segmentation strategies can be obtained first, and then sorted according to the payoff parameters. For example, the segmentation strategies, ranked from highest to lowest payoff, are C, A, and B. These strategies can be propagated in this order. If no conflict occurs during propagation, the currently propagated segmentation strategy is determined as the target candidate strategy. After segmentation strategy A has been propagated, the number of target candidate strategies has reached the bundle width of 2, the first round of search ends, segmentation strategies A and C are determined as target candidate strategies, and segmentation strategy B is no longer propagated.
[0125] In the second round of search, three segmentation methods (H, D, and E) were found, resulting in six segmentation strategies: AH, AD, AE, CH, CD, and CE. These six strategies were then sorted according to their payoff parameters. For example, the segmentation strategies, ranked from highest to lowest payoff, were CH, AH, CE, AE, CD, and AD. First, segmentation method H within strategy CH was propagated. If a conflict occurred during propagation, the aforementioned rollback operation was executed. Then, the segmentation methods within strategies AH, CE, AE, CD, and AD were propagated sequentially until the number of identified target candidate strategies reached a bundle width of 2. After the second round of search, two target candidate strategies, CE and AE, were obtained. The third round of search was similar to the above process and will not be described further.
[0126] In this embodiment, a beam search algorithm is used to search for segmentation strategies. After each round of search, N superior segmentation strategies are retained, effectively reducing search resource consumption and improving search efficiency. Furthermore, by adjusting the value of N, different requirements for search efficiency and search quality in different scenarios can be met.
[0127] This embodiment determines the current operator to be searched in a computation graph containing operators to be searched, identifies at least one segmentation method corresponding to the current operator, and, in response to the existence of other operators to be searched in the computation graph, determines a corresponding number of target candidate strategies from each segmentation strategy based on the propagation state of each segmentation strategy and the revenue parameters of the segmentation strategies formed by each segmentation strategy, and determines the next operator to be searched. In response to the completion of operator segmentation in the computation graph, a target strategy is determined from each target candidate strategy based on the revenue parameters corresponding to each target candidate strategy. The tensor is then segmented according to the target strategy and allocated to each computing device for parallel processing. Therefore, this embodiment can control the search space size of the segmentation strategy, improving search speed while ensuring a certain degree of search width and versatility. Furthermore, by adjusting the method of determining the revenue parameters and target candidate strategies, this embodiment can meet the needs of different scenarios, further improving versatility.
[0128] Figure 8 This is a schematic diagram of a tensor parallel processing device according to an embodiment of the present invention. Figure 8 As shown, the tensor parallel processing device in this embodiment includes an operator determination unit 81, a segmentation unit 82, a strategy filtering unit 83, a strategy determination unit 84, and a processing unit 85.
[0129] The operator determination unit 81 is configured to determine the current operator to be searched in response to the existence of an operator to be searched in the computation graph. The computation graph includes multiple operators and the connection relationships between them.
[0130] The segmentation unit 82 is configured to determine at least one segmentation method corresponding to the current search operator.
[0131] The strategy selection unit 83 is configured to, in response to the existence of operators to be searched in the computation graph, determine a corresponding number of target candidate strategies from each segmentation strategy based on the propagation state of each segmentation method and the payoff parameters of the segmentation strategies formed by each segmentation method, and determine the next operator to be searched. The corresponding number is less than or equal to a predetermined number, and the segmentation strategy is determined based on a combination of candidate strategies and segmentation strategies corresponding to the operators to be searched determined before the current operator to be searched.
[0132] In one optional implementation, the strategy filtering unit 83 includes a revenue calculation subunit. The revenue calculation subunit is configured to calculate the revenue parameters corresponding to each segmentation strategy. Optionally, the revenue parameters corresponding to the segmentation strategy include a combination of one or more of the following: the revenue from hard weight space occupied by each computing device under the segmentation strategy, the revenue from excessively large space weights, the revenue from soft weight space occupied, and the revenue from inter-device communication. Specifically, the revenue from hard weight space occupied is determined by negating the larger of the maximum space limit prohibited by the segmentation strategy and the sum of the space sizes of the segmented weights; the revenue from excessively large space weights is determined by negating the number of weights whose size exceeds the minimum space limit of excessively large weights; the revenue from soft weight space occupied is the larger of the difference between the maximum space limit prohibited by the segmentation strategy and the sum of the space sizes of the segmented weights, and zero; and the revenue from inter-device communication is determined by negating the sum of the communication volumes of operators requiring device communication.
[0133] Optionally, the revenue calculation subunit can also input the segmentation strategy and its corresponding cost into the neural network model to obtain the revenue parameters corresponding to the segmentation strategy. The cost corresponding to the segmentation strategy includes the memory, computing resources, and communication resources required by each computing device to execute the segmentation strategy.
[0134] In one optional implementation, the strategy selection unit 83 further includes a propagation subunit, a sorting subunit, and a selection subunit. The propagation subunit is configured to traverse each segmentation method, propagate each segmentation method from the current search operator based on connection relationships, obtain the propagation state of each segmentation method, and combine the propagated segmentation methods with the candidate strategies corresponding to historical search operators determined before the current search operator to obtain the segmentation strategy formed by each segmentation method. The sorting subunit is configured to obtain the payoff parameters of each segmentation strategy. The selection subunit is configured to determine a corresponding number of target candidate strategies from the segmentation strategies based on a predetermined search algorithm and the payoff parameters of each segmentation strategy. Optionally, the predetermined search algorithm can be a beam search algorithm, a heuristic search algorithm, a random beam search algorithm, or other search algorithms capable of controlling the search width.
[0135] In one optional implementation, the strategy filtering unit 83 further includes a rollback subunit. When the segmentation method of the target tensor in the propagation path conflicts with the current segmentation method, the strategy filtering unit 83 calls the rollback subunit to perform a rollback operation. The rollback subunit is configured to backtrack each operator in the propagation path to a state where no segmentation method is determined. In response to the existence of an available segmentation method for the current operator to be searched, it selects the next segmentation method of the current operator to be searched as the target segmentation method for propagation. In response to the absence of an available segmentation method for the current operator to be searched or the absence of a conflict with the target segmentation method, it ends the rollback operation.
[0136] The strategy determination unit 84 is configured to determine the target strategy from the target candidate strategies based on the payoff parameters corresponding to each target candidate strategy in response to the completion of operator segmentation in the computation graph.
[0137] Processing unit 85 is configured to split tensors according to the target strategy and distribute them to various computing devices for parallel processing.
[0138] In one optional implementation, the tensor parallel processing device further includes a filtering unit for filtering duplicate partitioning methods. The filtering unit is configured to perform a hash algorithm to sum the partitioning coefficients and partitioning order of each operator in each propagation path according to the propagation order of the operators, obtaining a hash value corresponding to each partitioning method, and filtering duplicate partitioning methods based on the hash value. Here, the partitioning coefficients and partitioning order of each operator represent the current partitioning method.
[0139] This embodiment determines the current operator to be searched in a computation graph containing operators to be searched, identifies at least one segmentation method corresponding to the current operator, and, in response to the existence of other operators to be searched in the computation graph, determines a corresponding number of target candidate strategies from each segmentation strategy based on the propagation state of each segmentation strategy and the revenue parameters of the segmentation strategies formed by each segmentation strategy, and determines the next operator to be searched. In response to the completion of operator segmentation in the computation graph, a target strategy is determined from each target candidate strategy based on the revenue parameters corresponding to each target candidate strategy. The tensor is then segmented according to the target strategy and allocated to each computing device for parallel processing. Therefore, this embodiment of the invention can control the search space size of the segmentation strategy, improving search speed while ensuring a certain degree of search width and versatility.
[0140] Figure 9 This is a schematic diagram of an electronic device according to an embodiment of the present invention. (For example...) Figure 9 As shown, electronic device 9 is a general-purpose data processing device, which includes a general-purpose computer hardware structure, including at least a processor 91 and a memory 92. The processor 91 and memory 92 are connected via a bus 93. The memory 92 is adapted to store instructions or programs executable by the processor 91. The processor 91 can be a standalone microprocessor or a collection of one or more microprocessors. Thus, the processor 91 executes the instructions stored in the memory 92, thereby deploying the request processing unit as described above. The bus 93 connects the aforementioned components together, and also connects these components to a display controller 94, a display device, and an input / output (I / O) device 95. The input / output (I / O) device 95 can be a mouse, keyboard, modem, network interface, touch input device, motion-sensing input device, printer, and other devices known in the art. Typically, the input / output device 95 is connected to the system via an input / output (I / O) controller 96.
[0141] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0142] This application is described with reference to flowchart illustrations of methods, apparatus (devices), and computer program products according to embodiments of this application. It should be understood that each step in the flowchart can be implemented by computer program instructions.
[0143] These computer program instructions may be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction means, the implementation process of which is described in the instruction means. Figure 1 The function specified in one or more processes.
[0144] These computer program instructions may also be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, produce instructions for implementing processes. Figure 1 A device for a function specified in one or more processes.
[0145] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program for use by a computer to execute some or all of the above-described method embodiments.
[0146] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program specifying the relevant hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0147] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A tensor parallel processing method, characterized in that, The method includes: In response to the existence of an operator to be searched in the computation graph, the current operator to be searched is determined, wherein the computation graph includes multiple operators and the connection relationships between the operators; Determine at least one segmentation method for the current operator to be searched; In response to the existence of operators to be searched in the computation graph, a corresponding number of target candidate strategies are determined from each of the segmentation strategies based on the propagation state of each segmentation method and the revenue parameters of the segmentation strategies formed by each segmentation method, and the next operator to be searched is determined. The corresponding number is less than or equal to a predetermined number. The segmentation strategy is determined based on the combination of each candidate strategy corresponding to the historical operators to be searched determined before the current operator to be searched and each of the segmentation methods. In response to the completion of operator segmentation in the computation graph, a target strategy is determined from each of the target candidate strategies based on the payoff parameters corresponding to each target candidate strategy. The tensor is segmented according to the target strategy and distributed to various computing devices for parallel processing.
2. The method according to claim 1, characterized in that, The revenue parameters corresponding to the partitioning strategy include one or more of the following: the revenue from hard weight occupancy space, the revenue from ultra-large weight, the revenue from soft weight occupancy space, and the revenue from inter-device communication under the partitioning strategy. The revenue from the space occupied by hard weights is determined based on a predetermined upper limit for space occupied and the total space occupied by weights after segmentation. The revenue from the ultra-large space weights is determined based on the number of weights exceeding the lower limit of the ultra-large weight space. The revenue from the space occupied by soft weights is determined based on the difference between the predetermined upper limit for space occupied and the total space occupied by weights after segmentation. The revenue from inter-device communication is determined based on the total communication volume of operators that need to perform device communication.
3. The method according to claim 2, characterized in that, The process of determining a corresponding number of target candidate strategies from each of the segmentation strategies based on the propagation state of each segmentation method and the revenue parameters of the segmentation strategies formed by each segmentation method includes: Traverse each of the aforementioned segmentation methods, propagate each of the aforementioned segmentation methods starting from the current operator to be searched based on the connection relationship, and obtain the propagation status of each of the aforementioned segmentation methods. The propagation status is used to indicate whether the segmentation method has completed propagation. The segmentation methods that have completed propagation are combined with the candidate strategies corresponding to the historical search operators determined before the current search operator to obtain the segmentation strategy formed by each segmentation method. Obtain the revenue parameters for each of the aforementioned segmentation strategies; Based on a predetermined search algorithm and the payoff parameters of each of the segmentation strategies, a corresponding number of target candidate strategies are determined from the segmentation strategies. The predetermined search algorithm is a beam search algorithm, a heuristic search algorithm, or a random beam search algorithm.
4. The method according to claim 3, characterized in that, The segmentation method includes a segmentation coefficient and a segmentation arrangement order; the method further includes: After the propagation of the segmentation method is completed, the segmentation coefficients and segmentation order of each operator in the current propagation path are summed using a hash algorithm according to the propagation order of the operators in the current propagation path to obtain the hash value corresponding to the segmentation method. The splitting method filters out duplicates based on hash values.
5. The method according to any one of claims 1-4, characterized in that, The method further includes: When the propagation of the segmentation method is performed, if the existing segmentation method of the target tensor conflicts with the current segmentation method and there is no available propagation method, a rollback operation is performed. Here, the target tensor is the tensor of the segmentation method to be propagated in the current propagation process.
6. The method according to any one of claims 1-4, characterized in that, The method further includes: The segmentation strategy and its corresponding cost are input into a neural network model to obtain the revenue parameters corresponding to the segmentation strategy. The cost includes the memory, computing resources, and communication resources required by each computing device to execute the segmentation strategy.
7. A tensor parallel processing device, characterized in that, The device includes: An operator determination unit is configured to determine the current operator to be searched in response to the existence of an operator to be searched in a computation graph, wherein the computation graph includes multiple operators and the connection relationships between the operators; The segmentation unit is configured to determine at least one segmentation method for the current operator to be searched; The strategy filtering unit is configured to, in response to the existence of operators to be searched in the computation graph, determine a corresponding number of target candidate strategies from each of the segmentation strategies based on the propagation state of each segmentation method and the revenue parameters of the segmentation strategies formed by each segmentation method, and determine the next operator to be searched, wherein the corresponding number is less than or equal to a predetermined number, and the segmentation strategy is determined based on the combination of each candidate strategy corresponding to the historical operators to be searched determined before the current operator to be searched and each of the segmentation methods; The strategy determination unit is configured to determine a target strategy from each of the target candidate strategies based on the payoff parameters corresponding to each of the target candidate strategies in response to the completion of operator segmentation in the computation graph. The processing unit is configured to segment the tensor according to the target strategy and distribute it to each computing device for parallel processing.
8. An electronic device comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-6.
10. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-6.