A dataflow graph processing method, device, and computer-readable storage medium

Abstract

Embodiments of the present application disclose a data flow graph processing method and device based on artificial intelligence and a computer readable storage medium. The method comprises the following steps: if an original data flow graph comprises a split operation node and N sub-matrix multiplication operation nodes, detecting the position of the split operation node in the original data flow graph; if the split operation node is connected with the N sub-matrix multiplication operation nodes respectively, and the position of the split operation node in the original data flow graph is before the N sub-matrix multiplication operation nodes, updating the N sub-matrix multiplication operation nodes into a total matrix multiplication operation node, and updating the split operation node into a first deformation operation node; adding a merging operation node before the total matrix multiplication operation node, and adding a de-merging operation node after the total matrix multiplication operation node, to obtain an optimized data flow graph. The present application can improve the utilization rate of system resources of the data flow graph. The present application can be applied to a multimedia data analysis scene.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a data flow graph processing method, device, and computer-readable storage medium. Background Technology

[0002] A Data Flow Diagram (DFD) is a tool used in structured analysis methods. It graphically depicts the flow and processing of data within a system. Because it only reflects the logical functions that the system must perform, it is a functional model. From the perspective of data transmission and processing, the DFD graphically expresses the logical functions of the system, the logical flow of data within the system, and the logical transformation process. It is a primary expressive tool in structured systems analysis and a graphical method for representing software models.

[0003] In terms of application scenarios, data flow graphs can be used to: users first write data flow graphs, then submit them to a stream computing platform. The platform can parse the data flow graphs and schedule them for distributed execution across different physical machines, thereby enabling different business scenarios. However, existing data flow graphs may contain nodes with identical operation methods. Since each node requires resource allocation, this can lead to redundant resource allocation and waste of system resources. Summary of the Invention

[0004] This application provides a data flow graph processing method, device, and computer-readable storage medium, which can improve the utilization rate of data flow graph system resources.

[0005] One embodiment of this application provides a data flow graph processing method, including:

[0006] Obtain the original data flow graph and parse the original data flow graph;

[0007] If the original data flow graph is parsed to include split operation nodes and N submatrix multiplication operation nodes, then the position of the split operation node in the original data flow graph is detected; N is a positive integer, and N is equal to or greater than 2;

[0008] If, in the original data flow graph, the split operation node is connected to N submatrix multiplication operation nodes respectively, and the position of the split operation node in the original data flow graph is before the N submatrix multiplication operation nodes, then the N submatrix multiplication operation nodes are updated to the total matrix multiplication operation node, and the split operation node is updated to the first deformation operation node used to perform deformation operations on the total input tensor.

[0009] The merging operation node for performing the merging operation on the N sub-input tensors is added before the total matrix multiplication operation node, and the de-merging operation node is added after the total matrix multiplication operation node, to obtain an optimized data flow graph; the node relationship between the merging operation node and the first deformation operation node is a parallel node relationship; and the total input tensor and the N sub-input tensors have a size correlation relationship.

[0010] The embodiment of the application provides a data flow graph processing device, which comprises:

[0011] An acquisition data module is configured to acquire an original data flow graph and parse the original data flow graph.

[0012] A detection position module is configured to detect the position of the splitting operation node in the original data flow graph if the original data flow graph comprises the splitting operation node and the N sub-matrix multiplication operation nodes; N is a positive integer and N is greater than or equal to 2.

[0013] An update node module is configured to update the N sub-matrix multiplication operation nodes into a total matrix multiplication operation node and update the splitting operation node into a first deformation operation node for performing a deformation operation on a total input tensor if the splitting operation node and the N sub-matrix multiplication operation nodes are connected respectively in the original data flow graph and the position of the splitting operation node in the original data flow graph is before the N sub-matrix multiplication operation nodes.

[0014] An addition node module is configured to add a merging operation node for performing a merging operation on N sub-input tensors before the total matrix multiplication operation node and add a de-merging operation node after the total matrix multiplication operation node to obtain an optimized data flow graph; the node relationship between the merging operation node and the first deformation operation node is a parallel node relationship; and the total input tensor and the N sub-input tensors have a size correlation relationship.

[0015] The first deformation operation node comprises a pre-input deformation operation node or a post-input deformation operation node; and the total input tensor comprises a pre-input tensor or a post-input tensor.

[0016] The update node module comprises:

[0017] A first determination unit is configured to determine the input attribute of the output result of the splitting operation node for the N sub-matrix multiplication operation nodes and the splitting operation attribute of the splitting operation node.

[0018] A first update unit is configured to update the splitting operation node into a pre-input deformation operation node for performing a deformation operation on a pre-input tensor if the input attribute is a pre-input attribute and the splitting operation attribute is a row splitting operation attribute; and the deformation operation attribute of the pre-input deformation operation node is a row deformation operation attribute.

[0019] The second updating unit is configured to update the split operation node to a post-input morphing operation node for performing a morphing operation on the post-input tensor if the input attribute is the post-input attribute and the split operation attribute is the column split operation attribute, and the morphing operation attribute of the post-input morphing operation node is the column morphing operation attribute.

[0020] The N post-input sub-tensors have a size association relationship with the pre-input tensor and respectively carry different sequence numbers.

[0021] The adding node module is specifically configured to, when the first morphing operation node is the pre-input morphing operation node, add a stacking operation node for performing a stacking operation on the N post-input sub-tensors before the total matrix multiplication operation node and add a de-stacking operation node after the total matrix multiplication operation node to obtain the optimized data flow graph.

[0022] The N post-input sub-tensors respectively correspond to tensor dimensions that are the same, and the N tensor dimensions respectively correspond to sizes that are the same.

[0023] The data flow graph processing apparatus further includes:

[0024] The first operation module is configured to input the pre-input tensor into the pre-input morphing operation node in the optimized data flow graph, perform a morphing operation on the pre-input tensor through the pre-input morphing operation node, and obtain a pre-input dimension-increased tensor.

[0025] The first operation module is further configured to input the N post-input sub-tensors into the stacking operation node in the optimized data flow graph in sequence number order, perform a stacking operation on the N post-input sub-tensors through the stacking operation node, and obtain a post-input stacked tensor. The post-input stacked tensor includes a tensor batch dimension with a size equal to N, and a size of a tensor dimension of the post-input stacked tensor, except for the tensor batch dimension with the size equal to N, is equal to sizes of the N tensor dimensions respectively.

[0026] The first operation module is further configured to input the pre-input dimension-increased tensor and the post-input stacked tensor into the total matrix multiplication operation node respectively, perform a matrix multiplication operation on the pre-input dimension-increased tensor and the post-input stacked tensor through the total matrix multiplication operation node, and obtain a first total output tensor.

[0027] The first operation module is further configured to input the first total output tensor to a de-stacking operation node, and perform a de-stacking operation on tensors belonging to a tensor batch dimension in the first total output tensor through the de-stacking operation node to obtain N first output sub-tensors.

[0028] The first operation module comprises:

[0029] The second determination unit is configured to determine tensors belonging to a tensor row dimension in the front input tensor as first to-be-deformed tensors according to a row deformation operation attribute of the front input deformation operation node.

[0030] The first operation unit is configured to perform a deformation operation on the N first to-be-deformed tensors to obtain a first dimension-increased tensor, wherein a size of a first dimension corresponding to the first dimension-increased tensor is equal to N, and a size of the first to-be-deformed tensor is equal to N times a size of a second dimension corresponding to the first dimension-increased tensor.

[0031] The third updating unit is configured to update the first to-be-deformed tensors in the front input tensor to the first dimension-increased tensor to obtain a front input dimension-increased tensor.

[0032] The N sub-input tensors comprise N front input sub-tensors that have a size association relationship with the rear input tensor and respectively carry different serial numbers; the merging operation node comprises a stacking operation node, and the de-merging operation node comprises a de-stacking operation node associated with the stacking operation node.

[0033] The adding node module is specifically configured to, when the first deformation operation node is the rear input deformation operation node, add, before the total matrix multiplication operation node, a stacking operation node for performing a stacking operation on the N front input tensors, add a rear input transposition operation node between the total matrix multiplication operation node and the rear input deformation operation node, and add a de-stacking operation node after the total matrix multiplication operation node to obtain the optimized data flow graph.

[0034] The N front input sub-tensors respectively correspond to the same tensor dimension, and the N tensor dimensions respectively correspond to the same size; the tensor dimension corresponding to the rear input tensor is equal to the N tensor dimensions, and a size of a tensor row dimension corresponding to the rear input tensor is equal to sizes of tensor column dimensions respectively corresponding to the N front input sub-tensors.

[0035] The data flow graph processing apparatus further comprises:

[0036] The second operation module is configured to input the rear input tensor to a rear input deformation operation node in the optimized data flow graph, and perform a deformation operation on the rear input tensor through the rear input deformation operation node to obtain a rear input dimension-increased tensor.

[0037] The second operation module is further configured to input the post-input augmented dimension tensor to a post-input transposition operation node, perform a transposition operation on tensors in the post-input augmented dimension tensor that belong to a tensor dimension to be transposed through the post-input transposition operation node, and obtain a post-input transposed tensor; the tensor dimension to be transposed includes a tensor row dimension and a tensor batch dimension corresponding to the post-input augmented dimension tensor;

[0038] The second operation module is further configured to input the N pre-input sub-tensors to a stacking operation node in the optimized data flow graph in a sequence order, perform a stacking operation on the N pre-input sub-tensors through the stacking operation node, and obtain a pre-input stacked tensor; the pre-input stacked tensor includes a tensor batch dimension with a size equal to N, and sizes of tensor dimensions in the pre-input stacked tensor corresponding to the tensor dimensions except the tensor batch dimension with the size equal to N are equal to sizes of the N tensor dimensions respectively;

[0039] The second operation module is further configured to input the post-input transposed tensor and the pre-input stacked tensor to a total matrix multiplication operation node respectively, perform a matrix multiplication operation on the post-input transposed tensor and the pre-input stacked tensor through the total matrix multiplication operation node, and obtain a second total output tensor;

[0040] The second operation module is further configured to input the second total output tensor to a de-stacking operation node, perform a de-stacking operation on tensors in the second total output tensor that belong to a tensor batch dimension through the de-stacking operation node, and obtain N second output sub-tensors.

[0041] The second operation module includes:

[0042] The third determination unit is configured to determine, according to a column deformation operation attribute of the post-input deformation operation node, tensors in the post-input tensor that belong to a tensor column dimension as second tensors to be deformed;

[0043] The second operation unit is configured to perform a deformation operation on the N second tensors to be deformed to obtain a second augmented dimension tensor; a size of a first dimension corresponding to the second augmented dimension tensor is equal to N, and a size of the second tensor to be deformed is equal to N times a size of a second dimension corresponding to the second augmented dimension tensor;

[0044] The fourth update unit is configured to update the second tensor to be deformed in the post-input tensor as the second augmented dimension tensor to obtain a post-input augmented dimension tensor.

[0045] The N sub-input tensors include N post-input sub-tensors that have a size association relationship with the pre-input tensor and respectively carry different sequence numbers; the merging operation node includes a splicing operation node, and the de-merging operation node includes a de-splicing operation node associated with the splicing operation node;

[0046] The adding node module includes:

[0047] The first adding unit is configured to add the second deformation operation node before the total matrix multiplication operation node and add the splicing operation node for performing the splicing operation on the N post-input sub-tensors before the second deformation operation node when the first deformation operation node is the pre-input deformation operation node.

[0048] The second adding unit is configured to add the third deformation operation node associated with the second deformation operation node after the total matrix multiplication operation node and add the de-splicing operation node after the third deformation operation node to obtain the optimized data flow graph; and the node relationship between the second deformation operation node and the pre-input deformation operation node is a parallel node relationship.

[0049] The N post-input sub-tensors correspond to the same tensor dimensions, and the N tensor dimensions correspond to the same sizes; the pre-input tensor corresponds to N tensor dimensions, and the size of the tensor column dimension corresponding to the pre-input tensor is equal to the size of the tensor row dimension corresponding to each of the N post-input sub-tensors.

[0050] The data flow graph processing apparatus further includes:

[0051] The third operation module is configured to input the pre-input tensor into the pre-input deformation operation node in the optimized data flow graph, perform the deformation operation on the pre-input tensor through the pre-input deformation operation node, and obtain a pre-input dimension-increased tensor.

[0052] The third operation module is further configured to input the N post-input sub-tensors into the splicing operation node in the optimized data flow graph in the order of the sequence numbers, perform the splicing operation on the N post-input sub-tensors through the splicing operation node, and obtain a post-input spliced tensor; the post-input spliced tensor corresponds to N tensor dimensions.

[0053] The third operation module is further configured to input the post-input spliced tensor into the second deformation operation node, perform the deformation operation on the post-input spliced tensor through the second deformation operation node, and obtain a post-input deformed tensor.

[0054] The third operation module is further configured to input the pre-input dimension-increased tensor and the post-input deformed tensor into the total matrix multiplication operation node, perform the matrix multiplication operation on the pre-input dimension-increased tensor and the post-input deformed tensor through the total matrix multiplication operation node, and obtain a first to-be-deformed total output tensor.

[0055] The third operation module is further configured to input the first to-be-deformed total output tensor into the third deformation operation node, perform the deformation operation on the first to-be-deformed total output tensor through the third deformation operation node, and obtain a third total output tensor.

[0056] The third operation module is further configured to input the third total output tensor into a de-stitching operation node, and split tensors belonging to the tensor row dimension in the third total output tensor through the de-stitching operation node to obtain N first output sub-tensors.

[0057] The third operation module comprises:

[0058] The third operation unit is configured to perform a stitching operation on the N post-input sub-tensors in the tensor row dimension through a stitching operation node to obtain a post-input stitched tensor; a size of the tensor row dimension corresponding to the post-input stitched tensor is equal to a sum of sizes of the tensor row dimensions corresponding to the N post-input sub-tensors, respectively.

[0059] The third operation module comprises:

[0060] The fourth determination unit is configured to determine tensors belonging to the tensor row dimension in the post-input stitched tensor as third to-be-deformed tensors.

[0061] The fourth operation unit is configured to perform a deformation operation on the third to-be-deformed tensors through a second deformation operation node to obtain third dimension-increased tensors; a size of a first dimension corresponding to the third dimension-increased tensors is equal to N, and a size of the third to-be-deformed tensors is equal to N times a size of a second dimension corresponding to the third dimension-increased tensors.

[0062] The fifth update unit is configured to update the third to-be-deformed tensors in the post-input stitched tensor as the third dimension-increased tensors to obtain post-input deformed tensors.

[0063] The third operation module comprises:

[0064] The fifth determination unit is configured to determine tensors belonging to a tensor target dimension in the first to-be-deformed total output tensor as fourth to-be-deformed tensors; the tensor target dimension comprises a tensor batch dimension and a tensor row dimension corresponding to the first to-be-deformed total output tensor.

[0065] The fifth operation unit is configured to perform a deformation operation on the fourth to-be-deformed tensors through a third deformation operation node to obtain a target dimension-reduced tensor; a size of the target dimension-reduced tensor is equal to N times a size of a second dimension corresponding to the fourth to-be-deformed tensors.

[0066] The sixth update unit is configured to update the fourth to-be-deformed tensors in the first to-be-deformed total output tensor as the target dimension-reduced tensor to obtain the third total output tensor.

[0067] The N sub-input tensors comprise N pre-input sub-tensors that have a size association relationship with the post-input tensor and carry different serial numbers, respectively; the merging operation node comprises a stitching operation node, and the de-merging operation node comprises a de-stitching operation node associated with the stitching operation node.

[0068] The adding node module comprises:

[0069] The third adding unit is configured to add a fourth deformation operation node before the total matrix multiplication operation node and add a splicing operation node for splicing the N front input sub-tensors before the fourth deformation operation node when the first deformation operation node is a back input deformation operation node.

[0070] The fourth adding unit is configured to add a back input transposition operation node between the total matrix multiplication operation node and the back input deformation operation node.

[0071] The fifth adding unit is configured to add a fifth deformation operation node associated with the fourth deformation operation node after the total matrix multiplication operation node and add a de-splicing operation node after the fifth deformation operation node to obtain the optimized data flow graph; and the node relationship between the fourth deformation operation node and the back input deformation operation node is a parallel node relationship.

[0072] The N front input sub-tensors correspond to the same tensor dimensions, and the N tensor dimensions correspond to the same sizes; the back input tensor corresponds to N tensor dimensions, and the size of the tensor row dimension of the back input tensor is equal to the size of the tensor column dimension of the N front input sub-tensors.

[0073] The data flow graph processing apparatus further comprises:

[0074] The fourth operation module is configured to input the back input tensor into the back input deformation operation node in the optimized data flow graph, perform a deformation operation on the back input tensor through the back input deformation operation node, and obtain a back input dimension-increased tensor.

[0075] The fourth operation module is further configured to input the back input dimension-increased tensor into the back input transposition operation node, perform a transposition operation on the tensors belonging to the tensor dimensions to be transposed in the back input dimension-increased tensor through the back input transposition operation node, and obtain a back input transposed tensor; the tensor dimensions to be transposed include the tensor row dimension and the tensor batch dimension corresponding to the back input dimension-increased tensor.

[0076] The fourth operation module is further configured to input the N front input sub-tensors into the splicing operation node in the optimized data flow graph in the order of the sequence numbers, perform a splicing operation on the N front input sub-tensors through the splicing operation node, and obtain a first front input spliced tensor; the first front input spliced tensor corresponds to N tensor dimensions.

[0077] The fourth operation module is further configured to input the first pre-input spliced tensor into the fourth deformation operation node, perform a deformation operation on the first pre-input spliced tensor through the fourth deformation operation node, and obtain a first pre-input deformation tensor;

[0078] The fourth operation module is further configured to input the first pre-input deformation tensor and the post-input transposed tensor into the total matrix multiplication operation node, perform a matrix multiplication operation on the first pre-input deformation tensor and the post-input transposed tensor through the total matrix multiplication operation node, and obtain a second total output tensor to be deformed;

[0079] The fourth operation module is further configured to input the second total output tensor to be deformed into the fifth deformation operation node, perform a deformation operation on the second total output tensor to be deformed through the fifth deformation operation node, and obtain a fourth total output tensor;

[0080] The fourth operation module is further configured to input the fourth total output tensor into the de-splicing operation node, perform a splitting operation on the tensors belonging to the tensor row dimension in the fourth total output tensor through the de-splicing operation node, and obtain N second output sub-tensors.

[0081] The fourth operation module includes:

[0082] The sixth operation unit is configured to perform a splicing operation on the N pre-input sub-tensors in the tensor row dimension through the splicing operation node, and obtain a first pre-input spliced tensor; the size of the tensor row dimension corresponding to the first pre-input spliced tensor is equal to the sum of the sizes of the tensor row dimensions corresponding to the N pre-input sub-tensors, respectively.

[0083] The fourth operation module includes:

[0084] The sixth determination unit is configured to determine the tensors belonging to the tensor row dimension in the first pre-input spliced tensor as a fifth tensor to be deformed.

[0085] The seventh operation unit is configured to perform a deformation operation on the fifth tensor to be deformed through the fourth deformation operation node, and obtain a fourth dimension-increased tensor; the size of the first dimension corresponding to the fourth dimension-increased tensor is equal to N, and the size of the fifth tensor to be deformed is equal to N times the size of the second dimension corresponding to the fourth dimension-increased tensor.

[0086] The seventh update unit is configured to update the fifth tensor to be deformed in the first pre-input spliced tensor as the fourth dimension-increased tensor, and obtain a first pre-input deformation tensor.

[0087] The fourth operation module includes:

[0088] The seventh determining unit is configured to determine a tensor belonging to a tensor target dimension in the second total output tensor to be deformed as a sixth tensor to be deformed; the tensor target dimension includes a tensor batch dimension and a tensor row dimension corresponding to the second total output tensor to be deformed;

[0089] The eighth operation unit is configured to perform a deformation operation on the sixth tensor to be deformed through the fifth deformation operation node to obtain a first reduced dimension tensor; a size of the first reduced dimension tensor is equal to N times of a size of the second dimension corresponding to the sixth tensor to be deformed;

[0090] The eighth updating unit is configured to update the sixth tensor to be deformed in the second total output tensor to be deformed as the first reduced dimension tensor to obtain a fourth total output tensor.

[0091] The N front input sub-tensors correspond to the same tensor dimension, and the N tensor dimensions correspond to the same size.

[0092] The data flow graph processing apparatus further includes:

[0093] The fifth operation module is configured to delete the post-input transposition operation node in the optimized data flow graph to obtain a target optimized data flow graph when the total matrix multiplication operation node has an operation permission for the tensor whose tensor batch dimension belongs to the intermediate dimension.

[0094] The fifth operation module is further configured to input the post-input tensor into the post-input deformation operation node in the target optimized data flow graph, and perform a deformation operation on the post-input tensor through the post-input deformation operation node to obtain a post-input dimension-increased tensor.

[0095] The fifth operation module is further configured to input the N front input sub-tensors into the splicing operation node in the target optimized data flow graph in a sequence order, and perform a splicing operation on the N front input sub-tensors through the splicing operation node to obtain a second front input spliced tensor; the second front input spliced tensor corresponds to the N tensor dimensions; and a size of a tensor column dimension corresponding to the second front input spliced tensor is equal to a sum of sizes of tensor column dimensions corresponding to the N post-input sub-tensors.

[0096] The fifth operation module is further configured to input the second front input spliced tensor into the fourth deformation operation node, and perform a deformation operation on the second front input spliced tensor through the fourth deformation operation node to obtain a second front input deformation tensor.

[0097] The fifth operation module is further configured to input the second pre-input deformation tensor and the post-input dimension-increased tensor into a total matrix multiplication operation node, perform matrix multiplication operation on the second pre-input deformation tensor and the post-input dimension-increased tensor through the total matrix multiplication operation node, and obtain a third total output tensor to be deformed;

[0098] The fifth operation module is further configured to input the third total output tensor to be deformed into a fifth deformation operation node, perform deformation operation on the third total output tensor to be deformed through the fifth deformation operation node, and obtain a fifth total output tensor;

[0099] The fifth operation module is further configured to input the fifth total output tensor into a de-splicing operation node, perform splitting operation on the tensors belonging to the tensor column dimension in the fifth total output tensor through the de-splicing operation node, and obtain N second output sub-tensors.

[0100] The fifth operation module comprises:

[0101] The ninth operation unit is configured to perform splicing operation on the N pre-input sub-tensors in the tensor column dimension through a splicing operation node to obtain a second pre-input splicing tensor; the size of the tensor column dimension corresponding to the second pre-input splicing tensor is equal to the sum of the sizes of the tensor column dimensions corresponding to the N post-input sub-tensors, respectively;

[0102] The fifth operation module comprises:

[0103] The eighth determination unit is configured to determine the tensors belonging to the tensor column dimension in the second pre-input splicing tensor as a seventh total output tensor to be deformed;

[0104] The ninth operation unit is further configured to perform deformation operation on the seventh total output tensor to be deformed through a fourth deformation operation node to obtain a fifth dimension-increased tensor; the size of the first dimension corresponding to the fifth dimension-increased tensor is equal to N, and the size of the seventh total output tensor to be deformed is equal to N times the size of the second dimension corresponding to the fifth dimension-increased tensor;

[0105] The ninth updating unit is configured to update the seventh total output tensor to be deformed in the second pre-input splicing tensor as the fifth dimension-increased tensor to obtain a second pre-input deformation tensor.

[0106] The fifth operation module comprises:

[0107] The ninth determination unit is configured to determine the tensors belonging to a tensor target dimension in the third total output tensor to be deformed as an eighth total output tensor to be deformed; the tensor target dimension comprises a tensor batch dimension and a tensor column dimension corresponding to the sixth total output tensor to be deformed;

[0108] The tenth operation unit is configured to perform a deformation operation on the eighth to-be-deformed tensor through a fifth deformation operation node to obtain a second reduced dimension tensor, wherein a size of the second reduced dimension tensor is equal to N times of a size of the second dimension corresponding to the eighth to-be-deformed tensor.

[0109] The tenth updating unit is configured to update the eighth to-be-deformed tensor in the third to-be-deformed total output tensor to the second reduced dimension tensor to obtain a fifth total output tensor.

[0110] In an aspect, the present application provides a computer device, comprising: a processor, a memory, and a network interface.

[0111] The processor is connected to the memory and the network interface, wherein the network interface is configured to provide a data communication function, the memory is configured to store a computer program, and the processor is configured to call the computer program to enable the computer device to execute the method in the embodiments of the present application.

[0112] In an aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is adapted to be loaded by a processor and execute the method in the embodiments of the present application.

[0113] In an aspect, the present application provides a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium; a processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to execute the method in the embodiments of the present application.

[0114] In the embodiments of the present application, by updating the N sub-matrix multiplication operation nodes in the original data flow graph to the total matrix multiplication operation node, the utilization rate of system resources can be improved; by updating the splitting operation node to the first deformation operation node for deforming the total input tensor, the total input tensor can be deformed into a tensor suitable for the total matrix multiplication operation node; further, by adding the merging operation node for merging the N sub-input tensors before the total matrix multiplication operation node, the N sub-input tensors can be merged into a tensor suitable for the total matrix multiplication operation node; by adding the de-merging operation node after the total matrix multiplication operation node, the data results of the original data flow graph can be maintained and the optimized data flow graph can be obtained. As can be seen above, the optimized data flow graph obtained by optimizing the original data flow graph can not only maintain the data results of the original data flow graph, but also improve the utilization rate of system resources. BRIEF DESCRIPTION OF DRAWINGS

[0115] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced. Obviously, the accompanying drawings in the following description only only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0116] Figure 1 is a system architecture schematic diagram provided by an embodiment of the present application;

[0117] Figure 2 is a flowchart of a data flow graph processing method provided by an embodiment of the present application;

[0118] Figure 3a is a structure schematic diagram of an original data flow graph provided by an embodiment of the present application;

[0119] Figure 3b is a structure schematic diagram of an original data flow graph provided by an embodiment of the present application;

[0120] Figure 4 is a flowchart of a data flow graph processing method provided by an embodiment of the present application;

[0121] Figure 5 is a structure schematic diagram of an optimized data flow graph provided by an embodiment of the present application;

[0122] Figure 6 is a flowchart of a data flow graph processing method provided by an embodiment of the present application;

[0123] Figure 7 is a structure schematic diagram of an optimized data flow graph provided by an embodiment of the present application;

[0124] Figure 8 is a flowchart of a data flow graph processing method provided by an embodiment of the present application;

[0125] Figure 9 is a structure schematic diagram of an optimized data flow graph provided by an embodiment of the present application;

[0126] Figure 10 is a flowchart of a data flow graph processing method provided by an embodiment of the present application;

[0127] Figure 11 is a structure schematic diagram of an optimized data flow graph provided by an embodiment of the present application;

[0128] Figure 12 is a flowchart of a data flow graph processing method provided by an embodiment of the present application;

[0129] Figure 13 is a structural schematic diagram of a target optimization data flow graph provided by an embodiment of the present application;

[0130] Figure 14 is a structural schematic diagram of a data flow graph processing apparatus provided by an embodiment of the present application;

[0131] Figure 15 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0132] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0133] For the convenience of understanding, first, some nouns are simply explained as follows:

[0134] Artificial intelligence (AI) is to use digital computers or digital computer controlled machine simulation, extension and expansion of human intelligence, perception of the environment, knowledge acquisition and use of knowledge to obtain the best results of theory, method, technology and application system. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0135] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning and other several major directions.

[0136] Computer Vision (CV) Computer vision is a scientific research on how to make the machine "see", and further, it refers to using cameras and computers to replace human eyes to identify, track and measure targets and other machine vision, and further do image processing, so that the computer processing becomes more suitable for human eye observation or image transmission to the instrument detection. As a scientific discipline, computer vision researches related theories and technologies, and tries to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and other technologies, and also includes common face recognition, fingerprint recognition and other biometric identification technologies.

[0137] Machine Learning (ML) is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It is a special research on how computers simulate or implement human learning behavior to obtain new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent, and its application is widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rule-based learning.

[0138] The scheme provided by the embodiments of the present application relates to computer vision technology and machine learning technology of artificial intelligence, and is specifically explained through the following embodiments.

[0139] Please refer to Figure 1 , Figure 1 is a system architecture schematic diagram provided by the embodiments of the present application. As Figure 1 shown, the system can include a server 1000a and a user terminal cluster, which can include: a user terminal 100a, a user terminal 100b, a user terminal 100c,..., a user terminal 100n. It can be understood that the above system can include one or more user terminals, and the number of user terminals will not be limited here.

[0140] Among them, there can be a communication connection between the user terminal clusters, for example, there is a communication connection between the user terminal 100b and the user terminal 100c, and there is a communication connection between the user terminal 100b and the user terminal 100n. At the same time, any user terminal in the user terminal cluster can have a communication connection with the server 1000a, for example, there is a communication connection between the user terminal 100b and the server 1000a, and there is a communication connection between the user terminal 100c and the server 1000a. Among them, the above-mentioned communication connection is not limited to the connection mode, which can be connected directly or indirectly through wired communication mode, or directly or indirectly through wireless communication mode, or through other ways, which are not limited by the present application.

[0141] It should be understood that each user terminal in the user terminal cluster as shown in Figure 1 may be installed with an application client, which can respectively interact with the server 1000a as shown in Figure 1 above when running in each user terminal, that is, the above-mentioned communication connection. Among them, the application client can be a social client, a multimedia client (such as a video client), an entertainment client (such as a game client), an education client, a live client and other application clients with data flow chart loading function. Among them, the application client can be an independent client, or an embedded sub-client integrated in a certain client (such as a social client, an education client and a multimedia client, etc.), which is not limited here.

[0142] The server 1000a provides services for the user terminal cluster through the communication connection function. When the user terminal (which can be the user terminal 100b, the user terminal 100c or the user terminal 100n) obtains the original data flow chart and needs to optimize the original data flow chart, the user terminal can send the original data flow chart to the server 1000a. After the server 1000a receives the original data flow chart sent by the user terminal, it optimizes the original data flow chart to obtain an optimized data flow chart, and returns the optimized data flow chart to the user terminal. The process of optimizing the original data flow chart by the server 1000a will be described in the following Figure 2 corresponding embodiments, which will not be described here for the time being.

[0143] After obtaining the optimized data flow graph, the user terminal can apply it to tensor data flow analysis optimization scenarios. For example, in the scenario of optimizing neural network models, the optimized data flow graph can be integrated into AI platforms and AI frameworks. It can also be applied to tensor graph optimization engines, such as graph optimization engines in deep learning, such as the Grappler computation graph optimization engine used in Tensorflow (a system that transmits complex data structures to artificial intelligence neural networks for analysis and processing).

[0144] It is understood that the aforementioned server 1000a can be a stream computing platform, which can include, for example, the neural network model and tensor graph optimization engine described above. Users upload the original data flow graph to the stream computing platform, which can then use the optimization methods described in this solution to optimize the original data flow graph, resulting in an optimized data flow graph. This optimized data flow graph is then deployed to various frameworks for application.

[0145] Optionally, if the user terminal has the optimization method stored locally, the original data flow graph can be optimized on the user terminal, and the optimization process is consistent with the optimization process performed by server 1000a on the original data flow graph. The optimization method in the user terminal can be sent to the user terminal by server 1000a.

[0146] in, Figure 1 Server 1000a, user terminal 100b, user terminal 100c, ..., user terminal 100n can all include mobile phones, tablets, laptops, PDAs, smart speakers, mobile internet devices (MIDs), POS (Point of Sales) machines, wearable devices (such as smartwatches, smart bracelets, etc.).

[0147] It should be noted that the aforementioned server 1000a, user terminal 100b, user terminal 100c, ..., user terminal 100n can all be blockchain nodes in the blockchain network. The data described in the entire text (such as the original data flow diagram and the optimized data flow diagram mentioned above) can be stored. The storage method can be that the blockchain node generates blocks based on the data and adds the blocks to the blockchain for storage.

[0148] The blockchain is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism and encryption algorithm, and is mainly used for arranging data in chronological order and encrypting the data into a ledger so that the data cannot be tampered with and forged, and the data can be verified, stored and updated. The blockchain is essentially a decentralized database, each node in the database stores a same blockchain, and the blockchain network can distinguish the nodes into core nodes, data nodes and light nodes. The core nodes, data nodes and light nodes together constitute the blockchain nodes. The core nodes are responsible for the consensus of the whole blockchain network, that is, the core nodes are consensus nodes in the blockchain network. The process of writing transaction data in the ledger in the blockchain network can be that the data nodes or light nodes in the blockchain network obtain the transaction data, deliver the transaction data in the blockchain network (that is, the nodes deliver the transaction data in the manner of a baton), until the consensus node receives the transaction data, and the consensus node packages the transaction data into a block, performs consensus on the block, and writes the transaction data into the ledger after the consensus is completed. Here, the transaction data is illustrated by an original data flow diagram and an optimized data flow diagram. After the server 1000a (a blockchain node) performs consensus on the transaction data, the server 1000a generates a block according to the transaction data, and stores the block into the blockchain network; and for reading of the transaction data (that is, the original data flow diagram and the optimized data flow diagram), the blockchain node can obtain the block containing the transaction data in the blockchain network, and further obtain the transaction data in the block.

[0149] It can be understood that the data flow diagram processing method provided by the embodiments of the present application can be executed by a computer device, which includes but is not limited to a user terminal or a server. The server can be a stand-alone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms. The user terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle terminal, a smart television and the like, but is not limited thereto. The user terminal and the server can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application.

[0150] Further, please refer to Figure 2 , Figure 2 is a flow diagram of a data flow diagram processing method provided by the embodiments of the present application. The data flow diagram processing method can be executed by a computer device, which can include Figure 1 a user terminal and a server in Figure 2As shown, the data flow graph processing process can include the following steps.

[0151] In step S101, an original data flow graph is acquired and parsed.

[0152] Specifically, in this application, the original data flow graph can be a flow graph written by a user, which can include data streams and data operation nodes. Among them, the data stream (data stream) is a sequence of ordered, starting and ending byte data, including input stream (input data) and output stream (output data). Data flow can be used to represent data flowing from one data operation node to another, such as output data processed by node a, input into node b, then the data flowing to node b (output data processed by node a) can be used as data flow (output data of node a and input data of node b). The data operation node can receive input data, process the input data according to the operation function, and generate output data.

[0153] In the embodiments of the present application, data is described as a tensor, which can be understood as an n-dimensional array or matrix, n is a non-negative integer, wherein the zero-dimensional tensor is a real number, for example, 5; the one-dimensional tensor is a vector, for example, the vector [5, 5] with a length of 2; the two-dimensional tensor is a matrix, for example, the matrix [[5, 5, 5], [5, 5, 5]] with 2 rows and 3 columns. The shape corresponding to the matrix can be represented as (2, 3), wherein 2 represents the size of the row dimension, and 3 represents the size of the column dimension. Generally, a tensor specifically refers to an array of 3 dimensions and above, for example, a 3-row 2-column 2-depth tensor, [[1, 2], [3, 4]], [[5, 6], [7, 8]], [[11, 12], [13, 14]]]. The shape corresponding to the tensor can be represented as (2, 3, 2), wherein the first 2 can represent the size of the depth dimension, and the second 2 can represent the size of the column dimension. It should be noted that in the embodiments of the present application, the tensor shape expression, such as the tensor shape (2, 3, 2), the size of the batch dimension (in addition to the row dimension and the column dimension) is placed in front of the size of the row dimension, and the size of the row dimension is placed in front of the size of the column dimension.

[0154] In step S102, if it is parsed that the original data flow graph includes a splitting operation node and N sub-matrix multiplication operation nodes, the position of the splitting operation node in the original data flow graph is detected; N is a positive integer, and N is equal to or greater than 2.

[0155] Specifically, a split operation is a tensor operation, and the application splits a tensor into multiple tensors through the split operation. A matrix multiplication (Matmul) operation is a tensor operation, and the application multiplies each element of an i-th row of a first tensor (a front input tensor) with an element at a corresponding position of a k-th column of a second tensor (a back input tensor) and accumulates the results one by one, which is recorded as an output result of an i-th row and a j-th column of an output tensor.

[0156] Please refer to Figure 3a , Figure 3a is a structural schematic diagram of an original data flow graph provided by an embodiment of the application. As shown in Figure 3a , the original data flow graph includes a front input split operation node 301a (which can be understood as a data operation node for performing a split operation on a total input tensor (A, B)), and N matrix multiplication operation nodes, namely a matrix multiplication operation node 1, a matrix multiplication operation node 2, …, and a matrix multiplication operation node N as shown in Figure 3a . In Figure 3a , the front input split operation node 301a is configured to perform a split operation on the total input tensor (A, B). In the embodiment of the application, the symbol “()” represents the shape of a tensor. Taking (A, B) as an example, other tensors or tensor shapes can be understood in the same way. The total number of characters in the symbol “()” can represent the total number of tensor dimensions of the tensor. The characters A and B are two characters, so the tensor is a two-dimensional tensor (i.e., a matrix). The character A can represent the size of the row dimension of the tensor, and the character B can represent the size of the column dimension of the tensor. If there are other dimensions, they can be understood as batch dimensions, which are placed in front of the row dimension and the column dimension by default.

[0157] In Figure 3a the original data flow graph shown in the example, the input data of the front input split operation node 301a is the total input tensor (A, B), and the output data can include N front input tensors, namely a front input tensor 1 (a, B), a front input tensor 2 (a, B), …, and a front input tensor N (a, B) in Figure 3a . Further, the front input tensor 1 (a, B) is input as the first input data to the matrix multiplication operation node 1, the back input tensor 1 (B, C) is input as the second input data, and the output data is an output result 1 (a, C). Similarly, the front input tensor 2 (a, B) is input as the first input data to the matrix multiplication operation node 2, the back input tensor 2 (B, C) is input as the second input data, and the output data is an output result 2 (a, C), …, the front input tensor N (a, B) is input as the first input data to the matrix multiplication operation node N, the back input tensor N (B, C) is input as the second input data, and the output data is an output result N (a, C).

[0158] It should be noted that, in Figure 3a , the tensors are represented by rectangles and the nodes are represented by ovals, so the word "tensor" and the word "node" are not shown in Figure 3a . Similarly, in the following Figure 3b , Figure 5 , Figure 7 , Figure 9 , Figure 11 and Figure 13 , the tensors are represented by rectangles and the nodes are represented by ovals.

[0159] Please refer to Figure 3b , Figure 3b for a structural schematic diagram of a raw data flow graph provided by an embodiment of the present application. As shown in Figure 3b , the raw data flow graph includes a post-input splitting operation node 301c (which can be understood as a data operation node for performing a splitting operation on a total input tensor (B, C)), and N matrix multiplication operation nodes, i.e., the matrix multiplication operation node 1, the matrix multiplication operation node 2, …, and the matrix multiplication operation node N shown in Figure 3b .

[0160] In the raw data flow graph shown in Figure 3b , the input data of the post-input splitting operation node 301c is the total input tensor (B, C), and the output data can include N post-input tensors, i.e., the post-input tensor 1 (B, c), the post-input tensor 2 (B, c), …, and the post-input tensor N (B, c) in Figure 3b . Further, the post-input tensor 1 (B, c) is input to the matrix multiplication operation node 1 as the second input data with the post-input tensor 1 (A, B) as the first input data, and the output data is the output result 1 (A, c); similarly, the post-input tensor 2 (B, c) is input to the matrix multiplication operation node 2 as the second input data with the post-input tensor 2 (A, B) as the first input data, and the output data is the output result 2 (A, c), …, and the post-input tensor N (B, c) is input to the matrix multiplication operation node N as the second input data with the post-input tensor N (A, B) as the first input data, and the output data is the output result N (A, c).

[0161] In combination with Figure 3a and Figure 3bIt can be known that the original data flow is generated by splitting the total input tensor through the split operation node to generate a plurality of sub-input tensors (which can include pre-sub-input tensors or post-sub-input tensors), and then performing matrix multiplication operation on the plurality of sub-input tensors (which can include post-sub-input tensors or pre-sub-input tensors). Obviously, in the original data flow graph, there are a plurality of same matrix multiplication operations, and repeated data operations will generate a large number of allocated resources, thereby reducing the utilization rate of system resources. Based on the above defects, the original data flow graph is optimized, and details are described in steps S103-S104.

[0162] In step S103, if the split operation node is connected with N sub-matrix multiplication operation nodes in the original data flow graph, and the position of the split operation node in the original data flow graph is before the N sub-matrix multiplication operation nodes, the N sub-matrix multiplication operation nodes are updated to a total matrix multiplication operation node, and the split operation node is updated to a first deformation operation node for deforming the total input tensor.

[0163] Specifically, the first deformation operation node includes a pre-input deformation operation node or a post-input deformation operation node; the total input tensor includes a pre-input tensor or a post-input tensor; the input attribute of the split operation node is determined for the N sub-matrix multiplication operation nodes, and the split operation attribute of the split operation node; if the input attribute is a pre-input attribute, and the split operation attribute is a row split operation attribute, the split operation node is updated to a pre-input deformation operation node for deforming the pre-input tensor; the deformation operation attribute of the pre-input deformation operation node is a row deformation operation attribute; if the input attribute is a post-input attribute, and the split operation attribute is a column split operation attribute, the split operation node is updated to a post-input deformation operation node for deforming the post-input tensor; the deformation operation attribute of the post-input deformation operation node is a column deformation operation attribute.

[0164] Please refer to Figure 3a , Figure 3a The original data flow graph shown in FIG. 3 includes a pre-input split operation node 301a and N sub-matrix multiplication operation nodes, and the output data (including Figure 3aThe preceding input tensors 1(a, B), 2(a, B), ..., N(a, B) serve as the input data for N sub-matrix multiplication operation nodes. Clearly, preceding input tensor 1(a, B) serves as the preceding input data (i.e., the first input data) for matrix multiplication operation node 1, preceding input tensor 2(a, B) serves as the preceding input data for matrix multiplication operation node 2, ..., preceding input tensor N(a, B) serves as the preceding input data for matrix multiplication operation node N. Therefore, the computer device can determine the total input tensor (A, B) as the preceding input tensor. At this time, the computer device can update the N sub-matrix multiplication operation nodes to the total matrix multiplication operation node, that is, aggregate multiple identical matrix multiplication operations into a single total matrix multiplication operation.

[0165] exist Figure 3a In the process, the pre-input split operation node 301a performs a uniform split on the size (i.e., A) of the tensor row dimension corresponding to the total input tensor. That is, in the numerical representation, a*N=A. Therefore, the computer device can determine that the split operation attribute of the pre-input split operation node 301a is the row split operation attribute. At this time, the computer device updates the pre-input split operation node 301a to a pre-input deformation operation node used to perform deformation operations on the pre-input tensor.

[0166] Please see again. Figure 3b , Figure 3b The example original data flow graph includes a post-input split operation node 301c and N submatrix multiplication operation nodes, and the output data of the post-input split operation node 301c (including...) Figure 3b The following tensors are used as input data for N sub-matrix multiplication operation nodes: post-input tensor 1(B, c), post-input tensor 2(B, c), ..., post-input tensor N(B, c). Clearly, post-input tensor 1(B, c) serves as the post-input data (i.e., the second input data) for matrix multiplication operation node 1, post-input tensor 2(B, c) serves as the post-input data for matrix multiplication operation node 2, ..., post-input tensor N(B, c) serves as the post-input data for matrix multiplication operation node N. Therefore, the computer device can determine that the total input tensor (B, c) is the post-input tensor. At this point, the computer device can update the N sub-matrix multiplication operation nodes to the total matrix multiplication operation node, that is, aggregate multiple identical matrix multiplication operations into a single total matrix multiplication operation.

[0167] exist Figure 3bIn this way, the post-input split operation node 301c performs an even split on the size of the tensor column dimension corresponding to the total input tensor, that is, c*N=C in a digital representation, so that the computer device can determine that the split operation attribute of the post-input split operation node 301c is a column split operation attribute, and at this time, the computer device updates the post-input split operation node 301c to a post-input deformation operation node for performing a deformation operation on the post-input tensor.

[0168] In step S104, a merge operation node for performing a merge operation on the N sub-input tensors is added before the total matrix multiplication operation node, and a de-merge operation node is added after the total matrix multiplication operation node to obtain an optimized data flow graph. The node relationship between the merge operation node and the first deformation operation node is a parallel node relationship. There is a size association relationship between the total input tensor and the N sub-input tensors.

[0169] Specifically, the merge operation is a tensor operation. The present application merges multiple tensors into one tensor through the merge operation. The merge operation includes two modes of a concatenate operation and a stack operation. The concatenate operation does not generate a new dimension, but only merges in the existing dimension. The stack operation generates a new dimension. The concatenate operation can be performed on any dimension, and the only constraint is that the length of the non-merge dimension must be consistent. The stack operation needs the shapes of all tensors to be merged to be completely consistent.

[0170] According to the description of step S103, Figure 3a and Figure 3b , the embodiment of the present application aggregates the N sub-matrix multiplication operation nodes into one total matrix multiplication operation node, so that the N sub-input tensors need to be merged. Therefore, the computer device adds a merge operation node before the total matrix multiplication node. Since the output result of the original data flow graph includes N, a de-merge operation node is added after the total matrix multiplication node to obtain an optimized data flow graph. The structure diagram of the optimized data flow graph is described below. Figure 5 , Figure 7 , Figure 9 , Figure 11 and Figure 13 , which will not be described here.

[0171] In combination with steps S101-S104, it can be known that the embodiment of the present application proposes a new method, supports aggregating the MatMul operation nodes corresponding to the plurality of sub-tensors into the MatMul operation corresponding to the total tensor, because the allocation resource of one node is less than the allocation resource of the plurality of nodes, therefore, the utilization of the system resource can be improved without changing the output result of the original data flow graph, and the system can be efficiently run on the GPU.

[0172] In the embodiment of the present application, by updating the N sub-matrix multiplication operation nodes in the original data flow graph into the total matrix multiplication operation node, the utilization of the system resource can be improved; by updating the split operation node into the first deformation operation node for deforming the total input tensor, the total input tensor can be deformed into the tensor suitable for the total matrix multiplication operation node; further, by adding the merging operation node for merging the N sub-input tensors before the total matrix multiplication operation node, the N sub-input tensors can be merged into the tensor suitable for the total matrix multiplication operation node; by adding the de-merging operation node after the total matrix multiplication operation node, the data result of the original data flow graph can be maintained and the optimized data flow graph can be obtained. As known above, the optimized data flow graph obtained by the optimization processing of the original data flow graph can not only maintain the data result of the original data flow graph, but also improve the utilization of the system resource.

[0173] Further, please refer to Figure 4 , Figure 4 is a flow diagram of a data flow graph processing method provided by the embodiment of the present application. The data flow graph processing method can be executed by a computer device, which can include Figure 1 the user terminal and the server in the Figure 4 As shown in the Figure 4 , the data flow graph processing process can include the following steps.

[0174] Step S201, obtaining an original data flow graph, and parsing the original data flow graph.

[0175] Step S202, if the original data flow graph includes the split operation node and the N sub-matrix multiplication operation nodes, detecting the position of the split operation node in the original data flow graph; N is a positive integer, and N is equal to or greater than 2.

[0176] Step S203, if the splitting operation node is connected with the N sub-matrix multiplication operation nodes respectively in the original data flow graph, and the position of the splitting operation node in the original data flow graph is before the N sub-matrix multiplication operation nodes, the N sub-matrix multiplication operation nodes are updated to a total matrix multiplication operation node, and the splitting operation node is updated to a first deformation operation node for performing a deformation operation on a total input tensor.

[0177] The specific process of steps S201 to S203 can be referred to the foregoing description Figure 2 The description of steps S101 to S103 in the corresponding embodiment is not repeated here.

[0178] Step S204, when the first deformation operation node is a front input deformation operation node, a stacking operation node for performing a stacking operation on N back input sub-tensors is added before the total matrix multiplication operation node, and a de-stacking operation node is added after the total matrix multiplication operation node, to obtain an optimized data flow graph.

[0179] Specifically, the N sub-input tensors include N back input sub-tensors having a size association relationship with the front input tensor and carrying different serial numbers respectively; the merging operation node includes a stacking operation node, and the de-merging operation node includes a de-stacking operation node associated with the stacking operation node.

[0180] Please refer to Figure 5 , Figure 5 is a structural schematic diagram of an optimized data flow graph provided by an embodiment of the present application. As shown in Figure 5 , the first deformation operation node is a front input deformation operation node, that is, the deformation operation node 501a for a front input shown in Figure 5 , the total input tensor (A, B) is a front input tensor, and the N sub-input tensors are all back input sub-tensors, where the meaning of the symbol "()", the character A, the character B, and the character C correspond to the meanings described above Figure 2 in step S102 in the foregoing description, which is not repeated here.

[0181] As shown in Figure 5 , the computer device adds a stacking operation node 501b for performing a stacking operation on the N back input sub-tensors before the total matrix multiplication operation node 501c, and adds a de-stacking operation node 501d after the total matrix multiplication operation node 501c, to obtain Figure 5 the optimized data flow graph shown in

[0182] Step S205, inputting the front input tensor to the front input deformation operation node in the optimized data flow graph, and performing a deformation operation on the front input tensor through the front input deformation operation node to obtain a front input dimension-increased tensor.

[0183] Specifically, the N post-input sub-tensors correspond to tensor dimensions of the same size, and the N tensor dimensions correspond to sizes of the same size; the pre-input tensor corresponds to N tensor dimensions, and the size of the tensor column dimension corresponding to the pre-input tensor is equal to the size of the tensor row dimension corresponding to the N post-input sub-tensors.

[0184] According to the row deformation operation attribute of the pre-input deformation operation node, the tensors belonging to the tensor row dimension in the pre-input tensor are determined as the first tensors to be deformed; the N first tensors to be deformed are deformed to obtain the first dimension-increased tensor; the size of the first dimension corresponding to the first dimension-increased tensor is equal to N, and the size of the first dimension-increased tensor is equal to N times the size of the second dimension corresponding to the first dimension-increased tensor; the first dimension-increased tensor is updated to the first dimension-increased tensor in the pre-input tensor to obtain the pre-input dimension-increased tensor.

[0185] Please refer to Figure 5 , the computer device inputs the pre-input tensor (equivalent to the total input tensor (A, B) in Figure 5 ) as input data to the pre-input deformation operation node (equivalent to the deformation operation node 501a in Figure 5 ), and deforms the tensors corresponding to the tensor row dimension of the pre-input tensor according to the row deformation operation attribute of the deformation operation node 501a, that is, deforms the size A shown in Figure 5 . The deformation operation here can be understood as dimension increasing operation, that is, the size A of the tensor row dimension is deformed into N of the tensor batch dimension and the size a of the tensor row dimension, and a*N=A, so the output data of the deformation operation node 501a is the pre-input dimension-increased tensor, which is equivalent to the pre-input tensor (N, a, B) in Figure 5 .

[0186] In step S206, the N post-input sub-tensors are input to the stacking operation node in the optimized data flow graph in sequence, and the N post-input sub-tensors are stacked by the stacking operation node to obtain the post-input stacked tensor; the post-input stacked tensor includes a tensor batch dimension with a size equal to N, and the size of the tensor dimension corresponding to the post-input stacked tensor, except for the tensor batch dimension with a size equal to N, is equal to the size corresponding to the N tensor dimensions.

[0187] Specifically, it can be understood that in the matrix multiplication operation and the stacking operation, the input data has an input order, for example, the matrix multiplication operation is performed with the matrix M as the first input data (i.e. pre-input data) and the matrix K as the second input data (i.e. post-input data), which is not equivalent to the matrix multiplication operation with the matrix M as the second input data and the matrix K as the first input data.

[0188] In the embodiment of the present application, Arabic numerals (i.e. 1, 2, …, N) are taken as an example to illustrate the serial numbers. In actual application scenarios, the serial numbers corresponding to the N post-input sub-tensors can be limited according to the scenarios, and are unique, and the serial number order should also be set according to the scenarios, which is not limited in the embodiment of the present application.

[0189] Please refer to 5 again. The computer device inputs the N post-input sub-tensors to the stacking operation node 501b according to the serial numbers. The stacking operation is a kind of tensor operation, which can combine multiple tensors into one tensor, and the stacking operation will increase the dimension, as shown in Figure 5 Through the stacking operation node 501b, the computer device performs a stacking operation on the N post-input sub-tensors to obtain a post-input stacked tensor, that is, the post-input tensor (N, B, C) shown in Figure 5 Obviously, the post-input stacked tensor has one more tensor batch dimension than the post-input sub-tensor, and the size of the tensor batch dimension is equal to N.

[0190] In step S207, the pre-input increased dimension tensor and the post-input stacked tensor are respectively input to a total matrix multiplication operation node, and the total matrix multiplication operation node is used to perform a matrix multiplication operation on the pre-input increased dimension tensor and the post-input stacked tensor to obtain a first total output tensor.

[0191] Specifically, please refer to Figure 5 The computer device inputs the pre-input tensor (N, a, B) as pre-input data and the post-input tensor (N, B, C) as post-input data to the total matrix multiplication operation node, that is, the matrix multiplication operation node 501c in Figure 5 It can be understood that the first tensor dimension corresponding to the pre-input tensor (N, a, B) and the post-input tensor (N, B, C) is the tensor batch dimension, so the matrix multiplication operation node 501c does not process this dimension, and only performs a matrix multiplication operation on the tensor row dimension and the tensor column dimension corresponding to the two input tensors, that is, the pre-input tensor (N, a, B) and the post-input tensor (N, B, C), so as to obtain a first total output tensor, as shown in the output tensor (N, a, C) shown in Figure 5

[0192] In step S208, the first total output tensor is input to a de-stacking operation node, and the de-stacking operation node is used to perform a de-stacking operation on the tensors belonging to the tensor batch dimension in the first total output tensor to obtain N first output sub-tensors.

[0193] Specifically, in order to maintain consistency with Figure 5 ​The example original data flow graph has the same output result, and the computer device needs to perform a de-merging operation on the first total output tensor in the order of the sequence number. According to step S206, the de-merging operation is a de-stacking operation corresponding to the stacking operation, as shown in Figure 3a The first total output tensor (that is, the output tensor (N, a, C)) is input to the de-stacking operation node 501d, and the de-stacking operation node 501d is used to perform a de-stacking operation on the tensors belonging to the tensor batch dimension in the first total output tensor to obtain N first output sub-tensors. It can be understood that the stacking operation in step S206 above generates a new dimension, that is, the batch dimension, and correspondingly, the de-stacking operation in this step can remove the newly added dimension, so that the same output result as the original data flow graph can be maintained, that is, output result 1 (a, C), output result 2 (a, C), …, output result N (a, C).

[0194] The embodiment of the present application describes the data processing process of the data flow graph using a 2-dimensional tensor as an example. In actual application, the data can be extended to a higher-dimensional tensor. When other dimensions are all batch dimensions, the other dimensions can be kept unchanged.

[0195] In the embodiment of the present application, by updating the N sub-matrix multiplication operation nodes in the original data flow graph to the total matrix multiplication operation node, the utilization rate of system resources can be improved. By updating the splitting operation node to the front input transformation operation node for performing a transformation operation on the front input tensor, the front input tensor can be transformed into a tensor suitable for the total matrix multiplication operation node. Further, by adding the stacking operation node for stacking the N post-sub-input tensors before the total matrix multiplication operation node, the N post-sub-input tensors can be stacked into a tensor suitable for the total matrix multiplication operation node. By adding the de-stacking operation node after the total matrix multiplication operation node, the data result of the original data flow graph can be maintained and the optimized data flow graph can be obtained. As described above, the optimized data flow graph obtained by optimizing the original data flow graph can not only maintain the data result of the original data flow graph, but also improve the utilization rate of system resources.

[0196] Further, please refer to Figure 5 , Figure 6 is a flow diagram of a data flow graph processing method provided by an embodiment of the present application. The data flow graph processing method can be executed by a computer device, which can include Figure 6 a user terminal and a server in Figure 1 As shown in the figure, the data flow graph processing process can include the following steps.

[0197] Step S301, obtaining an original data flow graph and parsing the original data flow graph.

[0198] Step S302, if it is parsed that the original data flow graph includes the splitting operation node and the N sub-matrix multiplication operation nodes, the position of the splitting operation node in the original data flow graph is detected; N is a positive integer, and N is equal to or greater than 2.

[0199] Step S303, if in the original data flow graph, the splitting operation node is connected with the N sub-matrix multiplication operation nodes respectively, and the position of the splitting operation node in the original data flow graph is before the N sub-matrix multiplication operation nodes, the N sub-matrix multiplication operation nodes are updated as a total matrix multiplication operation node, and the splitting operation node is updated as a first deformation operation node for performing a deformation operation on a total input tensor.

[0200] The specific process of steps S301-S303 can be referred to the description of steps S101-S103 in the corresponding embodiment, which will not be repeated here. Figure 6

[0201] Step S304, when the first deformation operation node is a post-input deformation operation node, a stacking operation node for performing a stacking operation on N pre-input tensors is added before the total matrix multiplication operation node, a post-input transposition operation node is added between the total matrix multiplication operation node and the post-input deformation operation node, and a de-stacking operation node is added after the total matrix multiplication operation node, to obtain an optimized data flow graph.

[0202] Specifically, the N sub-input tensors include N pre-input sub-tensors that have a size association relationship with the post-input tensor and respectively carry different serial numbers; the merging operation node includes a stacking operation node, and the de-merging operation node includes a de-stacking operation node associated with the stacking operation node.

[0203] Please refer to Figure 2 , Figure 7 is a structural schematic diagram of an optimized data flow graph provided by an embodiment of the present application. As shown in Figure 7 , the first deformation operation node is a post-input deformation operation node, that is, a deformation operation node 502a for a post-input shown in Figure 7 , the total input tensor (B, C) is a post-input tensor, and the N sub-input tensors are all pre-input sub-tensors.

[0204] As shown in Figure 7 , the computer device adds a stacking operation node 502b for performing a stacking operation on the N pre-input sub-tensors before the total matrix multiplication operation node 502c, and adds a post-input transposition operation node between the total matrix multiplication operation node 502c and the post-input deformation operation node, that is, Figure 7 ​the transpose operation node 502e in the total matrix multiplication operation node 502c, and the de-stacking operation node 502d added after the total matrix multiplication operation node 502c, to obtain Figure 7 the optimized data flow graph shown in FIG. 5B.

[0205] In step S305, the post-input tensor is input to a post-input deformation operation node in the optimized data flow graph, and the post-input tensor is subjected to a deformation operation by the post-input deformation operation node to obtain a post-input augmented tensor.

[0206] Specifically, the N pre-input sub-tensors correspond to the same tensor dimension, and the N tensor dimensions correspond to the same size; the post-input tensor corresponds to a tensor dimension equal to the N tensor dimensions, and the size of the tensor row dimension corresponding to the post-input tensor is equal to the size of the tensor column dimension corresponding to the N pre-input sub-tensors.

[0207] According to the column deformation operation attribute of the post-input deformation operation node, the tensors belonging to the tensor column dimension in the post-input tensor are determined as second to-be-deformed tensors; the N second to-be-deformed tensors are subjected to a deformation operation to obtain a second augmented tensor; the size of the first dimension corresponding to the second augmented tensor is equal to N, and the size of the second to-be-deformed tensor is equal to N times the size of the second dimension corresponding to the second augmented tensor; the second to-be-deformed tensors in the post-input tensor are updated to the second augmented tensor to obtain the post-input augmented tensor.

[0208] Please refer to Figure 7 , the computer device inputs the post-input tensor (equivalent to the total input tensor (B, C) in Figure 7 ) as input data to the post-input deformation operation node (equivalent to the deformation operation node 502a in Figure 7 ), and according to the column deformation operation attribute of the deformation operation node 502a, the tensors corresponding to the tensor column dimension of the post-input tensor are subjected to deformation processing, that is, the size C shown in Figure 7 is subjected to a deformation operation. Here, the deformation operation can be understood as a dimension increasing operation, that is, the size C of the tensor column dimension is deformed into N of the tensor batch dimension and the size c of the tensor column dimension, and c*N=C, so the output data of the deformation operation node 502a is the post-input augmented tensor, which is equivalent to the post-input temporary expression tensor (B, N, c) in Figure 7 .

[0209] In step S306, the post-input augmented tensor is input to the post-input transpose operation node, and the tensors belonging to the to-be-transposed tensor dimension in the post-input augmented tensor are subjected to a transpose operation by the post-input transpose operation node to obtain a post-input transposed tensor; the to-be-transposed tensor dimension includes the tensor row dimension and the tensor batch dimension corresponding to the post-input augmented tensor.

[0210] Specifically, please refer toFigure 7 Since the position of the tensor batch dimension corresponding to the post-input temporary expression tensor (B, N, c) in the tensor shape is different from the position of the tensor batch dimension corresponding to the pre-input tensor (N, A, B) in the tensor shape (the former is the second, and the latter is the first), the computer device needs to perform transpose processing on the post-input temporary expression tensor (B, N, c), that is, input the post-input augmented tensor to the post-input transpose operation node (equivalent to the transpose operation node 502e in Figure 7 ). Figure 7 The post-input transpose tensor is obtained by performing a transpose operation on the tensor belonging to the tensor dimension to be transposed in the post-input augmented tensor through the post-input transpose operation node, which is equivalent to the post-input temporary expression tensor (N, B, c) in Figure 7 .

[0211] In step S307, the N pre-input sub-tensors are input to the stacking operation node in the optimized data flow graph in sequence, and the N pre-input sub-tensors are stacked through the stacking operation node to obtain a pre-input stacked tensor. The pre-input stacked tensor includes a tensor batch dimension with a size equal to N, and the size of the tensor dimension corresponding to the pre-input stacked tensor, except for the tensor batch dimension with a size equal to N, is equal to the size corresponding to the N tensor dimensions respectively.

[0212] Please refer to 7 again. The computer device inputs the N pre-input sub-tensors to the stacking operation node 502b in sequence, as shown in Figure 7 , the computer device performs a stacking operation on the N pre-input sub-tensors through the stacking operation node 502b to obtain a pre-input stacked tensor, that is, the pre-input tensor (N, A, B) shown in Figure 7 . Obviously, the pre-input stacked tensor has one more tensor batch dimension than the pre-input sub-tensor, and the size of the tensor batch dimension is equal to N.

[0213] In step S308, the post-input transpose tensor and the pre-input stacked tensor are input to the general matrix multiplication operation node respectively, and the post-input transpose tensor and the pre-input stacked tensor are multiplied through the general matrix multiplication operation node to obtain a second general output tensor.

[0214] Specifically, please refer to Figure 7 , the computer device inputs the post-input tensor (N, B, c) as post-input data and the pre-input tensor (N, A, B) as pre-input data to the general matrix multiplication operation node, that is, Figure 7the matrix multiplication operation node 502c in the original data flow graph, it can be understood that the first tensor dimension corresponding to the pre-input tensor (N, A, B,) and the post-input tensor (N, B, c) is the tensor batch dimension, so the matrix multiplication operation node 502c does not process this dimension, and only performs the matrix multiplication operation on the tensor row dimension and the tensor column dimension corresponding to the pre-input tensor (N, A, B,) and the post-input tensor (N, B, c), so as to obtain the second total output tensor, as shown in the output tensor (N, A, c). Figure 7

[0215] In step S309, the second total output tensor is input to a de-stacking operation node, and the tensors belonging to the tensor batch dimension in the second total output tensor are de-stacked by the de-stacking operation node to obtain N second output sub-tensors.

[0216] Specifically, in order to maintain the same output result as the original data flow graph shown in Figure 7 , the computer device needs to perform a de-merging operation on the second total output tensor in the order of the sequence number, and according to step S307, the de-merging operation is a de-stacking operation corresponding to the stacking operation, as shown in Figure 7 , the second total output tensor (i.e., the output tensor (N, A, c)) is input to a de-stacking operation node 502d, and the tensors belonging to the tensor batch dimension in the second total output tensor are de-stacked by the de-stacking operation node 502d to obtain N second output sub-tensors. It can be understood that the stacking operation in step S307 above produces a new dimension, i.e., the batch dimension, and correspondingly, the de-stacking operation in this step can remove the newly added dimension, so that the same output result as the original data flow graph can be maintained, i.e., the output result 1 (A, c), the output result 2 (A, c), …, and the output result N (A, c).

[0217] In the embodiments of the present application, by updating the N sub-matrix multiplication operation nodes in the original data flow graph to the total matrix multiplication operation node, the utilization rate of system resources can be improved; by updating the splitting operation node to the post-input transformation operation node for performing the transformation operation on the post-input tensor, the post-input tensor can be transformed into a tensor suitable for the total matrix multiplication operation node; further, by adding the stacking operation node for stacking the N pre-input sub-tensors before the total matrix multiplication operation node, the N pre-input sub-tensors can be stacked into a tensor suitable for the total matrix multiplication operation node; by adding the de-stacking operation node after the total matrix multiplication operation node, the data result of the original data flow graph can be maintained and the optimized data flow graph can be obtained. As can be seen from the above, the optimized data flow graph obtained by the present application for optimizing the original data flow graph not only maintains the data result of the original data flow graph, but also improves the utilization rate of system resources.​

[0218] Further, please refer to Figure 3b , Figure 7 is a flow diagram of a data flow graph processing method provided by an embodiment of the present application. The data flow graph processing method can be executed by a computer device, which can include Figure 8 a user terminal and a server. As shown in Figure 8 , the data flow graph processing process can include the following steps.

[0219] Step S401, obtaining an original data flow graph and parsing the original data flow graph.

[0220] Step S402, if the original data flow graph includes a splitting operation node and N sub-matrix multiplication operation nodes, detecting the position of the splitting operation node in the original data flow graph; N is a positive integer and N is equal to or greater than 2.

[0221] Step S403, if in the original data flow graph, the splitting operation node and the N sub-matrix multiplication operation nodes are connected respectively, and the position of the splitting operation node in the original data flow graph is before the N sub-matrix multiplication operation nodes, updating the N sub-matrix multiplication operation nodes to a total matrix multiplication operation node, and updating the splitting operation node to a first deformation operation node for performing a deformation operation on a total input tensor.

[0222] The specific process of steps S401-S403 can be referred to the description of steps S101-S103 in the embodiments corresponding to Figure 1 herein.

[0223] Step S404, when the first deformation operation node is a front input deformation operation node, adding a second deformation operation node before the total matrix multiplication operation node, and adding a splicing operation node for performing a splicing operation on N post-input sub-tensors before the second deformation operation node.

[0224] Specifically, the N sub-input tensors include N post-input sub-tensors having a size association relationship with a front input tensor and carrying different serial numbers respectively; the merging operation node includes a splicing operation node, and the de-merging operation node includes a de-splicing operation node associated with the splicing operation node.

[0225] Please refer to Figure 8 , Figure 2 is a structure diagram of an optimized data flow graph provided by an embodiment of the present application. As shown in Figure 9 , the first deformation operation node is a front input deformation operation node, that is Figure 9The total input tensor (A, B) is a pre-input tensor, and the N sub-input tensors are all post-input sub-tensors, as shown in the pre-input transformation operation node 503a in the foregoing embodiment.

[0226] As shown in Figure 9 , the computer device adds a second transformation operation node before the total matrix multiplication operation node 503c, which is equivalent to Figure 9 the post-input transformation operation node 503e shown in the foregoing embodiment, a splicing operation node for splicing the N post-input sub-tensors is added before the second transformation operation node, which is equivalent to Figure 9 the post-input splicing operation node 503b shown in the foregoing embodiment.

[0227] Step S405: adding a third transformation operation node associated with the second transformation operation node after the total matrix multiplication operation node, and adding a de-splicing operation node after the third transformation operation node to obtain an optimized data flow graph; the node relationship between the second transformation operation node and the pre-input transformation operation node is a parallel node relationship.

[0228] Specifically, referring to Figure 9 , the computer device adds a third transformation operation node associated with the second transformation operation node after the total matrix multiplication operation node 503c, which is equivalent to Figure 9 the post-input transformation operation node 503f in the foregoing embodiment, and adding a de-splicing operation node 503d after the third transformation operation node to obtain Figure 9 the optimized data flow graph shown in .

[0229] Step S406: inputting the pre-input tensor into the pre-input transformation operation node in the optimized data flow graph, and performing a transformation operation on the pre-input tensor through the pre-input transformation operation node to obtain a pre-input augmented tensor.

[0230] Specifically, the N post-input sub-tensors respectively correspond to the same tensor dimension, and the N tensor dimensions respectively correspond to the same size; the pre-input tensor corresponds to a tensor dimension equal to the N tensor dimensions, and the size of the tensor column dimension corresponding to the pre-input tensor is equal to the size of the tensor row dimension corresponding to the N post-input sub-tensors.

[0231] For the specific process of this step, refer to the description in step S205 in the embodiment corresponding to Figure 9 .

[0232] In step S407, the N post-input sub-tensors are input to the splicing operation node in the optimized data flow graph in sequence, and the N post-input sub-tensors are spliced by the splicing operation node to obtain a post-input splicing tensor; the tensor dimension corresponding to the post-input splicing tensor is equal to the N tensor dimensions.

[0233] Specifically, the N post-input sub-tensors are spliced in the tensor row dimension by the splicing operation node to obtain the post-input splicing tensor; the size of the tensor row dimension corresponding to the post-input splicing tensor is equal to the sum of the sizes of the tensor row dimensions corresponding to the N post-input sub-tensors, respectively.

[0234] Please refer to Figure 9 , the computer device inputs the N post-input sub-tensors to the splicing operation node 503b in sequence, wherein the splicing operation is a tensor operation that can combine multiple tensors into one tensor, and the splicing operation does not increase the dimension, as shown in Figure 4 , the computer device splices the N post-input sub-tensors by the splicing operation node 503b to obtain a post-input splicing tensor, that is, the post-input temporary expression (NB, C) shown in Figure 9 Obviously, the total number of tensor dimensions corresponding to the post-input splicing tensor is equal to the total number of tensor dimensions corresponding to the post-input sub-tensors, but the size of the tensor row dimension corresponding to the post-input splicing tensor is equal to N times the size of the tensor row dimension corresponding to the post-input sub-tensors.

[0235] In step S408, the post-input splicing tensor is input to the second deformation operation node, and the post-input splicing tensor is deformed by the second deformation operation node to obtain a post-input deformation tensor.

[0236] Specifically, the tensors belonging to the tensor row dimension in the post-input splicing tensor are determined as the third to-be-deformed tensor; the third to-be-deformed tensor is deformed by the second deformation operation node to obtain a third dimension-increased tensor; the size of the first dimension corresponding to the third dimension-increased tensor is equal to N, and the size of the third to-be-deformed tensor is equal to N times the size of the second dimension corresponding to the third dimension-increased tensor; the third to-be-deformed tensor in the post-input splicing tensor is updated to the third dimension-increased tensor to obtain the post-input deformation tensor.

[0237] It can be understood that, for the matrix multiplication operation, the row size corresponding to the first input data needs to be equal to the column size corresponding to the second input data. Please refer to Figure 9 , the computer device inputs the post-input splicing tensor to the deformation operation node 503e, and the function of the deformation operation node 503e is the same as that of the deformation operation node 503a, so it will not be described here. The computer device can obtain the post-input deformation tensor by the deformation operation node 503e, that is,Figure 9 the post-input tensor (N, B, C) in

[0238] In step S409, the pre-input augmented tensor and the post-input transformed tensor are input into a total matrix multiplication operation node, and the pre-input augmented tensor and the post-input transformed tensor are subjected to matrix multiplication operation by the total matrix multiplication operation node to obtain a first to-be-transformed total output tensor.

[0239] For the specific implementation process of this step, please refer to the description in the foregoing Figure 9 corresponding embodiment in step S207, which will not be described here again.

[0240] In step S410, the first to-be-transformed total output tensor is input into a third transformation operation node, and the first to-be-transformed total output tensor is subjected to transformation operation by the third transformation operation node to obtain a third total output tensor.

[0241] Specifically, the tensors belonging to a tensor target dimension in the first to-be-transformed total output tensor are determined as a fourth to-be-transformed tensor; the tensor target dimension includes a tensor batch dimension and a tensor row dimension corresponding to the first to-be-transformed total output tensor; the fourth to-be-transformed tensor is subjected to transformation operation by the third transformation operation node to obtain a target reduced dimension tensor; the size of the target reduced dimension tensor is equal to N times the size of the second dimension corresponding to the fourth to-be-transformed tensor; the fourth to-be-transformed tensor in the first to-be-transformed total output tensor is updated as the target reduced dimension tensor to obtain the third total output tensor.

[0242] Specifically, please refer to the foregoing Figure 9 The first to-be-transformed total output tensor (output tensor (N, a, C)) includes three dimensions, and the unconcatenation operation cannot reduce the dimension, so the computer device inputs the first to-be-transformed total output tensor into the third transformation operation node, that is, Figure 5 the transformation operation node 503f for the output in , the first to-be-transformed total output tensor is subjected to transformation operation by the third transformation operation node, that is, the first to-be-transformed total output tensor is subjected to dimension reduction processing to obtain the third total output tensor, which is equivalent to Figure 9 the output temporary expression (A, C) in .

[0243] In step S411, the third total output tensor is input into an unconcatenation operation node, and the tensors belonging to the tensor row dimension in the third total output tensor are subjected to splitting operation by the unconcatenation operation node to obtain N first output sub-tensors.

[0244] As Figure 9As shown, the computer device inputs the third total output tensor (i.e., the output tensor (A, C)) to the de-tile operation node 503d, and performs a de-tile operation on the tensors belonging to the tensor row dimension in the third total output tensor through the de-tile operation node 503d to obtain N first output sub-tensors, i.e., an output result 1 (a, C), an output result 2 (a, C), …, and an output result N (a, C).

[0245] In the embodiment of the present application, by updating the N sub-matrix multiplication operation nodes in the original data flow graph to the total matrix multiplication operation node, the utilization rate of system resources can be improved; by updating the split operation node to the pre-input deformation operation node for performing deformation operation on the pre-input tensor, the pre-input tensor can be deformed into a tensor suitable for the total matrix multiplication operation node; further, by adding the tile operation node for performing tile operation on the N post-input sub-tensors before the total matrix multiplication operation node, the N post-input sub-tensors can be tiled into a tensor suitable for the total matrix multiplication operation node; by adding the de-merge operation node after the total matrix multiplication operation node, the data result of the original data flow graph and the optimized data flow graph can be obtained. As known from the above, the optimized data flow graph obtained by the present application for optimizing the original data flow graph can not only maintain the data result of the original data flow graph, but also improve the utilization rate of system resources.

[0246] Further, please refer to Figure 9 , Figure 9 is a flow diagram of a data flow graph processing method provided by an embodiment of the present application. The data flow graph processing method can be executed by a computer device, which can include Figure 10 a user terminal and a server in Figure 10 As shown, the data flow graph processing process can include the following steps.

[0247] Step S501, an original data flow graph is acquired and parsed.

[0248] Step S502, if it is parsed that the original data flow graph includes a split operation node and N sub-matrix multiplication operation nodes, the position of the split operation node in the original data flow graph is detected; N is a positive integer and N is equal to or greater than 2.

[0249] Step S503, if in the original data flow graph, the split operation node and the N sub-matrix multiplication operation nodes are connected respectively, and the position of the split operation node in the original data flow graph is before the N sub-matrix multiplication operation nodes, the N sub-matrix multiplication operation nodes are updated to a total matrix multiplication operation node, and the split operation node is updated to a first deformation operation node for performing deformation operation on a total input tensor.

[0250] The specific process of steps S501-S503 can be referred to the description in the foregoing Figure 1 The description of steps S101-S103 in the corresponding embodiment is not repeated here.

[0251] In step S504, when the first deformation operation node is a post-input deformation operation node, a fourth deformation operation node is added before the total matrix multiplication operation node, and a splicing operation node for splicing the N pre-input sub-tensors is added before the fourth deformation operation node.

[0252] Specifically, the N sub-input tensors include N pre-input sub-tensors that have a size association relationship with the post-input tensor and respectively carry different serial numbers; the merging operation node includes a splicing operation node, and the de-merging operation node includes a de-splicing operation node associated with the splicing operation node.

[0253] In step S505, a post-input transposition operation node is added between the total matrix multiplication operation node and the post-input deformation operation node.

[0254] In step S506, a fifth deformation operation node associated with the fourth deformation operation node is added after the total matrix multiplication operation node, and a de-splicing operation node is added after the fifth deformation operation node to obtain an optimized data flow graph; the node relationship between the fourth deformation operation node and the post-input deformation operation node is a parallel node relationship.

[0255] In combination with steps S504-S506, please refer to Figure 10 , Figure 2 is a structural schematic diagram of an optimized data flow graph provided by an embodiment of the present application. The computer device performs the optimization processing on the original data flow graph shown in Figure 11 , and the result of the optimization processing is an optimized data flow graph as shown in Figure 11 . The specific process can be referred to the description of steps S404-S405 in the foregoing Figure 3b . The description is not repeated here.

[0256] In step S507, the post-input tensor is input to the post-input deformation operation node in the optimized data flow graph, and the post-input tensor is subjected to a deformation operation through the post-input deformation operation node to obtain a post-input dimension-increased tensor.

[0257] Specifically, the N pre-input sub-tensors respectively correspond to the same tensor dimension, and the N tensor dimensions respectively correspond to the same size; the post-input tensor corresponds to a tensor dimension equal to the N tensor dimensions, and the tensor row dimension of the post-input tensor corresponds to a size equal to the tensor column dimension of the N pre-input sub-tensors.

[0258] In step S508, the post-input augmented tensor is input to a post-input transposition operation node. The post-input transposition operation node is used to perform a transposition operation on the tensors in the post-input augmented tensor that belong to the tensor dimensions to be transposed to obtain a post-input transposed tensor. The tensor dimensions to be transposed include the tensor row dimension and the tensor batch dimension corresponding to the post-input augmented tensor.

[0259] For details of steps S507-S508, please refer to the foregoing description Figure 8 For details of steps S305-S306 in the corresponding embodiment, please refer to the foregoing description.

[0260] In step S509, the N pre-input sub-tensors are input to a splicing operation node in the optimized data flow graph in sequence. The splicing operation node is used to perform a splicing operation on the N pre-input sub-tensors to obtain a first pre-input spliced tensor. The tensor dimensions corresponding to the first pre-input spliced tensor are equal to the N tensor dimensions.

[0261] Specifically, the splicing operation node is used to perform a splicing operation on the N pre-input sub-tensors in the tensor row dimension to obtain the first pre-input spliced tensor. The size of the tensor row dimension corresponding to the first pre-input spliced tensor is equal to the sum of the sizes of the tensor row dimensions corresponding to the N pre-input sub-tensors.

[0262] For details, please refer to Figure 6 The computer device inputs the N pre-input sub-tensors to the splicing operation node 504b in sequence. The splicing operation node 504b is used to perform a splicing operation on the N pre-input sub-tensors to obtain a pre-input spliced tensor, i.e. Figure 11 As shown in the pre-input temporary expression (NA, B) in the example in

[0263] In step S510, the first pre-input spliced tensor is input to a fourth deformation operation node. The fourth deformation operation node is used to perform a deformation operation on the first pre-input spliced tensor to obtain a first pre-input deformed tensor.

[0264] Specifically, the tensors belonging to the tensor row dimension in the first pre-input splicing tensor are determined as a fifth to-be-deformed tensor; the fourth deforming operation node is used for performing a deforming operation on the fifth to-be-deformed tensor to obtain a fourth dimension-increased tensor; the size of the first dimension corresponding to the fourth dimension-increased tensor is equal to N, and the size of the fifth to-be-deformed tensor is equal to N times the size of the second dimension corresponding to the fourth dimension-increased tensor; the fifth to-be-deformed tensor in the first pre-input splicing tensor is updated as the fourth dimension-increased tensor to obtain a first pre-input deformed tensor.

[0265] In step S511, the first pre-input deformed tensor and the post-input transposed tensor are input into a total matrix multiplication operation node, and a matrix multiplication operation is performed on the first pre-input deformed tensor and the post-input transposed tensor by the total matrix multiplication operation node to obtain a second to-be-deformed total output tensor.

[0266] In step S512, the second to-be-deformed total output tensor is input into a fifth deforming operation node, and a deforming operation is performed on the second to-be-deformed total output tensor by the fifth deforming operation node to obtain a fourth total output tensor.

[0267] Specifically, the tensors belonging to the tensor target dimension in the second to-be-deformed total output tensor are determined as a sixth to-be-deformed tensor; the tensor target dimension includes the tensor batch dimension and the tensor row dimension corresponding to the second to-be-deformed total output tensor; the fifth deforming operation node is used for performing a deforming operation on the sixth to-be-deformed tensor to obtain a first dimension-reduced tensor; the size of the first dimension-reduced tensor is equal to N times the size of the second dimension corresponding to the sixth to-be-deformed tensor; the sixth to-be-deformed tensor in the second to-be-deformed total output tensor is updated as the first dimension-reduced tensor to obtain the fourth total output tensor.

[0268] In step S513, the fourth total output tensor is input into a de-splicing operation node, and a splitting operation is performed on the tensors belonging to the tensor row dimension in the fourth total output tensor by the de-splicing operation node to obtain N second output sub-tensors.

[0269] For the specific process of steps S510-S513, refer to the foregoing description Figure 11 For the description in steps S408-S411 in the corresponding embodiment, details are not described herein again.

[0270] In the embodiment of the present application, by updating the N sub-matrix multiplication operation nodes in the original data flow graph to the total matrix multiplication operation node, the utilization rate of system resources can be improved; by updating the split operation node to the pre-input deformation operation node for deforming the pre-input tensor, the pre-input tensor can be deformed to a tensor suitable for the total matrix multiplication operation node; further, by adding the splicing operation node for splicing the N post-sub-input tensors before the total matrix multiplication operation node, the N post-sub-input tensors can be spliced to a tensor suitable for the total matrix multiplication operation node; by adding the de-splicing operation node after the total matrix multiplication operation node, the data result of the original data flow graph and the optimized data flow graph can be obtained. As can be seen from the above, the optimized data flow graph obtained by the present application by optimizing the original data flow graph can not only maintain the data result of the original data flow graph, but also improve the utilization rate of system resources.

[0271] Further, please refer to Figure 8 , Figure 12 is a flow diagram of a data flow graph processing method provided by an embodiment of the present application. The data flow graph processing method can be executed by a computer device, which can include Figure 12 a user terminal and a server in Figure 1 As shown in the figure, the data flow graph processing process can include the following steps.

[0272] Step S601, an original data flow graph is obtained and parsed.

[0273] Step S602, if the original data flow graph is parsed to include a split operation node and N sub-matrix multiplication operation nodes, the position of the split operation node in the original data flow graph is detected; N is a positive integer and N is equal to or greater than 2.

[0274] Step S603, if in the original data flow graph, the split operation node and the N sub-matrix multiplication operation nodes are connected respectively, and the position of the split operation node in the original data flow graph is before the N sub-matrix multiplication operation nodes, the N sub-matrix multiplication operation nodes are updated to a total matrix multiplication operation node, and the split operation node is updated to a first deformation operation node for deforming the total input tensor.

[0275] The specific process of steps S601-S603 can be referred to the description of steps S101-S103 in the embodiments corresponding to Figure 12 The above, which will not be repeated here.

[0276] Step S604, when the first deformation operation node is a post-input deformation operation node, a fourth deformation operation node is added before the total matrix multiplication operation node, and a splicing operation node for splicing the N pre-input sub-tensors is added before the fourth deformation operation node.

[0277] Step S605, a post-input transposition operation node is added between the total matrix multiplication operation node and the post-input deformation operation node.

[0278] Step S606, a fifth deformation operation node associated with the fourth deformation operation node is added after the total matrix multiplication operation node, and a de-splicing operation node is added after the fifth deformation operation node, to obtain an optimized data flow graph; the node relationship between the fourth deformation operation node and the post-input deformation operation node is a parallel node relationship.

[0279] The specific process of steps S604-S606 can be referred to the description of steps S504-S506 in the corresponding embodiment. Figure 2 The specific process of steps S604-S606 can be referred to the description of steps S504-S506 in the corresponding embodiment.

[0280] Step S607, when the total matrix multiplication operation node has the operation authority for the tensor whose batch dimension belongs to the intermediate dimension, the post-input transposition operation node in the optimized data flow graph is deleted, to obtain a target optimized data flow graph.

[0281] The tensor dimensions corresponding to the N pre-input sub-tensors are all the same, and the sizes of the N tensor dimensions are all the same; the tensor dimension corresponding to the post-input tensor is equal to the N tensor dimensions, and the size of the tensor row dimension corresponding to the post-input tensor is equal to the size of the tensor column dimension corresponding to the N pre-input sub-tensors.

[0282] Specifically, when the operator set supporting that the batch dimension can be located in the middle position of the tensor shape representation is supported, the computer device can add a splicing operation node for splicing the N pre-input sub-tensors before the total matrix multiplication operation node, and add a fourth deformation operation node before the splicing operation node. Figure 10 The example of the optimized data flow graph is simplified to the example of the target optimized data flow graph shown in FIG. 6D. Figure 11 The example of the optimized data flow graph is simplified to the example of the target optimized data flow graph shown in FIG. 6D. Figure 11 The example of the optimized data flow graph is simplified to the example of the target optimized data flow graph shown in FIG. 6D. Figure 13 The example of the optimized data flow graph is simplified to the example of the target optimized data flow graph shown in FIG. 6D.

[0283] Step S608, the post-input tensor is input to the post-input deformation operation node in the target optimized data flow graph, and the post-input deformation operation node is used to perform deformation operation on the post-input tensor, to obtain a post-input dimension-increased tensor.

[0284] The specific process of this step can be referred to the description of steps S504-S506 in the corresponding embodiment. Figure 13The description in step S305 in the corresponding embodiment will not be repeated here.

[0285] In step S609, the N pre-input sub-tensors are input to the splicing operation node in the target optimization data flow graph in sequence, and the N pre-input sub-tensors are spliced by the splicing operation node to obtain a second pre-input splicing tensor; the tensor dimension corresponding to the second pre-input splicing tensor is equal to the N tensor dimensions; and the size of the tensor column dimension corresponding to the second pre-input splicing tensor is equal to the sum of the sizes of the tensor column dimensions corresponding to the N post-input sub-tensors respectively.

[0286] Specifically, the N pre-input sub-tensors are spliced by the splicing operation node in the tensor column dimension to obtain a second pre-input splicing tensor; and the size of the tensor column dimension corresponding to the second pre-input splicing tensor is equal to the sum of the sizes of the tensor column dimensions corresponding to the N post-input sub-tensors respectively.

[0287] Please refer to Figure 6 At this time, the computer device splices the N pre-input sub-tensors in the tensor column dimension by the splicing operation node 504b to obtain a second pre-input splicing tensor, that is, Figure 11 The temporary expression (A, NB) of the pre-input shown in the embodiment.

[0288] In step S610, the second pre-input splicing tensor is input to the fourth deformation operation node, and the second pre-input splicing tensor is deformed by the fourth deformation operation node to obtain a second pre-input deformation tensor.

[0289] Specifically, the tensors belonging to the tensor column dimension in the second pre-input splicing tensor are determined as a seventh to-be-deformed tensor; the seventh to-be-deformed tensor is deformed by the fourth deformation operation node to obtain a fifth dimension-increased tensor; the size of the first dimension corresponding to the fifth dimension-increased tensor is equal to N, and the size of the seventh to-be-deformed tensor is equal to N times the size of the second dimension corresponding to the fifth dimension-increased tensor; the seventh to-be-deformed tensor in the second pre-input splicing tensor is updated as the fifth dimension-increased tensor to obtain the second pre-input deformation tensor.

[0290] In step S611, the second pre-input deformation tensor and the post-input dimension-increased tensor are input to the total matrix multiplication operation node respectively, and the second pre-input deformation tensor and the post-input dimension-increased tensor are multiplied by the total matrix multiplication operation node to obtain a third to-be-deformed total output tensor.

[0291] In step S612, the third to-be-deformed total output tensor is input to the fifth deformation operation node, and the third to-be-deformed total output tensor is deformed by the fifth deformation operation node to obtain a fifth total output tensor.

[0292] Specifically, a tensor belonging to a tensor target dimension in the third total output tensor to be deformed is determined as an eighth total output tensor to be deformed; the tensor target dimension includes a tensor batch dimension and a tensor column dimension corresponding to the sixth total output tensor to be deformed; the eighth total output tensor to be deformed is subjected to a deformation operation by the fifth deformation operation node to obtain a second dimension-reduced tensor; a size of the second dimension-reduced tensor is equal to N times of a size of a second dimension corresponding to the eighth total output tensor to be deformed; the eighth total output tensor to be deformed in the third total output tensor to be deformed is updated as the second dimension-reduced tensor to obtain a fifth total output tensor.

[0293] In step S613, the fifth total output tensor is input to a de-splicing operation node, and tensors belonging to the tensor column dimension in the fifth total output tensor are subjected to a splitting operation by the de-splicing operation node to obtain N second output sub-tensors.

[0294] For specific processes of steps S610-S613, refer to the foregoing description Figure 13 For specific processes of steps S408-S411 in the corresponding embodiment, refer to the foregoing description

[0295] In the embodiment of the present application, the N sub-matrix multiplication operation nodes in the original data flow graph are updated as the total matrix multiplication operation node, so that the utilization rate of system resources can be improved; the splitting operation node is updated as the post-input deformation operation node for deforming the post-input tensor, so that the post-input tensor can be deformed into a tensor suitable for the total matrix multiplication operation node; further, the splicing operation node for splicing the N pre-input sub-tensors is added before the total matrix multiplication operation node, so that the N post-input sub-tensors can be spliced into a tensor suitable for the total matrix multiplication operation node; the de-splicing operation node is added after the total matrix multiplication operation node, so that the data results of the original data flow graph can be maintained and the optimized data flow graph can be obtained. As can be seen from the foregoing, the optimized data flow graph obtained by the optimization processing of the original data flow graph not only can maintain the data results of the original data flow graph, but also can improve the utilization rate of system resources.

[0296] Further, refer to Figure 8 , Figure 14 is a structural schematic diagram of a data flow graph processing device provided in an embodiment of the present application. The data flow graph processing device can be a computer program (including program code) running in a computer device, for example, the data flow graph processing device is an application software; the device can be used to execute corresponding steps in the method provided in the embodiment of the present application. As shown in Figure 14 the data flow graph processing device 1 can include an acquisition data module 11, a detection position module 12, an update node module 13, and an add node module 14.

[0297] The data acquisition module 11 is configured to acquire an original data flow graph and parse the original data flow graph.

[0298] The position detection module 12 is configured to, if the original data flow graph includes a split operation node and N sub-matrix multiplication operation nodes, detect a position of the split operation node in the original data flow graph; N is a positive integer, and N is equal to or greater than 2.

[0299] The node updating module 13 is configured to, if the split operation node and the N sub-matrix multiplication operation nodes are connected respectively in the original data flow graph, and the position of the split operation node in the original data flow graph is before the N sub-matrix multiplication operation nodes, update the N sub-matrix multiplication operation nodes to a total matrix multiplication operation node, and update the split operation node to a first deformation operation node for performing a deformation operation on a total input tensor.

[0300] The node adding module 14 is configured to add a merge operation node for performing a merge operation on N sub-input tensors before the total matrix multiplication operation node, and add a de-merge operation node after the total matrix multiplication operation node, to obtain an optimized data flow graph; a node relationship between the merge operation node and the first deformation operation node is a parallel node relationship; and there is a size correlation relationship between the total input tensor and the N sub-input tensors.

[0301] The specific function implementation manners of the data acquisition module 11, the position detection module 12, the node updating module 13, and the node adding module 14 can be referred to the above Figure 14 The steps S101-S104 in the corresponding embodiments will not be repeated here.

[0302] Please refer to Figure 2 The first deformation operation node includes a pre-input deformation operation node or a post-input deformation operation node; and the total input tensor includes a pre-input tensor or a post-input tensor.

[0303] The node updating module 13 can include a first determination unit 131, a first updating unit 132, and a second updating unit 133.

[0304] The first determination unit 131 is configured to determine an input attribute of an output result of the split operation node for the N sub-matrix multiplication operation nodes, and a split operation attribute of the split operation node.

[0305] The first updating unit 132 is configured to, if the input attribute is a pre-input attribute, and the split operation attribute is a row split operation attribute, update the split operation node to a pre-input deformation operation node for performing a deformation operation on a pre-input tensor; and a deformation operation attribute of the pre-input deformation operation node is a row deformation operation attribute.

[0306] The second updating unit 133 is configured to update the split operation node to a post-input deformation operation node for performing a deformation operation on the post-input tensor if the input attribute is the post-input attribute and the split operation attribute is the column split operation attribute; and the deformation operation attribute of the post-input deformation operation node is the column deformation operation attribute.

[0307] The specific function implementation manners of the first determining unit 131, the first updating unit 132, and the second updating unit 133 can be referred to the above Figure 14 The step S103 in the corresponding embodiment will not be repeated here.

[0308] Please refer to Figure 2 The N post-input sub-tensors have a size association relationship with the pre-input tensor and respectively carry different sequence numbers; the merging operation node includes a stacking operation node, and the de-merging operation node includes a de-stacking operation node associated with the stacking operation node.

[0309] The adding node module 14 is specifically configured to, when the first deformation operation node is the pre-input deformation operation node, add a stacking operation node for performing a stacking operation on the N post-input sub-tensors before the total matrix multiplication operation node and add a de-stacking operation node after the total matrix multiplication operation node, to obtain the optimized data flow graph.

[0310] The specific function implementation manner of the adding node module 14 can be referred to the above Figure 14 The step S204 in the corresponding embodiment will not be repeated here.

[0311] Please refer to Figure 4 The tensor dimensions corresponding to the N post-input sub-tensors are all the same, and the sizes corresponding to the N tensor dimensions are all the same; the tensor dimension corresponding to the pre-input tensor is equal to the N tensor dimensions, and the size of the tensor column dimension corresponding to the pre-input tensor is equal to the size of the tensor row dimension corresponding to the N post-input sub-tensors.

[0312] The data flow graph processing apparatus 1 can further include a first operation module 15.

[0313] The first operation module 15 is configured to input the pre-input tensor into the pre-input deformation operation node in the optimized data flow graph, and perform a deformation operation on the pre-input tensor through the pre-input deformation operation node to obtain a pre-input dimension-increased tensor.

[0314] The first operation module 15 is further configured to input the N post-input sub-tensors into a stacking operation node in the optimized data flow graph in sequence according to the sequence number, perform a stacking operation on the N post-input sub-tensors through the stacking operation node, and obtain a post-input stacked tensor; the post-input stacked tensor comprises a tensor batch dimension with a size equal to N, and a size of a tensor dimension corresponding to the post-input stacked tensor, except for the tensor batch dimension with the size equal to N, is equal to sizes of N tensor dimensions respectively corresponding thereto.

[0315] The first operation module 15 is further configured to input the pre-input augmented tensor and the post-input stacked tensor into a total matrix multiplication operation node, perform a matrix multiplication operation on the pre-input augmented tensor and the post-input stacked tensor through the total matrix multiplication operation node, and obtain a first total output tensor.

[0316] The first operation module 15 is further configured to input the first total output tensor into a de-stacking operation node, perform a de-stacking operation on a tensor belonging to a tensor batch dimension in the first total output tensor through the de-stacking operation node, and obtain N first output sub-tensors.

[0317] The specific function implementation of the first operation module 15 can be referred to the foregoing Figure 14 The steps S205-S208 in the corresponding embodiment will not be repeated here.

[0318] Please refer to Figure 4 The first operation module 15 can comprise a second determination unit 151, a first operation unit 152, and a third update unit 153.

[0319] The second determination unit 151 is configured to determine, according to a row deformation operation attribute of the pre-input deformation operation node, a tensor belonging to a tensor row dimension in the pre-input tensor as a first to-be-deformed tensor.

[0320] The first operation unit 152 is configured to perform a deformation operation on the N first to-be-deformed tensors to obtain a first augmented tensor; a size of a first dimension corresponding to the first augmented tensor is equal to N, and a size of the first to-be-deformed tensor is equal to N times a size of a second dimension corresponding to the first augmented tensor.

[0321] The third update unit 153 is configured to update the first to-be-deformed tensor in the pre-input tensor as the first augmented tensor to obtain the pre-input augmented tensor.

[0322] The specific function implementation of the second determination unit 151, the first operation unit 152, and the third update unit 153 can be referred to the foregoing Figure 14 The step S205 in the corresponding embodiment will not be repeated here.

[0323] Please refer toFigure 4 The N input sub-tensors include a dimension association relationship with the post-input tensor, and respectively carry N pre-input sub-tensors with different serial numbers; the merging operation node includes a stacking operation node, and the de-merging operation node includes a de-stacking operation node associated with the stacking operation node;

[0324] The adding node module 14 is specifically configured to, when the first deformation operation node is a post-input deformation operation node, add a stacking operation node for performing a stacking operation on the N pre-input tensors before the total matrix multiplication operation node, add a post-input transposition operation node between the total matrix multiplication operation node and the post-input deformation operation node, and add a de-stacking operation node after the total matrix multiplication operation node, to obtain the optimized data flow graph.

[0325] The specific function implementation manner of the adding node module 14 can be referred to the above Figure 14 Corresponding to step S304 in the embodiment, details are not repeated here.

[0326] Please refer to Figure 6 The N pre-input sub-tensors respectively correspond to the same tensor dimension, and the N tensor dimensions respectively correspond to the same size; the tensor dimension corresponding to the post-input tensor is equal to the N tensor dimensions, and the size of the tensor row dimension corresponding to the post-input tensor is equal to the size of the tensor column dimension respectively corresponding to the N pre-input sub-tensors;

[0327] The data flow graph processing apparatus 1 can further include a second operation module 16.

[0328] The second operation module 16 is configured to input the post-input tensor into a post-input deformation operation node in the optimized data flow graph, and perform a deformation operation on the post-input tensor through the post-input deformation operation node to obtain a post-input dimension-increased tensor.

[0329] The second operation module 16 is further configured to input the post-input dimension-increased tensor into a post-input transposition operation node, and perform a transposition operation on the tensors belonging to the tensor dimension to be transposed in the post-input dimension-increased tensor through the post-input transposition operation node to obtain a post-input transposed tensor; the tensor dimension to be transposed includes the tensor row dimension and the tensor batch dimension corresponding to the post-input dimension-increased tensor.

[0330] The second operation module 16 is further configured to input the N pre-input sub-tensors into a stacking operation node in the optimized data flow graph in a serial number order, and perform a stacking operation on the N pre-input sub-tensors through the stacking operation node to obtain a pre-input stacked tensor; the pre-input stacked tensor includes a tensor batch dimension with a size equal to N, and the size of the tensor dimension of the pre-input stacked tensor, except for the tensor batch dimension with the size equal to N, is equal to the size of the N tensor dimensions respectively corresponding to the size.

[0331] The second operation module 16 is further configured to input the post-input transposed tensor and the pre-input stacked tensor into a total matrix multiplication operation node, and perform matrix multiplication operation on the post-input transposed tensor and the pre-input stacked tensor through the total matrix multiplication operation node to obtain a second total output tensor.

[0332] The second operation module 16 is further configured to input the second total output tensor into a de-stacking operation node, and perform de-stacking operation on the tensors belonging to the tensor batch dimension in the second total output tensor through the de-stacking operation node to obtain N second output sub-tensors.

[0333] The specific function implementation of the second operation module 16 can be referred to the above Figure 14 The steps S305-S309 in the corresponding embodiment will not be repeated here.

[0334] Please refer to Figure 6 The second operation module 16 can include a third determination unit 161, a second operation unit 162, and a fourth update unit 163.

[0335] The third determination unit 161 is configured to determine the tensors belonging to the tensor column dimension in the post-input tensor as second to-be-transformed tensors according to the column transformation operation attribute of the post-input transformation operation node.

[0336] The second operation unit 162 is configured to perform transformation operation on the N second to-be-transformed tensors to obtain a second dimension-increased tensor; the size of the first dimension corresponding to the second dimension-increased tensor is equal to N, and the size of the second to-be-transformed tensor is equal to N times the size of the second dimension corresponding to the second dimension-increased tensor.

[0337] The fourth update unit 163 is configured to update the second to-be-transformed tensors in the post-input tensor as the second dimension-increased tensor to obtain a post-input dimension-increased tensor.

[0338] The specific function implementation of the third determination unit 161, the second operation unit 162, and the fourth update unit 163 can be referred to the above Figure 14 The step S306 in the corresponding embodiment will not be repeated here.

[0339] Please refer to Figure 6 The N sub-input tensors include N post-input sub-tensors having a size association relationship with the pre-input tensor and carrying different sequence numbers respectively; the merging operation node includes a splicing operation node, and the de-merging operation node includes a de-splicing operation node associated with the splicing operation node.

[0340] The adding node module 14 can include a first adding unit 141 and a second adding unit 142.

[0341] The first adding unit 141 is configured to add the second morphing operation node before the total matrix multiplication operation node and add the splicing operation node for splicing the N post-input sub-tensors before the second morphing operation node when the first morphing operation node is a pre-input morphing operation node.

[0342] The second adding unit 142 is configured to add the third morphing operation node associated with the second morphing operation node after the total matrix multiplication operation node and add the de-splicing operation node after the third morphing operation node to obtain the optimized data flow graph. The node relationship between the second morphing operation node and the pre-input morphing operation node is a parallel node relationship.

[0343] The specific function implementation of the first adding unit 141 and the second adding unit 142 can be referred to the above Figure 14 The steps S404-S405 in the corresponding embodiment will not be repeated here.

[0344] Please refer to Figure 8 The N post-input sub-tensors respectively correspond to the same tensor dimensions, and the N tensor dimensions respectively correspond to the same sizes. The pre-input tensor corresponds to the N tensor dimensions, and the size of the tensor column dimension corresponding to the pre-input tensor is equal to the size of the tensor row dimension corresponding to the N post-input sub-tensors.

[0345] The data flow graph processing apparatus 1 can further include a third operation module 17.

[0346] The third operation module 17 is configured to input the pre-input tensor into the pre-input morphing operation node in the optimized data flow graph, and perform a morphing operation on the pre-input tensor through the pre-input morphing operation node to obtain a pre-input dimension-increased tensor.

[0347] The third operation module 17 is further configured to input the N post-input sub-tensors into the splicing operation node in the optimized data flow graph in the order of the sequence numbers, and perform a splicing operation on the N post-input sub-tensors through the splicing operation node to obtain a post-input spliced tensor. The post-input spliced tensor corresponds to the N tensor dimensions.

[0348] The third operation module 17 is further configured to input the post-input spliced tensor into the second morphing operation node, and perform a morphing operation on the post-input spliced tensor through the second morphing operation node to obtain a post-input morphed tensor.

[0349] The third operation module 17 is further configured to input the pre-input augmented dimension tensor and the post-input deformed tensor into a total matrix multiplication operation node, perform matrix multiplication operation on the pre-input augmented dimension tensor and the post-input deformed tensor through the total matrix multiplication operation node, and obtain a first deformed total output tensor;

[0350] The third operation module 17 is further configured to input the first deformed total output tensor into a third deformed operation node, perform deformed operation on the first deformed total output tensor through the third deformed operation node, and obtain a third total output tensor.

[0351] The third operation module 17 is further configured to input the third total output tensor into a de-splicing operation node, perform splitting operation on the tensors belonging to the tensor row dimension in the third total output tensor through the de-splicing operation node, and obtain N first output sub-tensors.

[0352] The specific function implementation of the third operation module 17 can be referred to the above Figure 14 The steps S406-S411 in the corresponding embodiment will not be repeated here.

[0353] Please refer to Figure 8 The third operation module 17 can include a third operation unit 171, a fourth determination unit 172, a fourth operation unit 173, and a fifth update unit 174.

[0354] The third operation unit 171 is configured to perform splicing operation on the N post-input sub-tensors in the tensor row dimension through a splicing operation node to obtain a post-input splicing tensor; the size of the tensor row dimension corresponding to the post-input splicing tensor is equal to the sum of the sizes of the tensor row dimensions corresponding to the N post-input sub-tensors respectively.

[0355] The third operation module 17 includes:

[0356] The fourth determination unit 172 is configured to determine the tensors belonging to the tensor row dimension in the post-input splicing tensor as a third deformed tensor.

[0357] The fourth operation unit 173 is configured to perform deformed operation on the third deformed tensor through a second deformed operation node to obtain a third augmented dimension tensor; the size of the first dimension corresponding to the third augmented dimension tensor is equal to N, and the size of the third deformed tensor is equal to N times the size of the second dimension corresponding to the third augmented dimension tensor.

[0358] The fifth update unit 174 is configured to update the third deformed tensor in the post-input splicing tensor as the third augmented dimension tensor to obtain a post-input deformed tensor.

[0359] The specific function implementation manners of the third operation unit 171, the fourth determination unit 172, the fourth operation unit 173, and the fifth update unit 174 can be referred to the above Figure 14 Steps S407-S408 in the corresponding embodiment will not be repeated here.

[0360] Please refer to Figure 8 The third operation module 17 can include a fifth determination unit 175, a fifth operation unit 176, and a sixth update unit 177.

[0361] The fifth determination unit 175 is configured to determine a tensor belonging to a tensor target dimension in the first total output tensor to be deformed as a fourth tensor to be deformed; the tensor target dimension includes a tensor batch dimension and a tensor row dimension corresponding to the first total output tensor to be deformed.

[0362] The fifth operation unit 176 is configured to perform a deformation operation on the fourth tensor to be deformed through a third deformation operation node to obtain a target reduced dimension tensor; the size of the target reduced dimension tensor is equal to N times the size of the second dimension corresponding to the fourth tensor to be deformed.

[0363] The sixth update unit 177 is configured to update the fourth tensor to be deformed in the first total output tensor to be deformed as the target reduced dimension tensor to obtain a third total output tensor.

[0364] The specific function implementation manners of the fifth determination unit 175, the fifth operation unit 176, and the sixth update unit 177 can be referred to the above Figure 14 Step S410 in the corresponding embodiment will not be repeated here.

[0365] Please refer to Figure 8 The N input tensors include N front input sub-tensors having a size association relationship with the rear input tensor and carrying different serial numbers; the merging operation node includes a splicing operation node, and the de-merging operation node includes a de-splicing operation node associated with the splicing operation node.

[0366] The adding node module 14 can include a third adding unit 143, a fourth adding unit 144, and a fifth adding unit 145.

[0367] The third adding unit 143 is configured to add a fourth deformation operation node before the total matrix multiplication operation node and add a splicing operation node for splicing the N front input sub-tensors when the first deformation operation node is a rear input deformation operation node.

[0368] The fourth adding unit 144 is configured to add a rear input transposition operation node between the total matrix multiplication operation node and the rear input deformation operation node.

[0369] The fifth adding unit 145 is configured to add a fifth deformation operation node associated with the fourth deformation operation node after the total matrix multiplication operation node, and add a de-splicing operation node after the fifth deformation operation node to obtain the optimized data flow graph; the node relationship between the fourth deformation operation node and the post-input deformation operation node is a parallel node relationship.

[0370] The specific function implementation manners of the third adding unit 143, the fourth adding unit 144 and the fifth adding unit 145 can be referred to the above Figure 14 The steps S501-S506 in the corresponding embodiment will not be repeated here.

[0371] Please refer to Figure 10 The tensor dimensions corresponding to the N pre-input sub-tensors are all the same, and the sizes of the N tensor dimensions are all the same; the tensor dimension corresponding to the post-input tensor is equal to the N tensor dimensions, and the size of the tensor row dimension corresponding to the post-input tensor is equal to the size of the tensor column dimension corresponding to the N pre-input sub-tensors;

[0372] The data flow graph processing apparatus 1 can further include a fourth operation module 18.

[0373] The fourth operation module 18 is configured to input the post-input tensor into a post-input deformation operation node in the optimized data flow graph, and perform a deformation operation on the post-input tensor through the post-input deformation operation node to obtain a post-input dimension-increased tensor.

[0374] The fourth operation module 18 is further configured to input the post-input dimension-increased tensor into a post-input transposition operation node, and perform a transposition operation on the tensors belonging to the tensor dimensions to be transposed in the post-input dimension-increased tensor through the post-input transposition operation node to obtain a post-input transposed tensor; the tensor dimensions to be transposed include the tensor row dimension and the tensor batch dimension corresponding to the post-input dimension-increased tensor.

[0375] The fourth operation module 18 is further configured to input the N pre-input sub-tensors into a splicing operation node in the optimized data flow graph in the order of the sequence numbers, and perform a splicing operation on the N pre-input sub-tensors through the splicing operation node to obtain a first pre-input spliced tensor; the tensor dimension corresponding to the first pre-input spliced tensor is equal to the N tensor dimensions.

[0376] The fourth operation module 18 is further configured to input the first pre-input spliced tensor into the fourth deformation operation node, and perform a deformation operation on the first pre-input spliced tensor through the fourth deformation operation node to obtain a first pre-input deformed tensor.

[0377] The fourth operation module 18 is further configured to input the first pre-input deformation tensor and the post-input transposed tensor into a total matrix multiplication operation node, perform matrix multiplication operation on the first pre-input deformation tensor and the post-input transposed tensor through the total matrix multiplication operation node, and obtain a second total output tensor to be deformed.

[0378] The fourth operation module 18 is further configured to input the second total output tensor to be deformed into a fifth deformation operation node, perform deformation operation on the second total output tensor to be deformed through the fifth deformation operation node, and obtain a fourth total output tensor.

[0379] The fourth operation module 18 is further configured to input the fourth total output tensor into a de-splicing operation node, perform splitting operation on the tensors belonging to the tensor row dimension in the fourth total output tensor through the de-splicing operation node, and obtain N second output sub-tensors.

[0380] The specific function implementation of the fourth operation module 18 can be referred to the above Figure 14 The steps S507-S513 in the corresponding embodiment will not be repeated here.

[0381] Please refer to Figure 10 The fourth operation module 18 can include a sixth operation unit 181, a sixth determination unit 182, a seventh operation unit 183, and a seventh update unit 184.

[0382] The sixth operation unit 181 is configured to perform splicing operation on the N pre-input sub-tensors in the tensor row dimension through a splicing operation node to obtain a first pre-input spliced tensor; the size of the tensor row dimension corresponding to the first pre-input spliced tensor is equal to the sum of the sizes of the tensor row dimensions corresponding to the N pre-input sub-tensors, respectively.

[0383] The fourth operation module 18 includes:

[0384] The sixth determination unit 182 is configured to determine the tensors belonging to the tensor row dimension in the first pre-input spliced tensor as a fifth tensor to be deformed.

[0385] The seventh operation unit 183 is configured to perform deformation operation on the fifth tensor to be deformed through a fourth deformation operation node to obtain a fourth dimension-increased tensor; the size of the first dimension corresponding to the fourth dimension-increased tensor is equal to N, and the size of the fifth tensor to be deformed is equal to N times the size of the second dimension corresponding to the fourth dimension-increased tensor.

[0386] The seventh update unit 184 is configured to update the fifth tensor to be deformed in the first pre-input spliced tensor as the fourth dimension-increased tensor to obtain a first pre-input deformation tensor.

[0387] The specific function implementation manners of the sixth operation unit 181, the sixth determination unit 182, the seventh operation unit 183, and the seventh update unit 184 can be referred to the above Figure 14 Steps S509-S510 in the corresponding embodiment will not be repeated here.

[0388] Please refer to Figure 10 The fourth operation module 18 can include a seventh determination unit 185, an eighth operation unit 186, and an eighth update unit 187.

[0389] The seventh determination unit 185 is configured to determine a tensor belonging to a tensor target dimension in the second total output tensor to be deformed as a sixth tensor to be deformed; the tensor target dimension includes a tensor batch dimension and a tensor row dimension corresponding to the second total output tensor to be deformed.

[0390] The eighth operation unit 186 is configured to perform a deformation operation on the sixth tensor to be deformed through a fifth deformation operation node to obtain a first reduced dimension tensor; the size of the first reduced dimension tensor is equal to N times the size of the second dimension corresponding to the sixth tensor to be deformed.

[0391] The eighth update unit 187 is configured to update the sixth tensor to be deformed in the second total output tensor to be deformed as the first reduced dimension tensor to obtain a fourth total output tensor.

[0392] The specific function implementation manners of the seventh determination unit 185, the eighth operation unit 186, and the eighth update unit 187 can be referred to the above Figure 14 Steps S512 in the corresponding embodiment will not be repeated here.

[0393] Please refer to Figure 10 The tensor dimensions corresponding to the N front input sub-tensors are all the same, and the sizes of the N tensor dimensions are all the same; the tensor dimension corresponding to the post-input tensor is equal to the N tensor dimensions, and the size of the tensor row dimension corresponding to the post-input tensor is equal to the size of the tensor column dimension corresponding to the N front input sub-tensors.

[0394] The data flow graph processing apparatus 1 can further include a fifth operation module 19.

[0395] The fifth operation module 19 is configured to delete the post-input transposition operation node in the optimized data flow graph to obtain a target optimized data flow graph when the total matrix multiplication operation node has operation authority on the tensor whose tensor batch dimension belongs to the intermediate dimension.

[0396] The fifth operation module 19 is further configured to input the post-input tensor to a post-input deformation operation node in the target optimized data flow graph, and perform a deformation operation on the post-input tensor through the post-input deformation operation node to obtain a post-input dimension-increased tensor.

[0397] The fifth operation module 19 is further configured to input the N front input sub-tensors into a splicing operation node in the target optimization data flow graph in sequence, perform splicing operation on the N front input sub-tensors through the splicing operation node, and obtain a second front input splicing tensor; the tensor dimension corresponding to the second front input splicing tensor is equal to the N tensor dimensions; the size of the tensor column dimension corresponding to the second front input splicing tensor is equal to the sum of the sizes of the tensor column dimensions corresponding to the N back input sub-tensors respectively;

[0398] The fifth operation module 19 is further configured to input the second front input splicing tensor into a fourth deformation operation node, perform deformation operation on the second front input splicing tensor through the fourth deformation operation node, and obtain a second front input deformation tensor;

[0399] The fifth operation module 19 is further configured to input the second front input deformation tensor and the back input dimension-increasing tensor into a total matrix multiplication operation node respectively, perform matrix multiplication operation on the second front input deformation tensor and the back input dimension-increasing tensor through the total matrix multiplication operation node, and obtain a third to-be-deformed total output tensor;

[0400] The fifth operation module 19 is further configured to input the third to-be-deformed total output tensor into a fifth deformation operation node, perform deformation operation on the third to-be-deformed total output tensor through the fifth deformation operation node, and obtain a fifth total output tensor;

[0401] The fifth operation module 19 is further configured to input the fifth total output tensor into a de-splicing operation node, perform splitting operation on the tensors belonging to the tensor column dimension in the fifth total output tensor through the de-splicing operation node, and obtain N second output sub-tensors.

[0402] The specific function implementation of the fifth operation module 19 can be referred to the above Figure 14 The steps S607-S613 in the corresponding embodiments will not be repeated here.

[0403] Please refer to Figure 2 The fifth operation module 19 can include a ninth operation unit 191, an eighth determination unit 192, and a ninth update unit 193.

[0404] The ninth operation unit 191 is configured to perform splicing operation on the N front input sub-tensors in the tensor column dimension through the splicing operation node, and obtain a second front input splicing tensor; the size of the tensor column dimension corresponding to the second front input splicing tensor is equal to the sum of the sizes of the tensor column dimensions corresponding to the N back input sub-tensors respectively;

[0405] The fifth operation module 19 includes:

[0406] The eighth determination unit 192 is configured to determine, as the seventh to-be-deformed tensor, a tensor belonging to a tensor column dimension in the second pre-input spliced tensor.

[0407] The ninth operation unit 191 is further configured to perform a deformation operation on the seventh to-be-deformed tensor through a fourth deformation operation node to obtain a fifth dimension-increased tensor; a size of a first dimension corresponding to the fifth dimension-increased tensor is equal to N, and a size of the seventh to-be-deformed tensor is equal to N times a size of a second dimension corresponding to the fifth dimension-increased tensor.

[0408] The ninth update unit 193 is configured to update the seventh to-be-deformed tensor in the second pre-input spliced tensor as the fifth dimension-increased tensor to obtain a second pre-input deformed tensor.

[0409] The specific function implementation manners of the ninth operation unit 191, the eighth determination unit 192, and the ninth update unit 193 can be referred to the above Figure 14 Steps S609-S610 in the corresponding embodiment will not be repeated here.

[0410] Please refer to Figure 12 The third acquisition module 13 can include a ninth determination unit 194, a tenth operation unit 195, and a tenth update unit 196.

[0411] The fifth operation module 19 includes:

[0412] The ninth determination unit 194 is configured to determine, as the eighth to-be-deformed tensor, a tensor belonging to a tensor target dimension in the third to-be-deformed total output tensor; the tensor target dimension includes a tensor batch dimension and a tensor column dimension corresponding to the sixth to-be-deformed total output tensor.

[0413] The tenth operation unit 195 is configured to perform a deformation operation on the eighth to-be-deformed tensor through a fifth deformation operation node to obtain a second dimension-reduced tensor; a size of the second dimension-reduced tensor is equal to N times a size of a second dimension corresponding to the eighth to-be-deformed tensor.

[0414] The tenth update unit 196 is configured to update the eighth to-be-deformed tensor in the third to-be-deformed total output tensor as the second dimension-reduced tensor to obtain a fifth total output tensor.

[0415] The specific function implementation manners of the ninth determination unit 194, the tenth operation unit 195, and the tenth update unit 196 can be referred to the above Figure 14 Step S612 in the corresponding embodiment will not be repeated here.

[0416] In the embodiment of the present application, by updating the N sub-matrix multiplication operation nodes in the original data flow graph to a total matrix multiplication operation node, the utilization rate of system resources can be improved; by updating the splitting operation node to a first deformation operation node for deforming the total input tensor, the total input tensor can be deformed into a tensor suitable for the total matrix multiplication operation node; further, by adding a merging operation node for merging the N sub-input tensors before the total matrix multiplication operation node, the N sub-input tensors can be merged into a tensor suitable for the total matrix multiplication operation node; by adding a de-merging operation node after the total matrix multiplication operation node, the data result of the original data flow graph can be maintained and the optimized data flow graph can be obtained. As can be seen from the above, the optimized data flow graph obtained by the present application by optimizing the original data flow graph can not only maintain the data result of the original data flow graph, but also improve the utilization rate of system resources.

[0417] Further, please refer to Figure 12 , Figure 15 is a structural schematic diagram of a computer device provided by an embodiment of the present application. As shown in Figure 15 , the computer device 1000 can include at least one processor 1001, such as a CPU, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can include a display screen (Display) and a keyboard (Keyboard), and the network interface 1004 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The memory 1005 can also be at least one storage device located away from the aforementioned processor 1001. As shown in Figure 15 , the memory 1005 as a computer storage medium can include an operating system, a network communication module, a user interface module, and a device control application program.

[0418] In the computer device 1000 shown in Figure 15 , the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application program stored in the memory 1005 to realize:

[0419] obtaining an original data flow graph and parsing the original data flow graph;

[0420] If it is determined that the original data flow graph comprises a split operation node and N sub-matrix multiplication operation nodes, a position of the split operation node in the original data flow graph is detected; N is a positive integer, and N is equal to or greater than 2;

[0421] If, in the original data flow graph, the split operation node is connected with the N sub-matrix multiplication operation nodes respectively, and the position of the split operation node in the original data flow graph is before the N sub-matrix multiplication operation nodes, the N sub-matrix multiplication operation nodes are updated as a total matrix multiplication operation node, and the split operation node is updated as a first deformation operation node for performing a deformation operation on a total input tensor;

[0422] A merge operation node for performing a merge operation on the N sub-input tensors is added before the total matrix multiplication operation node, and a de-merge operation node is added after the total matrix multiplication operation node, to obtain an optimized data flow graph; a node relationship between the merge operation node and the first deformation operation node is a parallel node relationship; and there is a size correlation relationship between the total input tensor and the N sub-input tensors.

[0423] It should be understood that the computer device 1000 described in the embodiments of the present application can perform the description of the data flow graph processing method in the foregoing Figure 15 、 Figure 2 、 Figure 4 、 Figure 6 、 Figure 8 and Figure 10 corresponding embodiments, and can also perform the description of the data flow graph processing apparatus 1 in the foregoing Figure 12 corresponding embodiments, which will not be described here again. In addition, the beneficial effects of using the same method will not be described again.

[0424] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, the computer program comprising program instructions, the program instructions being executed by a processor to implement the data flow graph processing method provided by each of the steps in the foregoing Figure 14 、 Figure 2 、 Figure 4 、 Figure 6 、 Figure 8 and Figure 10 , and the specific implementation can refer to the implementation provided by each of the steps in the foregoing Figure 12 、 Figure 2 、 Figure 4 、 Figure 6 、 Figure 8 and Figure 10 , which will not be described here again. In addition, the beneficial effects of using the same method will not be described again.

[0425] The computer readable storage medium can be an internal storage unit of the data flow graph processing apparatus or the computer device, for example, a hard disk or a memory of the computer device. The computer readable storage medium can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like. Further, the computer readable storage medium can include both the internal storage unit and the external storage device. The computer readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0426] The computer program product or the computer program includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium. The processor executes the computer instructions, so that the computer device can execute the description of the data flow graph processing method in the foregoing Figure 12 、 Figure 2 、 Figure 4 、 Figure 6 、 Figure 8 and Figure 10 embodiments, which will not be described herein. In addition, the description of the beneficial effects of using the same method will not be described herein.

[0427] The terms "first", "second", and the like in the specification and claims of the embodiments of the present application and the accompanying drawings are used to distinguish different objects, and are not used to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units that are not listed, or can optionally include other steps or units inherent to the process, method, device, product, or apparatus.

[0428] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in the above description in a general manner. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0429] The method and related apparatus provided by the embodiments of the present application are described with reference to the method flowchart and / or structural schematic diagram provided by the embodiments of the present application. Each flow and / or block in the method flowchart and / or structural schematic diagram and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. The computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device implemented in the flow Figure 12 The computer program instructions can also be stored in a computer readable memory capable of causing a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices that realize the functions specified in one or more flows and / or structural schematic diagrams. Figure 1 The computer program instructions can also be stored in a computer readable memory capable of causing a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices that realize the functions specified in one or more flows and / or structural schematic diagrams. Figure 1 The computer program instructions can also be stored in a computer readable memory capable of causing a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices that realize the functions specified in one or more flows and / or structural schematic diagrams. Figure 1 The computer program instructions can also be stored in a computer readable memory capable of causing a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices that realize the functions specified in one or more flows and / or structural schematic diagrams. Figure 1 Figure 1 The computer program instructions can also be stored in a computer readable memory capable of causing a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction devices that realize the functions specified in one or more flows and / or structural schematic diagrams.

[0430] The above disclosure is only the preferred embodiments of the present application, and of course cannot limit the scope of the rights of the present application, so the equivalent changes made by the claims of the present application are still within the scope of the present application.

Claims

1. A data flow graph processing method, characterized in that, include: Obtain the original data flow graph and parse the original data flow graph; If the original data flow graph is parsed to include split operation nodes and N submatrix multiplication operation nodes, then the position of the split operation node in the original data flow graph is detected; N is a positive integer, and N is equal to or greater than 2; If, in the original data flow graph, the split operation node is connected to the N submatrix multiplication operation nodes respectively, and the position of the split operation node in the original data flow graph is before the N submatrix multiplication operation nodes, then the N submatrix multiplication operation nodes are updated to the total matrix multiplication operation node, and the split operation node is updated to the first deformation operation node for performing deformation operations on the total input tensor. A merge operation node for merging N sub-input tensors is added before the total matrix multiplication operation node, and a demerge operation node is added after the total matrix multiplication operation node to obtain an optimized data flow graph; the node relationship between the merge operation node and the first deformation operation node is a parallel node relationship; There is a size relationship between the total input tensor and the N sub-input tensors.

2. The method according to claim 1, characterized in that, The first deformation operation node includes a pre-input deformation operation node or a post-input deformation operation node; the total input tensor includes a pre-input tensor or a post-input tensor; The step of updating the splitting operation node to a first deformation operation node for performing deformation operations on the total input tensor includes: Determine the input attributes of the output result of the split operation node for the N submatrix multiplication operation nodes, and the split operation attributes of the split operation node; If the input attribute is a preceding input attribute and the split operation attribute is a row split operation attribute, then the split operation node is updated to the preceding input deformation operation node for performing deformation operations on the preceding input tensor; the deformation operation attribute of the preceding input deformation operation node is a row deformation operation attribute. If the input attribute is a post-input attribute and the split operation attribute is a column split operation attribute, then the split operation node is updated to the post-input deformation operation node for performing deformation operations on the post-input tensor; the deformation operation attribute of the post-input deformation operation node is a column deformation operation attribute.

3. The method according to claim 2, characterized in that, The N sub-input tensors include N subsequent input sub-tensors that are size-related to the preceding input tensor and carry different indices; the merge operation node includes a stack operation node, and the de-merge operation node includes a de-stack operation node associated with the stack operation node; The process involves adding a merge operation node before the total matrix multiplication operation node to perform a merge operation on the N sub-input tensors, and adding a demerge operation node after the total matrix multiplication operation node to obtain an optimized data flow graph, including: When the first deformation operation node is the pre-input deformation operation node, a stacking operation node for stacking the N post-input sub-tensors is added before the total matrix multiplication operation node, and a destacking operation node is added after the total matrix multiplication operation node to obtain the optimized data flow graph.

4. The method according to claim 3, characterized in that, The N subsequent input sub-tensors each have the same tensor dimension, and the N tensor dimensions each have the same size; the tensor dimension corresponding to the preceding input tensor is equal to the N tensor dimensions, and the size of the tensor column dimension corresponding to the preceding input tensor is equal to the size of the tensor row dimension corresponding to the N subsequent input sub-tensors. The method further includes: The preceding input tensor is input to the preceding input deformation operation node in the optimized data flow graph. The preceding input tensor is deformed through the preceding input deformation operation node to obtain the preceding input dimension-enhanced tensor. In sequence, the N subsequent input sub-tensors are input to the stacking operation node in the optimized data flow graph. The stacking operation node performs a stacking operation on the N subsequent input sub-tensors to obtain a subsequent input stacked tensor. The subsequent input stacked tensor includes a tensor batch dimension of size N, and the size of the tensor dimensions other than the tensor batch dimension of size N in the tensor dimensions corresponding to the subsequent input stacked tensor is equal to the size corresponding to each of the N tensor dimensions. The preceding input dimension-enhanced tensor and the following input stacked tensor are respectively input to the total matrix multiplication operation node. The total matrix multiplication operation node performs matrix multiplication on the preceding input dimension-enhanced tensor and the following input stacked tensor to obtain the first total output tensor. The first total output tensor is input to the destacking operation node. Through the destacking operation node, the tensors belonging to the tensor batch dimension in the first total output tensor are destacking to obtain N first output sub-tensors.

5. The method according to claim 4, characterized in that, The step of performing a deformation operation on the preceding input tensor through the preceding input deformation operation node to obtain a preceding input dimension-enhanced tensor includes: Based on the row deformation operation attribute of the preceding input deformation operation node, the tensor belonging to the row dimension of the preceding input tensor is determined as the first tensor to be deformed. The first tensor to be deformed is deformed according to N to obtain a first dimension-enhanced tensor; the size of the first dimension of the first dimension-enhanced tensor is equal to N, and the size of the first tensor to be deformed is equal to N times the size of the second dimension of the first dimension-enhanced tensor. The first tensor to be deformed in the preceding input tensor is updated to the first dimension-increasing tensor to obtain the preceding input dimension-increasing tensor.

6. The method according to claim 2, characterized in that, The N sub-input tensors include N preceding input sub-tensors that are size-related to the following input tensor and carry different indices; the merge operation node includes a stack operation node, and the de-merge operation node includes a de-stack operation node associated with the stack operation node; Before the total matrix multiplication operation node, a merge operation node for merging N sub-input tensors is added; after the total matrix multiplication operation node, a demerge operation node is added to obtain an optimized data flow graph, including: When the first deformation operation node is the post-input deformation operation node, a stacking operation node for stacking the N pre-input tensors is added before the total matrix multiplication operation node, a post-input transpose operation node is added between the total matrix multiplication operation node and the post-input deformation operation node, and the destacking operation node is added after the total matrix multiplication operation node to obtain the optimized data flow graph.

7. The method according to claim 6, characterized in that, The N preceding input sub-tensors each have the same tensor dimension, and the N tensor dimensions each have the same size; the tensor dimension corresponding to the following input tensor is equal to the N tensor dimensions, and the size of the tensor row dimension corresponding to the following input tensor is equal to the size of the tensor column dimension corresponding to the N preceding input sub-tensors. The method further includes: The post-input tensor is input to the post-input deformation operation node in the optimized data flow graph. The post-input tensor is deformed through the post-input deformation operation node to obtain the post-input dimension-enhanced tensor. The post-input dimension-enhanced tensor is input to the post-input transpose operation node. Through the post-input transpose operation node, the tensors in the post-input dimension-enhanced tensor that belong to the dimension of the tensor to be transposed are transposed to obtain the post-input transpose tensor. The dimension of the tensor to be transposed includes the tensor row dimension and the tensor batch dimension corresponding to the post-input dimension-enhanced tensor. The N preceding input sub-tensors are input to the stacking operation node in the optimized data flow graph in sequence. The N preceding input sub-tensors are stacked through the stacking operation node to obtain the preceding input stacked tensor. The preceding input stacked tensor includes a tensor batch dimension of size N, and the size of the tensor dimension other than the tensor batch dimension of size N is equal to the size of the N tensor dimensions respectively. The transposed tensor of the post-input and the stacked tensor of the pre-input are respectively input to the total matrix multiplication operation node. The total matrix multiplication operation node performs matrix multiplication on the transposed tensor of the post-input and the stacked tensor of the pre-input to obtain the second total output tensor. The second total output tensor is input to the destacking operation node. Through the destacking operation node, the tensors belonging to the tensor batch dimension in the second total output tensor are destacking to obtain N second output sub-tensors.

8. The method according to claim 7, characterized in that, The step of performing a deformation operation on the post-input tensor through the post-input deformation operation node to obtain a post-input dimension-enhanced tensor includes: Based on the column deformation operation attribute of the post-input deformation operation node, the tensors belonging to the column dimension of the post-input tensor are determined as the second tensor to be deformed. The second tensor to be deformed is deformed according to N to obtain a second dimension-enhanced tensor; the size of the first dimension of the second dimension-enhanced tensor is equal to N, and the size of the second tensor to be deformed is equal to N times the size of the second dimension of the second dimension-enhanced tensor. The second tensor to be deformed in the post-input tensor is updated to the second dimension-increasing tensor to obtain the post-input dimension-increasing tensor.

9. The method according to claim 2, characterized in that, The N sub-input tensors include N subsequent input sub-tensors that are size-related to the preceding input tensor and carry different indices; the merge operation node includes a splicing operation node, and the de-merge operation node includes a de-splicing operation node associated with the splicing operation node; The process involves adding a merge operation node before the total matrix multiplication operation node to perform a merge operation on the N sub-input tensors, and adding a demerge operation node after the total matrix multiplication operation node to obtain an optimized data flow graph, including: When the first deformation operation node is the pre-input deformation operation node, a second deformation operation node is added before the total matrix multiplication operation node, and a splicing operation node for splicing the N post-input sub-tensors is added before the second deformation operation node. A third deformation operation node associated with the second deformation operation node is added after the total matrix multiplication operation node, and the de-splitting operation node is added after the third deformation operation node to obtain the optimized data flow graph; the node relationship between the second deformation operation node and the preceding input deformation operation node is a parallel node relationship.

10. The method according to claim 9, characterized in that, The N subsequent input sub-tensors each have the same tensor dimension, and the N tensor dimensions each have the same size; the tensor dimension corresponding to the preceding input tensor is equal to the N tensor dimensions, and the size of the tensor column dimension corresponding to the preceding input tensor is equal to the size of the tensor row dimension corresponding to the N subsequent input sub-tensors. The method further includes: The preceding input tensor is input to the preceding input deformation operation node in the optimized data flow graph. The preceding input tensor is deformed through the preceding input deformation operation node to obtain the preceding input dimension-enhanced tensor. In sequence, the N post-input sub-tensors are input to the concatenation operation node in the optimized data flow graph. The concatenation operation node performs a concatenation operation on the N post-input sub-tensors to obtain the post-input concatenated tensor. The tensor dimension corresponding to the post-input concatenated tensor is equal to the N tensor dimensions. The post-input splicing tensor is input to the second deformation operation node, and the post-input splicing tensor is deformed through the second deformation operation node to obtain the post-input deformed tensor. The preceding input dimension-enhanced tensor and the following input deformation tensor are respectively input to the total matrix multiplication operation node. Through the total matrix multiplication operation node, matrix multiplication operation is performed on the preceding input dimension-enhanced tensor and the following input deformation tensor to obtain the first total output tensor to be deformed. The first total output tensor to be deformed is input to the third deformation operation node. The first total output tensor to be deformed is deformed through the third deformation operation node to obtain the third total output tensor. The third total output tensor is input to the de-splicing operation node. Through the de-splicing operation node, the tensors belonging to the tensor row dimension in the third total output tensor are split to obtain N first output sub-tensors.

11. The method according to claim 10, characterized in that, The step of concatenating the N post-input sub-tensors through the concatenation operation node to obtain the post-input concatenation tensor includes: Through the splicing operation node, the N subsequent input sub-tensors are spliced ​​in the tensor row dimension to obtain the subsequent input spliced ​​tensor; the size of the tensor row dimension corresponding to the subsequent input spliced ​​tensor is equal to the sum of the sizes of the tensor row dimensions corresponding to the N subsequent input sub-tensors respectively. The step of performing a deformation operation on the post-input splicing tensor through the second deformation operation node to obtain the post-input deformed tensor includes: The tensors belonging to the tensor row dimension in the post-input splicing tensor are determined as the third tensor to be deformed. The third tensor to be deformed is deformed by the second deformation operation node to obtain a third dimension-enhanced tensor; the size of the first dimension of the third dimension-enhanced tensor is equal to N, and the size of the third tensor to be deformed is equal to N times the size of the second dimension of the third dimension-enhanced tensor. The third tensor to be deformed in the post-input splicing tensor is updated to the third dimension-increasing tensor to obtain the post-input deformable tensor.

12. The method according to claim 10, characterized in that, The step of performing a deformation operation on the first total output tensor to be deformed through the third deformation operation node to obtain the third total output tensor includes: The tensors belonging to the target dimension of the first total output tensor to be deformed are determined as the fourth tensor to be deformed; the target dimension of the tensor includes the tensor batch dimension and the tensor row dimension corresponding to the first total output tensor to be deformed. The fourth tensor to be deformed is deformed through the third deformation operation node to obtain the target dimension-reduced tensor; the size of the target dimension-reduced tensor is N times the size of the second dimension corresponding to the fourth tensor to be deformed. The fourth tensor to be deformed in the first total output tensor to be deformed is updated to the target dimension reduction tensor to obtain the third total output tensor.

13. The method according to claim 3, characterized in that, The N sub-input tensors include N preceding input sub-tensors that are size-related to the following input tensor and carry different indices; the merge operation node includes a splicing operation node, and the de-merge operation node includes a de-splicing operation node associated with the splicing operation node; The process involves adding a merge operation node before the total matrix multiplication operation node to perform a merge operation on the N sub-input tensors, and adding a demerge operation node after the total matrix multiplication operation node to obtain an optimized data flow graph, including: When the first deformation operation node is the post-input deformation operation node, a fourth deformation operation node is added before the total matrix multiplication operation node, and a splicing operation node for splicing the N pre-input sub-tensors is added before the fourth deformation node. Add a post-input transpose operation node between the total matrix multiplication operation node and the post-input transformation operation node; A fifth deformation operation node associated with the fourth deformation operation node is added after the total matrix multiplication operation node, and the de-splitting operation node is added after the fifth deformation operation node to obtain the optimized data flow graph; the node relationship between the fourth deformation operation node and the subsequent input deformation operation node is a parallel node relationship.

14. The method according to claim 13, characterized in that, The N preceding input sub-tensors each have the same tensor dimension, and the N tensor dimensions each have the same size; the tensor dimension corresponding to the following input tensor is equal to the N tensor dimensions, and the size of the tensor row dimension corresponding to the following input tensor is equal to the size of the tensor column dimension corresponding to the N preceding input sub-tensors. The method further includes: The post-input tensor is input to the post-input deformation operation node in the optimized data flow graph. The post-input tensor is deformed through the post-input deformation operation node to obtain the post-input dimension-enhanced tensor. The post-input dimension-enhanced tensor is input to the post-input transpose operation node. Through the post-input transpose operation node, the tensors in the post-input dimension-enhanced tensor that belong to the dimension of the tensor to be transposed are transposed to obtain the post-input transpose tensor. The dimension of the tensor to be transposed includes the tensor row dimension and the tensor batch dimension corresponding to the post-input dimension-enhanced tensor. In sequence, the N pre-input sub-tensors are input to the concatenation operation node in the optimized data flow graph. The concatenation operation node performs a concatenation operation on the N pre-input sub-tensors to obtain the first pre-input concatenated tensor. The tensor dimension corresponding to the first pre-input concatenated tensor is equal to the N tensor dimensions. The first pre-input splicing tensor is input to the fourth deformation operation node, and the first pre-input splicing tensor is deformed through the fourth deformation operation node to obtain the first pre-input deformed tensor. The first pre-input deformable tensor and the post-input transpose tensor are respectively input to the total matrix multiplication operation node. Through the total matrix multiplication operation node, matrix multiplication operation is performed on the first pre-input deformable tensor and the post-input transpose tensor to obtain the second total output tensor to be deformed. The second total output tensor to be deformed is input to the fifth deformation operation node. The second total output tensor to be deformed is deformed through the fifth deformation operation node to obtain the fourth total output tensor. The fourth total output tensor is input to the de-splicing operation node. Through the de-splicing operation node, the tensors belonging to the tensor row dimension in the fourth total output tensor are split to obtain N second output sub-tensors.

15. The method according to claim 13, characterized in that, The N preceding input sub-tensors each have the same tensor dimension, and the N tensor dimensions each have the same size; the tensor dimension corresponding to the following input tensor is equal to the N tensor dimensions, and the size of the tensor row dimension corresponding to the following input tensor is equal to the size of the tensor column dimension corresponding to the N preceding input sub-tensors. The method further includes: When the total matrix multiplication operation node has the permission to operate on tensors whose batch dimension belongs to the middle dimension, delete the post-input transpose operation node in the optimized data flow graph to obtain the target optimized data flow graph. The post-input tensor is input to the post-input deformation operation node in the target optimized data flow graph. The post-input tensor is deformed through the post-input deformation operation node to obtain the post-input dimension-enhanced tensor. In sequence, the N preceding input sub-tensors are input to the concatenation operation node in the target optimized data flow graph. The concatenation operation node performs a concatenation operation on the N preceding input sub-tensors to obtain a second preceding input concatenated tensor. The tensor dimension corresponding to the second preceding input concatenated tensor is equal to the dimensions of the N tensors. The size of the tensor column dimension corresponding to the second preceding input concatenated tensor is equal to the sum of the sizes of the tensor column dimensions corresponding to the N subsequent input sub-tensors. The second pre-input splicing tensor is input to the fourth deformation operation node, and the second pre-input splicing tensor is deformed through the fourth deformation operation node to obtain the second pre-input deformed tensor. The second pre-input deformable tensor and the post-input dimension-increasing tensor are respectively input to the total matrix multiplication operation node. Through the total matrix multiplication operation node, matrix multiplication operation is performed on the second pre-input deformable tensor and the post-input dimension-increasing tensor to obtain the third total output tensor to be deformed. The third total output tensor to be deformed is input to the fifth deformation operation node. The fifth deformation operation node is used to deform the third total output tensor to obtain the fifth total output tensor. The fifth total output tensor is input to the de-splicing operation node. Through the de-splicing operation node, the tensors belonging to the tensor column dimension in the fifth total output tensor are split to obtain N second output sub-tensors.

16. A computer device, characterized in that, include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide data communication functions, the memory is used to store computer programs, and the processor is used to call the computer programs to cause the computer device to perform the method according to any one of claims 1 to 15.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-15.

Images