Data flow graph processing method, device and equipment, and computer storage medium

Abstract

The present application discloses a data flow graph processing method, apparatus, device and computer storage medium, and relates to the field of computer technology. In the method, when it is detected that a plurality of matrix multiplication nodes that can be aggregated are connected before a merge operation node, the matrix multiplication nodes can be aggregated to obtain a new aggregate matrix multiplication node, and the merge operation node can be updated to a plurality of stacking operation nodes, that is, a plurality of input tensors of the same type of each matrix multiplication are processed into an output tensor through a stacking operation node, and finally processed through a matrix multiplication node, thereby realizing that the matrix multiplication operations of multiple small tensors can be aggregated into a matrix multiplication operation of a large tensor, greatly reducing the number of matrix multiplication nodes, thereby reducing the resources allocated to the matrix multiplication nodes, reducing the scheduling overhead, so that resources can be fully utilized, improving resource utilization, and improving data processing speed.

Description

Technical Field

[0001] The present application relates to the field of computer technology, in particular to the field of graph optimization technology, and provides a data flow graph processing method, device and equipment, and a computer storage medium. Background Art

[0002] A data flow diagram (DFD) is a tool used in structured analysis methods. It graphically depicts the flow and processing of data within a system. Because a data flow diagram only reflects the logical functions that a system must perform, it is a functional model. From the perspective of data transmission and processing, a data flow diagram graphically expresses the system's logical functions, the logical flow of data within the system, and the logical transformation process. It is a primary tool in structured system analysis methods and a graphical method for representing software models.

[0003] In the application scenario, the application process of the data flow graph specifically includes: first writing the data flow graph, and then submitting the written data flow graph to the stream computing platform. The stream computing platform can parse the data flow graph and then schedule it to different physical machines for distributed execution, thereby realizing different business scenarios.

[0004] For some data flow graphs, there may be multiple nodes with the same operation mode in the same data flow graph. Under related technologies, for such nodes, corresponding resources need to be allocated to each node separately, which leads to redundant allocation of resources and causes serious waste of system resources. In addition, especially the scheduling system with serial scheduling, it is necessary to wait for each node to be executed in turn before executing the next node, so that resources are not fully utilized and the resource utilization rate is not high. Summary of the Invention

[0005] The embodiments of the present application provide a data flow graph processing method, apparatus and equipment, and computer storage medium for reducing resource scheduling overhead during data flow processing and improving resource utilization.

[0006] In one aspect, a data flow graph processing method is provided, the method comprising:

[0007] Obtaining an original data flow graph and parsing the original data flow graph;

[0008] If the original data flow graph is parsed to include a merge operation node, and the input tensor of the merge operation node is the output tensor of N matrix multiplication nodes, then detecting whether the N matrix multiplication nodes meet the aggregation condition;

[0009] If the aggregation condition is met, the N matrix multiplication nodes are aggregated to obtain an aggregated matrix multiplication node, and the merge operation node is replaced with M stacking operation nodes; wherein the input tensor of each stacking operation node is: an input tensor of the same type among the M types of input tensors corresponding to the N matrix multiplication nodes; wherein M and N are both positive integers greater than 1;

[0010] The positions of the aggregate matrix multiplication node and the M stacking operation nodes are swapped, and a first deformation node is added after the aggregate matrix multiplication node to obtain a first optimized data flow graph; wherein, in the first optimized data flow graph, the output tensors of the M stacking operation nodes are the input tensors of the aggregate matrix multiplication node, and the output tensor of the aggregate matrix multiplication node is the input tensor of the first deformation node.

[0011] In one aspect, a data flow graph processing device is provided, the device comprising:

[0012] A flow graph parsing unit, configured to obtain an original data flow graph and parse the original data flow graph;

[0013] a detection unit, configured to detect whether the N matrix multiplication nodes meet an aggregation condition if the parsed original data flow graph includes a merge operation node, and the input tensor of the merge operation node is the output tensor of N matrix multiplication nodes;

[0014] a flow graph optimization unit, configured to aggregate the N matrix multiplication nodes to obtain an aggregated matrix multiplication node if an aggregation condition is met, and replace the merge operation node with M stacking operation nodes; further, swap the positions of the aggregated matrix multiplication node and the M stacking operation nodes, and add a first deformation node after the aggregated matrix multiplication node to obtain a first optimized data flow graph;

[0015] Among them, in the first optimized data flow graph, the input tensor of each stacking operation node is: the input tensor of the same type among the M types of input tensors corresponding to the N matrix multiplication nodes, M and N are both positive integers greater than 1, and the output tensor of the M stacking operation nodes is the input tensor of the aggregate matrix multiplication node, and the output tensor of the aggregate matrix multiplication node is the input tensor of the first deformation node.

[0016] Optionally, the M stacking operation nodes include a first stacking operation node and a second stacking operation node; the device further includes a configuration unit and a model running unit;

[0017] The configuration unit is configured to configure a data processing model according to the first optimized data flow graph, obtain N groups of input tensor combinations, and input the N groups of input tensor combinations into the data processing model; each group of input tensor combinations includes a pre-input tensor and a post-input tensor to be subjected to a matrix multiplication operation;

[0018] The model running unit is used to perform a stacking operation on the leading input tensor in the N groups of tensor combinations through the first stacking operation node in the data processing model to obtain a first stacked tensor; and to perform a stacking operation on the trailing input tensor in the N groups of tensor combinations through the second stacking operation node to obtain a second stacked tensor; to perform a matrix multiplication operation on the first stacked tensor and the second stacked tensor through the aggregate matrix multiplication node to obtain a first output tensor; and to transform the shape of the first output tensor through the first deformation node to obtain a target output tensor.

[0019] Optionally, the flow graph optimization unit is specifically configured to:

[0020] Determining whether an operator library includes a stacking operation operator corresponding to the stacking operation node; wherein the operator library provides a data processing operator corresponding to each node;

[0021] If the operator library includes the stacking operation operator, the merge operation node is replaced with the M stacking operation nodes.

[0022] Optionally, the flow graph optimization unit is further configured to:

[0023] If the operator library does not support the stacking operation, add M-1 merge operation nodes and add a second deformation node after each merge operation node;

[0024] Aggregating the N matrix multiplication nodes to obtain an aggregated matrix multiplication node;

[0025] Adjusting the position of the aggregation matrix multiplication node to after the second deformation node, and adding the first deformation node after the aggregation matrix multiplication node to obtain a second optimized data flow graph;

[0026] In which, in the second optimized data flow graph, the input tensor of each merge operation node is an input tensor of the same type among the M types of input tensors, the output tensor of the second deformation node is the input tensor of the aggregation matrix multiplication node, and the output tensor of the aggregation matrix multiplication node is the input tensor of the first deformation node.

[0027] Optionally, the second optimized data flow graph includes a first merge operation node and a second merge operation node;

[0028] The configuration unit is further configured to configure a data processing model according to the second optimized data flow graph; and obtain N groups of input tensor combinations, and input the N groups of input tensor combinations into the data processing model; each group of input tensor combinations includes a pre-input tensor and a post-input tensor to be subjected to a matrix multiplication operation;

[0029] The model running unit is also used to perform a merge operation on the leading input tensors in the N groups of tensor combinations through the first merge operation node in the data processing model to obtain a first merged tensor; and to perform a merge operation on the trailing input tensors in the N groups of tensor combinations through the second merge operation node to obtain a second merged tensor; respectively transform the shapes of the corresponding first merged tensor and the second merged tensor through each second deformation node to obtain a first deformed tensor and a second deformed tensor respectively; perform a matrix multiplication operation on the first deformed tensor and the second deformed tensor through the aggregate matrix multiplication node to obtain a first output tensor; and transform the shape of the first output tensor through the first deformation node to obtain a target output tensor.

[0030] Optionally, the flow graph optimization unit is further configured to add a split operation node after the first deformation node if it is found that the output tensor of the matrix multiplication node is the input tensor of other operation nodes in the original data flow graph; wherein the other operation nodes are operation nodes other than the merge operation node;

[0031] The model running unit is also used to perform a splitting operation on the target output tensor through the splitting operation node to obtain N splitting tensors corresponding to the N matrix multiplication nodes in the original data flow graph; wherein, one splitting tensor corresponds to one matrix multiplication node in the original data flow graph, and in the third optimized data flow graph, the N splitting tensors are the input tensors of the other operation nodes.

[0032] Optionally, the polymerization conditions include:

[0033] Among the M types of input tensors, the N input tensors of the N matrix multiplication nodes corresponding to any type have the same size.

[0034] Optionally, the merge operation node is a merge operation node that performs a merge operation on a column basis;

[0035] The detection unit is further used to detect whether the matrix multiplication operation operator in the operator library supports the intermediate dimension as the stacking dimension;

[0036] The flow graph optimization unit is further configured to, if the matrix multiplication operator does not support the intermediate dimension as the stacking dimension, replace the merge operation node with the M stacking operation nodes, and aggregate the N matrix multiplication nodes to obtain an aggregated matrix multiplication node; swap the positions of the aggregated matrix multiplication node and the M stacking operation nodes, and add a transpose operation node and the first deformation node after the aggregated matrix multiplication node to obtain a third optimized data flow graph;

[0037] In the third optimized data flow graph, the output tensors of the M stacking operation nodes are the input tensors of the aggregate matrix multiplication node, the output tensor of the aggregate matrix multiplication node is the input tensor of the transposition operation node, and the output tensor of the transposition operation node is the input tensor of the first deformation node.

[0038] Optionally, the M stacking operation nodes include a first stacking operation node and a second stacking operation node;

[0039] The configuration unit is further configured to configure a data processing model according to the third optimized data flow graph; and obtain N groups of input tensor combinations, and input the N groups of input tensor combinations into the data processing model; each group of input tensor combinations includes a pre-input tensor and a post-input tensor to be subjected to a matrix multiplication operation;

[0040] The model running unit is also used to perform a stacking operation on the leading input tensor in the N groups of tensor combinations through the first stacking operation node in the data processing model to obtain a first stacked tensor; and to perform a stacking operation on the trailing input tensor in the N groups of tensor combinations through the second stacking operation node to obtain a second stacked tensor; to perform a matrix multiplication operation on the first stacked tensor and the second stacked tensor through the aggregate matrix multiplication node to obtain a first output tensor; to perform a transposition operation on the first output tensor through the transposition operation node to obtain a transposed tensor; and to transform the shape of the transposed tensor through the first deformation node to obtain a target output tensor.

[0041] In one aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above methods when executing the computer program.

[0042] In one aspect, a computer storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the steps of any of the above methods are implemented.

[0043] In one aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the above methods.

[0044] In an embodiment of the present application, when it is detected that there is a merge operation node and the merge operation node is previously connected to multiple matrix multiplication nodes that can be aggregated, the matrix multiplication nodes can be aggregated to obtain a new aggregated matrix multiplication node. At the same time, in order to ensure the consistency of the final output result with the original result, the merge operation node is updated to multiple stacking operation nodes, that is, multiple input tensors of the same type of each matrix multiplication are processed into one output tensor through a stacking operation node, and finally processed through a matrix multiplication node, thereby realizing that the matrix multiplication operations of multiple small tensors can be aggregated into a matrix multiplication operation of a large tensor, greatly reducing the number of matrix multiplication nodes, thereby reducing the resources allocated to the matrix multiplication nodes. At the same time, the related scheme needs to schedule N resources for N matrix multiplication nodes, while the scheme of the embodiment of the present application merges N schedules into one schedule, reducing the scheduling overhead and enabling resources to be fully utilized within the same time period, thereby improving resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0046] Figure 1 Schematic diagram of application scenarios provided by embodiments of the present application;

[0047] Figure 2 A flow chart of a data flow graph processing method provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of the original data flow diagram provided in the embodiment of the present application;

[0049] Figure 4 A schematic diagram of a first optimized data flow graph provided in an embodiment of the present application;

[0050] Figure 5 Another schematic diagram of the original data flow diagram provided in the embodiment of the present application;

[0051] Figure 6Another schematic diagram of the first optimized data flow graph provided in an embodiment of the present application;

[0052] Figure 7 Another schematic diagram of the original data flow diagram provided in the embodiment of the present application;

[0053] Figure 8 Another schematic diagram of the first optimized data flow graph provided in an embodiment of the present application;

[0054] Figure 9 Another flow chart of the data flow graph processing method provided in the embodiment of the present application;

[0055] Figure 10 A schematic diagram of a third optimized data flow graph provided in an embodiment of the present application;

[0056] Figure 11 Another schematic diagram of the third optimized data flow graph provided in an embodiment of the present application;

[0057] Figure 12 A schematic diagram of another flow chart of the data flow graph processing method provided in an embodiment of the present application;

[0058] Figure 13 A schematic diagram of a second optimized data flow graph provided in an embodiment of the present application;

[0059] Figure 14 Another schematic diagram of the second optimized data flow graph provided in an embodiment of the present application;

[0060] Figure 15 Another schematic diagram of the second optimized data flow graph provided in an embodiment of the present application;

[0061] Figure 16 Another schematic diagram of the second optimized data flow graph provided in an embodiment of the present application;

[0062] Figure 17 A schematic diagram of an optimized data flow graph for adding a split operation node provided in an embodiment of the present application;

[0063] Figure 18 A structural diagram of a data flow graph processing method provided in an embodiment of the present application;

[0064] Figure 19 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other in any way. In addition, although a logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0066] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0067] Tensors: Tensors are tools used in linear algebra to represent the real physical world through the virtual digital world. In linear algebra, a point corresponds to a scalar, a 0-dimensional tensor. Therefore, it has 0 axes and a rank of 0. A line corresponds to a vector, a 1-dimensional tensor (often called an array in computer science). A vector consists of a sequence of numbers, has 1 axis, and a rank of 1. A face corresponds to a matrix, a rank-2 tensor with two axes. A matrix is ​​a 2-dimensional tensor that arranges numbers into a grid (corresponding to rows and columns). A volume corresponds to a tensor, a container that can hold N-dimensional data and its linear operations. Strictly speaking, all of the above structures are valid tensors, but when discussing tensors, the concept of a matrix is ​​generally generalized to N ≥ 3 dimensions, so tensors of 3 or higher dimensions are often referred to as tensors.

[0068] Split operation: A commonly used tensor operation that splits a large tensor into multiple small tensors.

[0069] The concat operation is a common tensor operation that combines multiple small tensors into a single large tensor along a certain dimension. For example, the tensor C obtained by concatenating two tensors A and B is shown below. For convenience, we use the two-dimensional array concatenation operation as an example:

[0070]

[0071]

[0072]

[0073] Matrix multiplication (Matmul) is a commonly used tensor operation that multiplies each element in the i-th row of the first tensor with each element in the k-th column of the second tensor, and adds them together. The result is recorded as the i-th row and j-th column of the output tensor. For example, the tensor C obtained by matrix multiplication of two tensors A and B is as follows:

[0074]

[0075]

[0076]

[0077] Stack operation: creates a new dimension when merging tensors.

[0078] Reshape operation: A commonly used tensor operation that transforms the shape of a tensor in a specified way.

[0079] The following is a brief introduction to the technologies and design concepts involved in the embodiments of this application.

[0080] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0081] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and smart transportation.

[0082] Computer vision (CV) is the science of making machines "see." Specifically, it refers to using cameras and computers to replace the human eye in identifying, tracking, and measuring objects. It then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, autonomous driving, and smart transportation. It also includes common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0083] Natural language processing (NLP) is a key area of ​​research in computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. Natural language processing (NLP) integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language we use in everyday life—and is closely linked to the study of linguistics. Natural language processing technologies typically include text processing, semantic understanding, machine translation, robotic question answering, and knowledge graphs.

[0084] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0085] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, Internet of Vehicles, automatic driving, smart transportation, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0086] The solution provided in the embodiments of the present application involves technologies such as deep learning of artificial intelligence, and can be specifically used in tensor graph optimization engines, such as graph optimization engines in deep learning, such as the Grappler computational graph optimization engine in Tensorflow (a system that transfers complex data structures to artificial intelligence neural networks for analysis and processing). Specifically, before performing deep learning, the data flow graph used for deep learning can be pre-written, and then when the data flow graph is parsed, the method of the embodiment of the present application can be used to optimize the data flow graph, generate a new data flow graph, and perform a deep learning process based on the new data flow graph to improve the resource utilization of deep learning, thereby improving the speed and efficiency of deep learning.

[0087] In related technologies, when optimizing data flow graphs, only element-by-element operations (such as the Sqrt square root operation) can be optimized. That is, after performing multiple element-by-element operations of the same type, when merging the small tensors output by the element-by-element operation nodes into a large tensor, the small tensors input to the element-by-element operation nodes can be first merged into a large tensor before performing the merge operation. However, this method can only support element-by-element operations, and due to the particularity of matrix multiplication, it cannot be applied to matrix multiplication. However, matrix multiplication is an operation that consumes relatively many resources, so it is necessary to find a feasible solution to support the optimization of matrix multiplication.

[0088] Based on this, an embodiment of the present application provides a data flow graph processing method, in which when it is detected that there is a merge operation node and the merge operation node is previously connected to multiple matrix multiplication nodes that can be aggregated, the matrix multiplication node can be aggregated to obtain a new aggregated matrix multiplication node. At the same time, in order to ensure the consistency of the final output result with the original result, the merge operation node is updated to multiple stacking operation nodes, that is, multiple input tensors of the same type of each matrix multiplication are processed into an output tensor through a stacking operation node, and finally processed through a matrix multiplication node, thereby realizing that the matrix multiplication operations of multiple small tensors can be aggregated into a matrix multiplication operation of a large tensor, greatly reducing the number of matrix multiplication nodes, thereby reducing the resources allocated to the matrix multiplication nodes. At the same time, the related scheme needs to schedule N resources for N matrix multiplication nodes, while the scheme of the embodiment of the present application merges N schedules into one schedule, reducing the scheduling overhead and enabling resources to be fully utilized within the same time period, thereby improving resource utilization.

[0089] The embodiment of the present application also takes into account that different operator libraries support different tensor operation operators. Therefore, when the operator library does not support the stacking operation operator, the embodiment of the present application also provides another implementation solution, namely, combining the merge operation and the deformation operation to replace the function of the stacking operation operator to achieve aggregation optimization of the matrix multiplication nodes.

[0090] In addition, the embodiment of the present application also takes into account that the output tensor of the matrix multiplication node may also be used as the input tensor of other operation nodes. Therefore, in order to enable other operation nodes to input normally, the final target output tensor can be split to obtain the same output tensor as the matrix multiplication node of the original data flow graph, which is used as the input tensor of other operation nodes.

[0091] The embodiments of the present application also take into account that the merge operation may include two types: row-based merging and column-based merging. However, column-based merging is more special because the intermediate dimension after column-based merging is the Batch dimension. However, it is possible that the matrix multiplication operators of some operator libraries do not support the intermediate dimension as the Batch dimension. Based on these considerations, the embodiments of the present application target the situation where the matrix multiplication operators of the operator library do not support the intermediate dimension as the Batch dimension. By merging by rows, performing a transpose operation after completing the matrix multiplication operation, and then adjusting the batch dimension to the intermediate dimension, the same result as the original data flow graph can be obtained.

[0092] After introducing the design concepts of the embodiments of the present application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limiting. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0093] The solution provided in the embodiments of the present application can be applied to most tensor data flow analysis and optimization scenarios, such as optimizing neural network model scenarios.

[0094] like Figure 1 As shown, an application scenario diagram provided by an embodiment of the present application may include a terminal device 101, a graph optimization server 102 and a computing node cluster. The computing node cluster may include multiple computing nodes, such as Figure 1 As shown, it may specifically include computing node 103a, computing node 103b, computing node 103c, ..., computing node 103n. Figure 1 As shown, the terminal device 101 can be connected to the graph optimization server 102 and each computing node of the computing node cluster through the network 104.

[0095] The terminal device 101 may be, for example, a mobile phone, a tablet computer (PAD), a laptop computer, a desktop computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, and an intelligent wearable device. The terminal device 101 may be installed with an application capable of editing or uploading a data flow graph, such as a browser or a data flow graph client.

[0096] The graph optimization server 102 may be a backend server corresponding to an application installed on the terminal device 101. The graph optimization server 102 may be, for example, an independent physical server, or a server cluster or distributed system composed of multiple physical servers. The graph optimization server 102 and the computing node cluster may also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms, but are not limited thereto.

[0097] The graph optimization server 102 may include one or more processors 1021, a memory 1022, and an I / O interface 1023 for interacting with a terminal. Furthermore, the graph optimization server 102 may be configured with a database 1024, which may be used to store information related to the original data flow graph and the optimized data flow graph. The memory 1022 of the graph optimization server 102 may also store program instructions for the data flow graph processing method provided in the embodiment of the present application. When these program instructions are executed by the processor 1021, they may be used to implement the steps of the data flow graph processing method provided in the embodiment of the present application to obtain an optimized data flow graph.

[0098] In a specific implementation, the user can edit the data flow graph on the application installed on the terminal device 101 to generate the original data flow graph, or can also upload the edited original data flow graph. The original data flow graph can be the data flow graph corresponding to the constructed deep learning model. Accordingly, after the graph optimization server 102 receives the original data flow graph, it can optimize the original data flow graph using the method of the embodiment of the present application to obtain an optimized data flow graph, and can call the computing resources of each computing node 103 of the computing node cluster based on the obtained optimized data flow graph to execute the data processing function of the deep learning model.

[0099] In an embodiment of the present application, the solution of the embodiment of the present application can be integrated into an AI platform or AI framework. For example, it can be applied to a tensor graph optimization engine, such as a graph optimization engine in deep learning, such as the Grappler computational graph optimization engine in Tensorflow.

[0100] Optionally, it is understood that the graph optimization server 102 may be a stream computing platform, such as the aforementioned neural network model, tensor graph optimization engine, etc. A user uploads the original data flow graph to the stream computing platform, which can use the optimization method of this solution as an optimization method to optimize the original data flow graph, obtain an optimized data flow graph, and deploy the optimized data flow graph to various frameworks for application.

[0101] The above-mentioned network 104 can be a wired network or a wireless network. For example, the wireless network can be a mobile cellular network, or a Wireless-Fidelity (WIFI) network. Of course, it can also be other possible networks, and the embodiments of the present application do not limit this.

[0102] Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing.

[0103] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form a resource pool for on-demand, flexible and convenient use. Cloud computing technology will become a key support. Backend services of technical network systems, such as video websites, image websites, and more portals, require extensive computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identification mark, which will need to be transmitted to backend systems for logical processing. Different levels of data will be processed separately. All types of industry data require strong system support, which can only be achieved through cloud computing.

[0104] Optionally, it is understood that the backend server may directly detect the original data flow graph collected on the respective corresponding terminal devices and optimize the original data flow graph to obtain an optimized data flow graph. The specific implementation of the backend server optimizing the original data flow graph to obtain the optimized data flow graph can be found in the description of the graph optimization server 102 optimizing the original data flow graph to obtain the optimized data flow graph, and will not be repeated here.

[0105] Of course, the method provided in the embodiment of the present application is not limited to Figure 1 The application scenarios shown can also be used in other possible application scenarios, and the embodiments of this application are not limited thereto. Figure 1 The functions that can be implemented by each device in the application scenario shown will be described in subsequent method embodiments and will not be described in detail here.

[0106] See Figure 2 , is a flow chart of a data flow graph processing method provided in an embodiment of the present application, which can be processed by Figure 1 The graph optimization server 102 or the terminal device 101 is used to execute the method. Here, the graph optimization server 102 is used as an example for the introduction. The process of the method is described as follows.

[0107] Step 201: Obtain the original data flow graph and parse the original data flow graph.

[0108] In the embodiment of the present application, the original data flow graph may be a flow graph compiled by a user, and the original data flow graph may include the following elements:

[0109] (1) Data processing node

[0110] A data processing node can be a unit that calls the various data processing operators included in the operator library to process data. It can be used to process input data. A data processing node can receive certain data input, process the data according to its corresponding processing function, and generate data output.

[0111] (2) Input data and output data.

[0112] The input data may be initially input data, and the output data may be finally output data.

[0113] (3) Data Flow

[0114] Data flow refers to data in motion, with a specific direction. It is the path along which data propagates within a system. Data flow can be used to represent the flow of data from one node to another. For example, if the output data processed by data processing node A is input to data processing node B, then this data flowing to data processing node B (the output data of data processing node A) can be considered a data flow.

[0115] Each data processing node has corresponding input data and output data. Generally speaking, the corresponding input variables and output variables are defined for each data processing node. For example, for data processing node A, its input number is variable 1 and its output data is variable 2. Of course, variable 1 and variable 2 can also be the input variables or output variables of other data processing nodes.

[0116] See Figure 3 , is a schematic diagram of an original data flow diagram provided in an embodiment of the present application. Figure 3 As shown, the data processing nodes of the original data flow graph include N matrix multiplication nodes and merge operation nodes. The N matrix multiplication nodes may include matrix multiplication node 1 to matrix multiplication node n, which are used to perform matrix multiplication operations on their corresponding input data; the merge operation node is used to merge the data output by matrix multiplication nodes 1 to matrix multiplication nodes n to obtain the final output result. Wherein, N is a positive integer greater than 1.

[0117] The original input data may be M types of input data corresponding to each matrix multiplication node, where M is a positive integer greater than 1.

[0118] Generally speaking, M can be 2, that is, the matrix multiplication node performs a matrix multiplication operation on two input data, and the two input data can be the front input and the back input respectively. Figure 3 This is specifically shown. Figure 3 As shown, the input data of matrix multiplication node 1 is pre-input 1 and post-input 1. Matrix multiplication node 1 performs a matrix multiplication operation on pre-input 1 and post-input 1 to obtain output result 1. The input data of matrix multiplication node 2 is pre-input 2 and post-input 2. Matrix multiplication node 2 performs a matrix multiplication operation on pre-input 2 and post-input 2 to obtain output result 2, and so on. Furthermore, output results 1 to n outputted by each matrix multiplication node flow as a data stream to the merge operation node. The merge operation node performs a merge operation on output results 1 to n to obtain the final output result.

[0119] Among them, when the matrix multiplication operation is to process two input data, the pre-input can generally be the first item in the matrix multiplication operation, that is, Figure 3 The parameter 1 shown is generally the second term in the matrix multiplication operation, that is, Figure 3 Parameter 2 shown.

[0120] In an embodiment of the present application, if the original data flow graph of the user is received, the original data flow graph can be parsed to obtain the data processing nodes and data flows included in the original data flow graph.

[0121] Step 202: If the original data flow graph is parsed to include a merge operation node, and the input tensor of the merge operation node is the output tensor of N matrix multiplication nodes, then check whether the N matrix multiplication nodes meet the aggregation condition.

[0122] In an embodiment of the present application, after parsing the original data flow graph, the data processing nodes and data flows included therein can be obtained. If it is determined that a merge operation node is included therein, and the input tensor of the merge operation node is the output tensor of multiple matrix multiplication nodes, it is further determined whether these N matrix multiplication nodes can be aggregated.

[0123] Specifically, for the original data flow graph, the attributes of each data flow are defined, that is, which data processing node the data flow is generated by and to which data processing node it flows. Therefore, after determining that there is a merge operation node, it is possible to find which data flow the input tensor of the merge operation node is, and further determine who produced the data flow. When it is determined based on the attributes of the data flow that the data flow is produced by multiple matrix multiplication nodes, it can be further determined whether these N matrix multiplication nodes can be aggregated.

[0124] In the embodiment of the present application, since multiple matrix multiplications can only be aggregated when the input tensors of the matrix multiplications have the same size, otherwise the final result will be inconsistent with the result in the original data flow graph. Therefore, the aggregation conditions of the embodiment of the present application can be specifically as follows:

[0125] Among the M types of input tensors corresponding to N matrix multiplication nodes, the N input tensors corresponding to any type of matrix multiplication nodes have the same size. Here, M and N are both positive integers greater than 1.

[0126] Generally speaking, each matrix multiplication node can include 2 input tensors, namely the pre-tensor and the post-tensor, so the aggregation condition can be understood as the size of all pre-tensors of N matrix multiplication nodes is the same, and the size of all post-tensors is the same.

[0127] Step 203: If the aggregation condition is met, the N matrix multiplication nodes are aggregated to obtain an aggregated matrix multiplication node, and the merge operation node is replaced with M stacking operation nodes; wherein the input tensor of each stacking operation node is: an input tensor of the same type among the M types of input tensors.

[0128] In an embodiment of the present application, if N matrix multiplication nodes meet the aggregation conditions, in order to reduce the number of nodes of the same type, the N matrix multiplication nodes can be aggregated into one data processing node, that is, an aggregated matrix multiplication node. The aggregated matrix multiplication node after aggregation is of the same type as the original matrix multiplication node, and the operation performed is also the same, that is, matrix multiplication operation is still performed.

[0129] At the same time, since the structure of the original data flow graph is changed after aggregation, the final output cannot be consistent with the original data flow graph. Figure 1 Therefore, in order to achieve the same Figure 1 To achieve the desired effect, the embodiment of the present application replaces the merge operation node with M stacking operation nodes.

[0130] The value of M is related to the number of input tensors of the matrix multiplication node. Generally speaking, the input tensors of the matrix multiplication node include pre-tensors and post-tensors, so the value of M can be 2.

[0131] Step 204: swap the positions of the aggregate matrix multiplication node and the M stacking operation nodes, and add a first deformation node after the aggregate matrix multiplication node to obtain a first optimized data flow graph.

[0132] Similarly, in order to achieve the same Figure 1 In order to achieve the same data processing effect, it is also necessary to swap the positions of the aggregation matrix multiplication node and the M stacking operation nodes, and add a first deformation node after the aggregation matrix multiplication node to obtain a first optimized data flow graph.

[0133] See Figure 4 FIG2 is a schematic diagram of a first optimized data flow graph provided by an embodiment of the present application. As can be seen, in the first optimized data flow graph, the input tensors of the stacking operation nodes are the input tensors of each matrix multiplication node in the original data flow graph, and each stacking operation node is used for input tensors of the same type. The output tensor of the stacking operation node is the input tensor of the aggregate matrix multiplication node, and the output tensor of the aggregate matrix multiplication node is the input tensor of the first deformation node.

[0134] In an embodiment of the present application, after obtaining the first optimized data flow graph, a data processing model can be configured according to the first optimized data flow graph, and the data processing model can be used to perform data processing on the N groups of input tensor combinations, wherein each group of input tensor combinations includes a front input tensor and a rear input tensor to be subjected to matrix multiplication operations.

[0135] For example Figure 4 As shown, the input tensor of the first stacking operation node is the N pre-input tensors of N matrix multiplication nodes. After the first stacking operation node performs stacking processing, a first stacked tensor is obtained, which serves as the pre-input tensor of the aggregate matrix multiplication node and flows to the aggregate matrix multiplication node. Similarly, the input tensor of the second stacking operation node is the N post-input tensors of N matrix multiplication nodes. After the second stacking operation node performs stacking processing, a second stacked tensor is obtained, which serves as the post-input tensor of the aggregate matrix multiplication node and flows to the aggregate matrix multiplication node. It can be seen that after processing by the stacking operation node, multiple small tensors are stacked to obtain a large tensor, which is then input to the aggregate matrix multiplication node. This can greatly reduce the number of matrix multiplication nodes. This effect is particularly obvious when the number of matrix multiplication nodes in the original data flow graph is large. In this way, the N-degree resource scheduling that should be performed on the N-time matrix multiplication nodes can be greatly reduced after optimization, thereby reducing scheduling overhead. Moreover, in the scenario of serial scheduling, the original scheduling method cannot make full use of resources. For example, the original processing resources have 100% processing power, but the resource scheduling of a small tensor matrix multiplication node can only consume 20% of the processing power. Then the remaining 80% of the processing power is idle, and the resources cannot be fully utilized. After aggregation, a large tensor matrix multiplication node needs to consume more processing power, so that resources can be better utilized each time the resources are scheduled, thereby improving resource utilization.

[0136] After the first stacking operation node and the second stacking tensor are processed by the aggregate matrix multiplication node, the generated first output tensor flows as a data stream to the first deformation node. After the first deformation node transforms the shape of the first output tensor, the target output tensor is obtained. The target output tensor is the same as the output result of the merge operation node in the original data flow graph.

[0137] In the embodiment of the present application, the merging operation node can generally include two processing methods: merging by row and merging by column. The results obtained are different for different processing methods, which are introduced below.

[0138] (1) Merge by row

[0139] For example, see Figure 5 The figure shows the original data flow diagram when N and M are both 2. The input data of matrix multiplication node 1 includes the pre-placed small tensor input 1[a, B] and the post-placed small tensor input 1[B, C]. After being processed by matrix multiplication node 1, the small tensor output result 1[a, C] is obtained. Similarly, the input data of matrix multiplication node 2 includes the pre-placed small tensor input 2[a, B] and the post-placed small tensor input 2[B, C]. After being processed by matrix multiplication node 2, the small tensor output result 2[a, C] is obtained. The small tensor output results 1[a, C] and 2[a, C] flow as data streams to the merge operation node, which merges the small tensor output results 1[a, C] and 2[a, C] to obtain the final large tensor output result [A, C].

[0140] Among them, [a, B] represents a tensor with a rows and B columns, A represents the number of rows after merging two small tensors by row, and A=2a.

[0141] See also Figure 6 As shown, for Figure 5The first optimized data flow graph after the original data flow graph is optimized is shown in the figure. In this optimized data flow graph, the input tensors of the first stacking operation node are the front small tensor input 1[a, B] and the front small tensor input 2[a, B]. After processing by the first stacking operation node, the output large tensor is obtained, namely the first stacking tensor [2, a, B]. The first stacking operation node here stacks rows. Similarly, the input tensors of the second stacking operation node are the rear input 1[B, C] and the rear input 2[B, C]. After processing by the second stacking operation node, the output large tensor is obtained, namely the second stacking tensor [2, B, C]. The first stacking tensor [2, a, B] and the second stacking tensor [2, B, C] serve as the input tensors of the aggregation matrix multiplication node. After processing by the aggregation matrix multiplication node, the first output tensor [2, a, C] is obtained. The shape of the first output tensor [2, a, C] is transformed by the first deformation node to obtain the target output tensor [A, C].

[0142] In an embodiment of the present application, the processing of the first deformation node can specifically be to flatten and stack the first output tensor [2, a, C], that is, to obtain the result [A, C] of the merge operation node output in the original data flow graph.

[0143] (2) Merge by column

[0144] For example, see Figure 7 The figure shows the original data flow diagram when N and M are both 2. The input data of matrix multiplication node 1 includes the pre-placed small tensor input 1 [A, B] and the post-placed small tensor input 1 [B, c]. After being processed by matrix multiplication node 1, the small tensor output result 1 [A, c] is obtained. Similarly, the input data of matrix multiplication node 2 includes the pre-placed small tensor input 2 [A, B] and the post-placed small tensor input 2 [B, c]. After being processed by matrix multiplication node 2, the small tensor output result 2 [A, c] is obtained. The small tensor output results 1 [A, c] and 2 [A, c] flow as data streams to the merge operation node, which merges the small tensor output results 1 [A, c] and 2 [A, c] to obtain the final large tensor output result [A, C], where C = 2c.

[0145] In the embodiment of the present application, when the merge operation node is merged by row, see Figure 6 As shown in the figure, the stacking (batch) dimension of [2, B, C] (that is, the dimension indicated by "2") is not located in the middle dimension. General matrix multiplication operations can process such tensors, but when the merge operation node is merged by column, the batch dimension of the resulting tensor is located in the middle dimension. In this case, it is necessary to consider whether the matrix multiplication operator of the current operator library can support the situation where the middle dimension is the batch dimension.

[0146] In one possible implementation, when the matrix multiplication operator of the current operator library supports the batch dimension as the intermediate dimension, see Figure 8 As shown, for Figure 7 The first optimized data flow graph after the original data flow graph is optimized is shown in the figure. In this optimized data flow graph, the input tensors of the first stacking operation node are the front small tensor input 1[A, B] and the front small tensor input 2[A, B]. After being processed by the first stacking operation node, a large output tensor is obtained, namely the first stacked tensor [A, 2, B]. Here, the first stacking operation node is stacked by column, and it can be seen that the intermediate dimension of the obtained first stacked tensor [A, 2, B] is the batch dimension. Similarly, the input tensors of the second stacking operation node are the rear input 1[B, c] and the rear input 2[B, c]. After being processed by the second stacking operation node, a large output tensor is obtained, namely the second stacked tensor [B, 2, c]. The intermediate dimension of the obtained second stacked tensor [B, 2, c] is also the batch dimension. The first stacked tensor [A, 2, B] and the second stacked tensor [B, 2, c] serve as input tensors of the aggregate matrix multiplication node. After being processed by the aggregate matrix multiplication node, the first output tensor [A, 2, c] is obtained, and the shape of the first output tensor [A, 2, c] is transformed through the first deformation node to obtain the target output tensor [A, C].

[0147] In an embodiment of the present application, the processing of the first deformation node can specifically be to flatten and stack the first output tensor [A, 2, c], that is, to obtain the result [A, C] of the merge operation node output in the original data flow graph.

[0148] In another possible implementation, if the matrix multiplication operator in the current operator library does not support the batch dimension as the intermediate dimension, the merge operation needs to be performed row by row, and a transpose operation is performed after the matrix multiplication operation is completed to adjust the batch dimension to the intermediate dimension. Figure 9 As shown, it is a flowchart of the data flow graph processing method when the matrix multiplication operator does not support the batch dimension as the intermediate dimension.

[0149] Step 901: Obtain the original data flow graph and parse the original data flow graph.

[0150] Step 902: If the original data flow graph is parsed to include a merge operation node, and the input tensor of the merge operation node is the output tensor of N matrix multiplication nodes, then check whether the N matrix multiplication nodes meet the aggregation conditions.

[0151] Step 903: If the aggregation condition is met, it is detected whether the merge operation node is a merge operation node that performs a column-based merge operation.

[0152] Step 904: If it is a merge operation node that performs a merge operation on a column basis, it is detected whether the matrix multiplication operator in the operator library supports the intermediate dimension being the batch dimension.

[0153] Step 905: If the intermediate dimension is not supported as the batch dimension, the merge operation node is replaced with M stacking operation nodes, and the N matrix multiplication nodes are aggregated to obtain an aggregated matrix multiplication node.

[0154] Step 906: swap the positions of the aggregate matrix multiplication node and the M stacking operation nodes, and add a transpose operation node and a first deformation node after the aggregate matrix multiplication node to obtain a third optimized data flow graph.

[0155] Specifically, if the matrix multiplication operator does not support the batch dimension as the intermediate dimension, then aggregate N matrix multiplication nodes to obtain an aggregate matrix multiplication node, replace the merge operation node with M stacking operation nodes, swap the positions of the aggregate matrix multiplication node and the M stacking operation nodes, and add a transpose operation node and the first deformation node after the aggregate matrix multiplication node to obtain the third optimized data flow graph. Figure 10 As shown in FIG, it is a schematic diagram of the third optimized data flow graph when the intermediate dimension is not supported as the batch dimension. Among them, M( Figure 10 Specifically shown as M being 2), the output tensor of the stacking operation nodes is the input tensor of the aggregate matrix multiplication node, the output tensor of the aggregate matrix multiplication node is the input tensor of the transpose operation node, and the output tensor of the transpose operation node is the input tensor of the first deformation node.

[0156] In an embodiment of the present application, after obtaining the third optimized data flow graph, a data processing model can be configured according to the third optimized data flow graph, and the data processing model can be used to perform data processing on the N groups of input tensor combinations, wherein each group of input tensor combinations includes a front input tensor and a rear input tensor to be subjected to matrix multiplication operations.

[0157] For example Figure 10As shown, the M stacking operation nodes include a first stacking operation node and a second stacking operation node. The input tensors of the first stacking operation node are the N pre-input tensors of the N matrix multiplication nodes. After the first stacking operation node performs stacking processing, a first stacking tensor is obtained, which serves as the pre-input tensor of the aggregate matrix multiplication node and flows to the aggregate matrix multiplication node. Similarly, the input tensors of the second stacking operation node are the N post-input tensors of the N matrix multiplication nodes. After the second stacking operation node performs stacking processing, a second stacking tensor is obtained, which serves as the post-input tensor of the aggregate matrix multiplication node and flows to the aggregate matrix multiplication node. After the first stacking operation node and the second stacking tensor are processed by the aggregate matrix multiplication node, the generated first output tensor flows as a data flow to the transposition operation node. After the transposition operation node performs a transposition operation, the generated transposed tensor flows as a data flow to the first deformation node. After the first deformation node transforms the shape of the first output tensor, a target output tensor is obtained. The target output tensor is the same output result as the merge operation node in the original data flow graph.

[0158] It should be noted that the first stacking operation node and the second stacking operation node here perform stacking operations on a row basis.

[0159] For example, Figure 7 Taking the original data flow shown as an example, the third optimized data flow diagram after optimization is shown in Figure 11. In this optimized data flow diagram, the input tensors of the first stacking operation node are the front small tensor input 1[A, B] and the front small tensor input 2[A, B]. After the first stacking operation node performs stacking processing by row, the output large tensor is obtained, that is, the first stacking tensor [2, A, B]. It can be seen that the intermediate dimension of the obtained first stacking tensor [2, A, B] is a non-batch dimension; similarly, the input tensors of the second stacking operation node are the rear input 1[B, c] and the rear input 2[B, c]. After the second stacking operation node performs stacking processing by row, the output large tensor is obtained, that is, the second stacking tensor [2, B, c]. The first stacked tensor [2, A, B] and the second stacked tensor [2, B, c] are used as input tensors of the aggregate matrix multiplication node. After being processed by the aggregate matrix multiplication node, the first output tensor [2, A, c] is obtained. Then, the first output tensor [2, A, c] is transposed by the transpose operation node to obtain the transposed tensor [A, 2, c]. The shape of the first output tensor [A, 2, c] is flattened by the stacking dimension through the first deformation node to obtain the target output tensor [A, C].

[0160] As you can see, Figure 11 The transposed tensor obtained after the transposition operation is the same as Figure 8The first output tensor obtained by the matrix multiplication node is the same, that is, the merge operation node is a column-by-column merge, and its matrix multiplication operator does not support the optimized operation when the intermediate dimension is the batch dimension.

[0161] In the embodiment of the present application, the operator library provides the data processing operators corresponding to each node. However, depending on the different operator libraries actually used, some operator libraries may support stacking operation operators, while there may also be cases where the operator library does not support stacking operation operators. For cases where the operator library supports stacking operation operators, the optimized data flow graph containing stacking operation nodes can be used for processing. For cases where the operator library does not support stacking operation operators, see Figure 12 As shown, it is a flowchart of the data flow graph processing method when the matrix multiplication operator does not support the stacking operation operator.

[0162] Step 1201: Obtain the original data flow graph and parse the original data flow graph.

[0163] Step 1202: If the original data flow graph is parsed to include a merge operation node, and the input tensor of the merge operation node is the output tensor of N matrix multiplication nodes, then check whether the N matrix multiplication nodes meet the aggregation conditions.

[0164] Step 1203: If the aggregation condition is met, check whether the operator library supports the stacking operation operator.

[0165] Step 1204: If the stacking operation operator is not supported, add M-1 of the merge operation nodes, add a second deformation node after each of the merge operation nodes, and aggregate the N matrix multiplication nodes to obtain an aggregated matrix multiplication node.

[0166] Step 1205: Adjust the position of the aggregation matrix multiplication node to after the second deformation node, and add the first deformation node after the aggregation matrix multiplication node to obtain a second optimized data flow graph. Figure 13 As shown, it is a schematic diagram of the second optimized data flow graph provided in an embodiment of the present application.

[0167] Specifically, the stacking operation process can be implemented by a merge operation plus a deformation operation. Therefore, when the operator library does not support the stacking operation, M-1 merge operation nodes can be added, and a second deformation node can be added after each merge operation node. At the same time, N matrix multiplication nodes are aggregated to obtain an aggregate matrix multiplication node, and the position of the aggregate matrix multiplication node is adjusted to after each second deformation node, and a first deformation node is added after the aggregate matrix multiplication node, so as to obtain the following: Figure 13 The second optimized data flow graph is shown.

[0168] Among them, Figure 13 In the second optimized data flow diagram shown, the input tensors of each merge operation node are input tensors of the same type among the M types of input tensors. Generally speaking, when the matrix multiplication is a matrix multiplication operation on two input tensors (i.e., the front input tensor and the back input tensor), a merge operation node ( Figure 13 Specifically shown in this way), such as Figure 13 The first merge operation node and the second merge operation node are shown, so that the input tensors of the first merge operation node can be all the preceding input tensors, and the input tensors of the second merge operation node can be all the following input tensors.

[0169] The output tensor of the second deformation node is the input tensor of the aggregate matrix multiplication node, and the output tensor of the aggregate matrix multiplication node is the input tensor of the first deformation node.

[0170] In an embodiment of the present application, after obtaining the second optimized data flow graph, a data processing model can be configured according to the second optimized data flow graph, and the data processing model can be used to perform data processing on the N groups of input tensor combinations, wherein each group of input tensor combinations includes a front input tensor and a rear input tensor to be subjected to matrix multiplication operations.

[0171] For example Figure 13 As shown, the input tensor of the first merge operation node is the N pre-input tensors of the N matrix multiplication nodes. After the first merge operation node performs the merge processing, the first merged tensor is obtained, which flows to a second deformation node as a data flow, that is, Figure 13 The second deformation node 1 shown is transformed by the second deformation node 1, and the first deformation tensor is obtained after the shape transformation is performed; similarly, the input tensor of the second merging operation node is the N post-input tensors of the N matrix multiplication nodes. After the second merging operation node performs merging processing, the second merged tensor is obtained, which flows to another second deformation node as a data flow, that is, Figure 13 The second deformation node 2 shown is used, and after the shape transformation is performed by the second deformation node 2, a second deformation tensor is obtained. The first deformation tensor and the second deformation tensor flow as data streams to the aggregation matrix multiplication node. After being processed by the aggregation matrix multiplication node, the generated first output tensor flows as data stream to the first deformation node. After the shape of the first output tensor is transformed by the first deformation node, the target output tensor result is obtained. The target output tensor is the same as the output result of the merge operation node in the original data flow graph.

[0172] It should be noted that since the stacking operation is essentially equivalent to the combination of the merge operation and the second deformation operation, the first deformation tensor and the second deformation tensor obtained by processing the second deformation node are essentially the above-mentioned first stacking tensor and second stacking tensor, only the first merged tensor and the second merged tensor are added as temporary expressions. However, when the operator library does not support the stacking operation, the optimization scheme of the embodiment of the present application can still be implemented.

[0173] Specifically, the deformation processing of the second deformation node here may be a process of dividing the merged vector and superimposing the merged vector, thereby obtaining the same effect as the stacking operation.

[0174] Similarly, the following introduces two processing methods: merging by row and merging by column according to the merge operation node.

[0175] (1) Merge by row

[0176] See also Figure 14 As shown, Figure 5 Taking the original data flow graph shown as an example, a schematic diagram of the second optimized data flow graph after optimization is shown. In this optimized data flow graph, the input tensors of the first merge operation node are the front small tensor input 1[a, B] and the front small tensor input 2[a, B]. After being processed by the first merge operation node, the output large tensor is obtained, namely the first merged tensor [2a, B]. Here, the first merge operation node merges by row; similarly, the input tensors of the second merge operation node are the rear input 1[B, C] and the rear input 2[B, C]. After being processed by the second merge operation node, the output large tensor is obtained, namely the second merged tensor [2B, C]. The first merged tensor [2a, B] and the second merged tensor [2B, C] flow to the second deformation node 1 and the second deformation node 2 respectively. After the second deformation node 1 performs a shape transformation on the first merged tensor [2a, B], the first deformation tensor [2, a, B] is obtained. After the second deformation node 2 performs a shape transformation on the second merged tensor [2B, C], the second deformation tensor [2, B, C] is obtained. The first deformation tensor [2, a, B] and the second deformation tensor [2, B, C] serve as input tensors of the aggregation matrix multiplication node. After being processed by the aggregation matrix multiplication node, the first output tensor [2, a, C] is obtained. The shape of the first output tensor [2, a, C] is transformed by the first deformation node to obtain the target output tensor [A, C].

[0177] (2) Merge by column

[0178] When the merge operation node in the original data flow graph merges by row, it is necessary to consider whether the matrix multiplication operator of the current operator library supports the batch dimension as the intermediate dimension.

[0179] In one possible implementation, if the matrix multiplication operator of the current operator library supports the batch dimension as the intermediate dimension, then see Figure 15 As shown, for Figure 7 The second optimized data flow graph after the original data flow graph is optimized is shown in the figure. In this optimized data flow graph, the input tensors of the first merge operation node are the front small tensor input 1 [A, B] and the front small tensor input 2 [A, B]. After being processed by the first merge operation node, the output large tensor is obtained, namely the first merged tensor [A, 2B]. Here, the first merge operation node performs column-based merging. Similarly, the input tensors of the second merge operation node are the rear input 1 [B, c] and the rear input 2 [B, c]. After being processed by the second merge operation node, the output large tensor is obtained, namely the second merged tensor [B, C]. The first merged tensor [A, 2B] and the second merged tensor [B, C] flow to the second deformation node 1 and the second deformation node 2 respectively. After the second deformation node 1 performs shape transformation on the first merged tensor [A, 2B], the first deformation tensor [A, 2, B] is obtained. After the second deformation node 2 performs shape transformation on the second merged tensor [B, C], the second deformation tensor [B, 2, C] is obtained. The first deformation tensor [A, 2, B] and the second deformation tensor [B, 2, C] serve as input tensors of the aggregation matrix multiplication node. After being processed by the aggregation matrix multiplication node, the first output tensor [A, 2, C] is obtained. The shape of the first output tensor [A, 2, C] is transformed by the first deformation node to obtain the target output tensor [A, C].

[0180] In another possible implementation, if the matrix multiplication operator in the current operator library does not support the batch dimension as the intermediate dimension, see Figure 16 As shown, for Figure 7The optimized data flow graph after the original data flow graph is optimized is shown in the figure. In this optimized data flow graph, the input tensors of the first merge operation node are the front small tensor input 1[A, B] and the front small tensor input 2[A, B]. After being processed by the first merge operation node, the output large tensor is obtained, namely the first merged tensor [2A, B]. Here, the first merge operation node merges by row. Similarly, the input tensors of the second merge operation node are the rear input 1[B, c] and the rear input 2[B, c]. After being processed by the second merge operation node, the output large tensor is obtained, namely the second merged tensor [2B, c]. The first merged tensor [2A, B] and the second merged tensor [2B, c] flow to the second deformation node 1 and the second deformation node 2 respectively. After the second deformation node 1 performs a shape transformation on the first merged tensor [2A, B], the first deformed tensor [2, A, B] is obtained. After the second deformation node 2 performs a shape transformation on the second merged tensor [2B, c], the second deformed tensor [2, B, c] is obtained. The first deformed tensor [2, A, B] and the second deformed tensor [2, B, c] serve as input tensors of the aggregation matrix multiplication node. After being processed by the aggregation matrix multiplication node, the first output tensor [2, A, c] is obtained. After the first output tensor [2, A, c] is transposed by the transpose operation node, the transposed tensor [A, 2, c] is obtained. The shape of the transposed tensor [A, 2, c] is transformed by the first deformation node to obtain the target output tensor [A, C].

[0181] In the embodiment of the present application, in the original data flow graph, in addition to the merge operation node, the output tensor of the matrix multiplication node may also be used as the input tensor of other operation nodes, so see Figure 17 As shown, a split operation node can be added after the first deformation node, so that after the shape of the first output tensor is transformed by the first deformation node to obtain the target output tensor, the target output tensor can be split by the split operation node to obtain N split tensors corresponding to the N matrix multiplication nodes in the original data flow graph, as shown in FIG. Figure 17 The split tensor 1 to split tensor n shown can be used as input tensors of other operation nodes.

[0182] In one possible implementation, when Figure 5 When the merge operation node of the original data flow graph shown is merged by row, the split operation node can perform a split operation along the row, that is, after obtaining the target output result [A, C], it can be split along the row to obtain the first split tensor [a, C] and the second split tensor [a, C].

[0183] In another possible embodiment, when Figure 5When the merge operation node of the original data flow graph shown is merged by column, the split operation node can perform a split operation along the column, that is, after obtaining the target output result [A, C], it can be split along the column to obtain the first split tensor [A, c] and the second split tensor [A, c].

[0184] In summary, using the method of the embodiment of the present application, multiple matrix multiplication nodes can be aggregated and optimized to greatly reduce the number of matrix multiplication nodes, so that data processing can be efficiently aggregated and run on a graphics processing unit (GPU). This effect is particularly obvious when the number of matrix multiplication nodes in the original data flow graph is large. In this way, N-degree resource scheduling should be performed on N matrix multiplication nodes. After optimization, the number of scheduling can be greatly reduced, thereby reducing scheduling overhead. In addition, in the scenario of serial scheduling, the original scheduling method cannot fully utilize resources. For example, the original processing resources have 100% processing power, but the resource scheduling of a small tensor matrix multiplication node can only consume 20% of the processing power. Then the remaining 80% of the processing power is idle, and the resources cannot be fully utilized. After aggregation, a large tensor matrix multiplication node needs to consume more processing power, so that resources can be better utilized each time resource scheduling is performed, thereby improving resource utilization. Moreover, after experimental measurement on a certain AI deep learning model, the method of the embodiment of the present application can achieve a 30% performance improvement on a single machine.

[0185] See Figure 18 Based on the same inventive concept, the embodiment of the present application further provides a data flow graph processing device 180, which includes:

[0186] The flow graph parsing unit 1801 is used to obtain the original data flow graph and parse the original data flow graph;

[0187] A detection unit 1802 is configured to detect whether the N matrix multiplication nodes meet an aggregation condition if the parsed original data flow graph includes a merge operation node, and the input tensor of the merge operation node is the output tensor of N matrix multiplication nodes;

[0188] The flow graph optimization unit 1803 is configured to aggregate the N matrix multiplication nodes to obtain an aggregated matrix multiplication node if the aggregation condition is met, and replace the merge operation node with M stacking operation nodes; further, swap the positions of the aggregated matrix multiplication node and the M stacking operation nodes, and add a first deformation node after the aggregated matrix multiplication node to obtain a first optimized data flow graph;

[0189] Among them, in the first optimized data flow graph, the input tensor of each stacking operation node is: the input tensor of the same type among the M types of input tensors corresponding to the N matrix multiplication nodes, M and N are both positive integers greater than 1, the output tensor of the M stacking operation nodes is the input tensor of the aggregate matrix multiplication node, and the output tensor of the aggregate matrix multiplication node is the input tensor of the first deformation node.

[0190] Optionally, the M stacking operation nodes include a first stacking operation node and a second stacking operation node; the apparatus further includes a configuration unit 1804 and a model running unit 1805;

[0191] a configuration unit 1804 configured to configure a data processing model according to the first optimized data flow graph, obtain N sets of input tensor combinations, and input the N sets of input tensor combinations into the data processing model; each set of input tensor combinations includes a leading input tensor and a trailing input tensor to be subjected to a matrix multiplication operation;

[0192] The model running unit 1805 is used to perform a stacking operation on the leading input tensor in the N groups of tensor combinations through the first stacking operation node in the data processing model to obtain a first stacked tensor; and to perform a stacking operation on the trailing input tensor in the N groups of tensor combinations through the second stacking operation node to obtain a second stacked tensor; to perform a matrix multiplication operation on the first stacked tensor and the second stacked tensor through the aggregated matrix multiplication node to obtain a first output tensor; and to transform the shape of the first output tensor through the first deformation node to obtain a target output tensor.

[0193] Optionally, the flow graph optimization unit 1803 is specifically configured to:

[0194] Determine whether the operator library includes a stacking operation operator corresponding to the stacking operation node; wherein the operator library provides data processing operators corresponding to each node;

[0195] If the operator library includes a stacking operation operator, the merge operation node is replaced with M stacking operation nodes.

[0196] Optionally, the flow graph optimization unit 1803 is further configured to:

[0197] If the operator library does not support stacking operations, add M-1 merge operation nodes and add a second deformation node after each merge operation node;

[0198] Aggregate N matrix multiplication nodes to obtain an aggregated matrix multiplication node;

[0199] Adjusting the position of the aggregation matrix multiplication node to after the second deformation node, and adding the first deformation node after the aggregation matrix multiplication node to obtain a second optimized data flow graph;

[0200] In which, in the second optimized data flow graph, the input tensor of each merge operation node is an input tensor of the same type among the M types of input tensors, the output tensor of the second deformation node is the input tensor of the aggregation matrix multiplication node, and the output tensor of the aggregation matrix multiplication node is the input tensor of the first deformation node.

[0201] Optionally, the second optimized data flow graph includes a first merge operation node and a second merge operation node;

[0202] The configuration unit 1804 is further configured to configure the data processing model according to the second optimized data flow graph; and obtain N groups of input tensor combinations, and input the N groups of input tensor combinations into the data processing model; each group of input tensor combinations includes a pre-input tensor and a post-input tensor to be subjected to a matrix multiplication operation;

[0203] The model running unit 1805 is also used to perform a merge operation on the leading input tensors in the N groups of tensor combinations through the first merge operation node in the data processing model to obtain a first merged tensor; and to perform a merge operation on the trailing input tensors in the N groups of tensor combinations through the second merge operation node to obtain a second merged tensor; respectively transform the shapes of the corresponding first merged tensor and the second merged tensor through each second deformation node to obtain a first deformed tensor and a second deformed tensor; perform a matrix multiplication operation on the first deformed tensor and the second deformed tensor through the aggregation matrix multiplication node to obtain a first output tensor; and transform the shape of the first output tensor through the first deformation node to obtain a target output tensor.

[0204] Optionally, the flow graph optimization unit 1803 is further configured to add a split operation node after the first deformation node if it is found that the output tensor of the matrix multiplication node is the input tensor of other operation nodes in the original data flow graph; wherein the other operation nodes are operation nodes other than the merge operation node;

[0205] The model running unit 1805 is also used to perform a splitting operation on the target output tensor through a splitting operation node to obtain N split tensors corresponding to the N matrix multiplication nodes in the original data flow graph; wherein, one split tensor corresponds to one matrix multiplication node in the original data flow graph, and in the third optimized data flow graph, the N split tensors are input tensors of other operation nodes.

[0206] Optionally, aggregation conditions include:

[0207] Among the M types of input tensors, the N input tensors of the N matrix multiplication nodes corresponding to any type have the same size.

[0208] Optionally, the merge operation node is a merge operation node that performs a merge operation on a column basis;

[0209] The detection unit 1802 is further configured to detect whether the matrix multiplication operator in the operator library supports the batch dimension as the intermediate dimension;

[0210] The flow graph optimization unit 1803 is further configured to, if the matrix multiplication operator does not support the intermediate dimension being a batch dimension, replace the merge operation node with M stacking operation nodes, aggregate the N matrix multiplication nodes to obtain an aggregated matrix multiplication node; swap the positions of the aggregated matrix multiplication node and the M stacking operation nodes, and add a transpose operation node and a first deformation node after the aggregated matrix multiplication node to obtain a third optimized data flow graph;

[0211] Among them, in the third optimized data flow graph, the output tensors of the M stacking operation nodes are the input tensors of the aggregate matrix multiplication node, the output tensor of the aggregate matrix multiplication node is the input tensor of the transpose operation node, and the output tensor of the transpose operation node is the input tensor of the first deformation node.

[0212] Optionally, the M stacking operation nodes include a first stacking operation node and a second stacking operation node;

[0213] The configuration unit 1804 is further configured to configure the data processing model according to the third optimized data flow graph; and obtain N groups of input tensor combinations, and input the N groups of input tensor combinations into the data processing model; each group of input tensor combinations includes a pre-input tensor and a post-input tensor to be subjected to a matrix multiplication operation;

[0214] The model running unit 1805 is also used to perform a stacking operation on the leading input tensor in the N groups of tensor combinations through the first stacking operation node in the data processing model to obtain a first stacked tensor; and to perform a stacking operation on the trailing input tensor in the N groups of tensor combinations through the second stacking operation node to obtain a second stacked tensor; to perform a matrix multiplication operation on the first stacked tensor and the second stacked tensor through the aggregate matrix multiplication node to obtain a first output tensor; to perform a transpose operation on the first output tensor through the transpose operation node to obtain a transposed tensor; and to transform the shape of the transposed tensor through the first deformation node to obtain a target output tensor.

[0215] The device can be used to perform Figures 2 to 17 The method shown in the embodiment shown, therefore, for the functions that can be realized by each functional module of the device, please refer to Figures 2 to 15 The description of the illustrated embodiment is omitted for brevity.

[0216] See Figure 19 Based on the same technical concept, an embodiment of the present application also provides a computer device 190 , which may include a memory 1901 and a processor 1902 .

[0217] The memory 1901 is used to store computer programs executed by the processor 1902. The memory 1901 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the computer device, etc. The processor 1902 may be a central processing unit (CPU), or a digital processing unit, etc. The specific connection medium between the above-mentioned memory 1901 and the processor 1902 is not limited in the embodiment of the present application. The embodiment of the present application is Figure 19 In the embodiment, the memory 1901 and the processor 1902 are connected via a bus 1903. The bus 1903 is connected to the processor 1902 via a bus 1903. Figure 19 The bus 1903 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 19 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0218] Memory 1901 may be a volatile memory, such as random-access memory (RAM); a non-volatile memory, such as read-only memory, flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1901 may be a combination of the aforementioned memories.

[0219] Processor 1902 is configured to execute the following when calling the computer program stored in the memory 1901: Figures 2 to 17 The method executed by the device in the embodiment shown.

[0220] In some possible implementations, various aspects of the method provided in the present application may also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to perform the steps of the method according to various exemplary embodiments of the present application described above in this specification. For example, the computer device may perform the following steps: Figures 2 to 17 The method executed by the device in the embodiment shown.

[0221] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0222] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0223] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A data flow graph processing method, characterized in that: The method comprises: Obtaining an original data flow graph and parsing the original data flow graph; If the original data flow graph is parsed to include a merge operation node, and the input tensor of the merge operation node is the output tensor of N matrix multiplication nodes, then detecting whether the N matrix multiplication nodes meet the aggregation condition; If the aggregation condition is met, the N matrix multiplication nodes are aggregated to obtain an aggregated matrix multiplication node, and the merge operation node is replaced with M stacking operation nodes; wherein the input tensor of each stacking operation node is: an input tensor of the same type among the M types of input tensors corresponding to the N matrix multiplication nodes; wherein M and N are both positive integers greater than 1; The positions of the aggregate matrix multiplication node and the M stacking operation nodes are swapped, and a first deformation node is added after the aggregate matrix multiplication node to obtain a first optimized data flow graph; wherein, in the first optimized data flow graph, the output tensors of the M stacking operation nodes are the input tensors of the aggregate matrix multiplication node, and the output tensor of the aggregate matrix multiplication node is the input tensor of the first deformation node.

2. The method according to claim 1, wherein The M stacking operation nodes include a first stacking operation node and a second stacking operation node; After obtaining the first optimized data flow graph, the method further includes: configuring a data processing model according to the first optimized data flow graph; Obtaining N sets of input tensor combinations, and inputting the N sets of input tensor combinations into the data processing model; each set of input tensor combinations includes a pre-input tensor and a post-input tensor to be subjected to a matrix multiplication operation; Performing a stacking operation on the preceding input tensors in the N groups of input tensor combinations through the first stacking operation node in the data processing model to obtain a first stacked tensor; and Performing a stacking operation on the subsequent input tensors in the N groups of input tensor combinations through the second stacking operation node to obtain a second stacked tensor; performing a matrix multiplication operation on the first stacked tensor and the second stacked tensor through the aggregate matrix multiplication node to obtain a first output tensor; The shape of the first output tensor is transformed by the first deformation node to obtain a target output tensor.

3. The method according to claim 1, wherein The merging operation node is replaced with M stacking operation nodes, including: Determining whether an operator library includes a stacking operation operator corresponding to the stacking operation node; wherein the operator library provides a data processing operator corresponding to each node; If the operator library includes the stacking operation operator, the merge operation node is replaced with the M stacking operation nodes.

4. The method according to claim 3, wherein After determining whether the operator library supports stacking operations, the method further includes: If the operator library does not support the stacking operation, add M-1 merge operation nodes and add a second deformation node after each merge operation node; Aggregating the N matrix multiplication nodes to obtain an aggregated matrix multiplication node; Adjusting the position of the aggregation matrix multiplication node to after the second deformation node, and adding the first deformation node after the aggregation matrix multiplication node to obtain a second optimized data flow graph; In which, in the second optimized data flow graph, the input tensor of each merge operation node is an input tensor of the same type among the M types of input tensors, the output tensor of the second deformation node is the input tensor of the aggregation matrix multiplication node, and the output tensor of the aggregation matrix multiplication node is the input tensor of the first deformation node.

5. The method according to claim 4, wherein The second optimized data flow graph includes a first merge operation node and a second merge operation node. After obtaining the second optimized data flow graph, the method further includes: configuring a data processing model according to the second optimized data flow graph; Obtaining N sets of input tensor combinations, and inputting the N sets of input tensor combinations into the data processing model; each set of input tensor combinations includes a pre-input tensor and a post-input tensor to be subjected to a matrix multiplication operation; Performing a merge operation on the preceding input tensors in the N groups of input tensor combinations through the first merge operation node in the data processing model to obtain a first merged tensor; and Performing a merge operation on the subsequent input tensors in the N groups of input tensor combinations through the second merge operation node to obtain a second merged tensor; Transforming the shapes of the corresponding first combined tensor and the second combined tensor through each second deformation node, respectively, to obtain a first deformation tensor and a second deformation tensor; Performing a matrix multiplication operation on the first deformation tensor and the second deformation tensor through the aggregate matrix multiplication node to obtain a first output tensor; The shape of the first output tensor is transformed by the first deformation node to obtain a target output tensor.

6. The method according to claim 2 or 5, characterized in that The method further comprises: If it is found that the output tensor of the matrix multiplication node is the input tensor of other operation nodes in the original data flow graph, a split operation node is added after the first deformation node; wherein the other operation nodes are operation nodes other than the merge operation node; After converting the shape of the first output tensor through the first deformation node to obtain a target output tensor, the method further includes: The target output tensor is subjected to a splitting operation through the splitting operation node to obtain N splitting tensors corresponding to the N matrix multiplication nodes in the original data flow graph; wherein, one splitting tensor corresponds to one matrix multiplication node in the original data flow graph, and in the obtained optimized data flow graph, the N splitting tensors are the input tensors of the other operation nodes.

7. The method according to any one of claims 1 to 5, characterized in that: The polymerization conditions include: Among the M types of input tensors, the N input tensors of the N matrix multiplication nodes corresponding to any type have the same size.

8. The method according to any one of claims 1 to 5, characterized in that: The merge operation node is a merge operation node that performs a column-based merge operation. After detecting whether the N matrix multiplication nodes meet the aggregation condition, the method further includes: If the aggregation conditions are met, check whether the matrix multiplication operator in the operator library supports the intermediate dimension as the stacking dimension; If the matrix multiplication operator does not support the middle dimension as the stacking dimension, the merge operation node is replaced with the M stacking operation nodes, and the N matrix multiplication nodes are aggregated to obtain an aggregated matrix multiplication node; Swapping the positions of the aggregate matrix multiplication node and the M stacking operation nodes, and adding a transposition operation node and the first deformation node after the aggregate matrix multiplication node to obtain a third optimized data flow graph; In the third optimized data flow graph, the output tensors of the M stacking operation nodes are the input tensors of the aggregate matrix multiplication node, the output tensor of the aggregate matrix multiplication node is the input tensor of the transposition operation node, and the output tensor of the transposition operation node is the input tensor of the first deformation node.

9. The method according to claim 8, wherein The M stacking operation nodes include a first stacking operation node and a second stacking operation node; after obtaining the third optimized data flow graph, the method further includes: configuring a data processing model according to the third optimized data flow graph; Obtaining N sets of input tensor combinations, and inputting the N sets of input tensor combinations into the data processing model; each set of input tensor combinations includes a pre-input tensor and a post-input tensor to be subjected to a matrix multiplication operation; Performing a stacking operation on the preceding input tensors in the N groups of input tensor combinations through the first stacking operation node in the data processing model to obtain a first stacked tensor; and Performing a stacking operation on the subsequent input tensors in the N groups of input tensor combinations through the second stacking operation node to obtain a second stacked tensor; performing a matrix multiplication operation on the first stacked tensor and the second stacked tensor through the aggregate matrix multiplication node to obtain a first output tensor; Performing a transposition operation on the first output tensor by using the transposition operation node to obtain a transposed tensor; The shape of the transposed tensor is transformed by the first deformation node to obtain a target output tensor.

10. A data flow graph processing device, characterized in that: The device comprises: A flow graph parsing unit, configured to obtain an original data flow graph and parse the original data flow graph; a detection unit, configured to detect whether the N matrix multiplication nodes meet an aggregation condition if the parsed original data flow graph includes a merge operation node, and the input tensor of the merge operation node is the output tensor of N matrix multiplication nodes; a flow graph optimization unit, configured to aggregate the N matrix multiplication nodes to obtain an aggregated matrix multiplication node if an aggregation condition is met, and replace the merge operation node with M stacking operation nodes; further, swap the positions of the aggregated matrix multiplication node and the M stacking operation nodes, and add a first deformation node after the aggregated matrix multiplication node to obtain a first optimized data flow graph; Among them, in the first optimized data flow graph, the input tensor of each stacking operation node is: the input tensor of the same type among the M types of input tensors corresponding to the N matrix multiplication nodes, M and N are both positive integers greater than 1, and the output tensor of the M stacking operation nodes is the input tensor of the aggregate matrix multiplication node, and the output tensor of the aggregate matrix multiplication node is the input tensor of the first deformation node.

11. The device according to claim 10, wherein The M stacking operation nodes include a first stacking operation node and a second stacking operation node; the device further includes a configuration unit and a model operation unit; The configuration unit is configured to configure a data processing model according to the first optimized data flow graph, obtain N groups of input tensor combinations, and input the N groups of input tensor combinations into the data processing model; each group of input tensor combinations includes a pre-input tensor and a post-input tensor to be subjected to a matrix multiplication operation; The model running unit is configured to perform a stacking operation on the preceding input tensors in the N groups of input tensor combinations through the first stacking operation node in the data processing model to obtain a first stacked tensor; and performing a stacking operation on the subsequent input tensors in the N groups of input tensor combinations through the second stacking operation node to obtain a second stacked tensor; performing a matrix multiplication operation on the first stacked tensor and the second stacked tensor through the aggregate matrix multiplication node to obtain a first output tensor; The shape of the first output tensor is transformed by the first deformation node to obtain a target output tensor.

12. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images