Optimization method of computational graph, computing device and related products

Abstract

The present disclosure discloses an optimization method of a computing graph, a computing device, a computer readable storage medium and a computer program product. The computing device performing the optimization method of the computing graph can be included in a combined processing device, which can further include an interface device and other processing devices. The computing device interacts with the other processing devices to jointly complete a user-specified computing operation. The combined processing device can further include a storage device connected to the computing device and the other processing devices, respectively, for storing data of the computing device and the other processing devices. The scheme of the present disclosure can reduce the memory transfer and operator call of the device end by optimizing the view class operator subgraph, thereby improving the processing efficiency of the machine.

Description

Technical Field

[0001] This disclosure generally relates to the field of intelligent computing, and more particularly to the field of compilers. More specifically, this disclosure relates to a method for optimizing a computation graph, a computing device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] In intelligent computing systems, programming frameworks provide programmers with an interface to use hardware and the system, serving as a crucial core hub. On one hand, programming frameworks can encapsulate commonly used operations in algorithms into operators for programmers to directly call, such as convolution and pooling. On the other hand, as the interface between software and hardware, programming frameworks can encapsulate hardware architecture, thereby reducing the complexity and difficulty of writing or applying deep learning algorithms and improving the efficiency of algorithm implementation.

[0003] TensorFlow and PyTorch are currently popular deep learning frameworks. These frameworks typically use computation graphs to describe the computation process of machine learning algorithms, tensors to represent all data within the computation graph, and operators to represent various operations. There exists a class of operators, such as transpose, slice, and split, that only change the outward appearance or look of the tensor data without altering its actual arrangement in memory; that is, they do not perform actual memory data movement. These operators can be called view operators.

[0004] Due to this characteristic of view-class operators, tensor data is typically non-contiguous in memory. Reading and operating on this non-contiguous data leads to low memory access efficiency and high processing time in hardware. Furthermore, when there are many view-class operators, a large amount of memory data contiguous processing is required, resulting in significant time overhead. Summary of the Invention

[0005] To at least partially address one or more of the technical problems mentioned in the background section, this disclosure provides solutions from multiple perspectives. On one hand, it provides an optimization method for computation graphs by constructing view-class operator subgraphs for subsequent memory data continuity processing. On the other hand, it provides a further optimization method for computation graphs, which can perform operator fusion based on the pre-constructed view-class operator subgraph and the relationships between view-class operators, reducing on-device memory movement and operator calls, thereby improving data access efficiency.

[0006] In a first aspect, this disclosure discloses a method for optimizing a computation graph, comprising: obtaining a view-class operator subgraph of tensor data in the computation graph, wherein the view-class operator subgraph includes source operators of view classes associated with the tensor data; replacing the source operators in the view-class operator subgraph with specified target operators whose functions can be mutually substituted, based on the functions of the source operators; and merging multiple consecutive identical target operators into a single target operator to generate a merged view-class operator subgraph.

[0007] In a second aspect, this disclosure discloses a computing apparatus for optimizing a computation graph, comprising: a processor configured to execute program instructions; and a memory configured to store the program instructions, which, when loaded and executed by the processor, cause the processor to perform an optimization method for the computation graph according to a first aspect of this disclosure.

[0008] In a third aspect, this disclosure discloses a computer-readable storage medium storing program instructions that, when loaded and executed by a processor, cause the processor to perform an optimization method for a computational graph according to a first aspect of this disclosure.

[0009] In a fourth aspect, this disclosure discloses a computer program product, including a computer program or instructions that, when executed by a processor, implement the method for optimizing the computational graph of the first aspect of this disclosure.

[0010] Based on the optimization method of the computation graph provided above, the operator subgraph pre-built based on the view class operator in the computation graph can be optimized and the same type of operator can be merged. This can reduce data movement in memory and operator calls, and improve data access efficiency. Attached Figure Description

[0011] The above and other objects, features, and advantages of exemplary embodiments of this disclosure will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this disclosure are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:

[0012] Figure 1 Examples illustrate different shapes of multidimensional arrays and their storage order in memory;

[0013] Figure 2 An exemplary flowchart of an optimization method for a computational graph according to an embodiment of this disclosure is shown;

[0014] Figure 3 An exemplary flowchart of an optimization method for a computational graph according to another embodiment of this disclosure is shown;

[0015] Figures 4a-4c The structures of several exemplary computation graphs and the corresponding constructed view class operator subgraphs are shown.

[0016] Figures 5a-5b A simple example of operator fusion is shown;

[0017] Figure 6 An exemplary method flowchart for operator fusion according to some embodiments of this disclosure is shown;

[0018] Figure 7 An exemplary flowchart of a data processing method according to an embodiment of this disclosure is shown;

[0019] Figure 8 A block diagram of the hardware configuration of a computing device that can implement various embodiments of the present disclosure is shown;

[0020] Figure 9 A structural diagram of a combined processing apparatus according to an embodiment of the present disclosure is shown; and

[0021] Figure 10 A schematic diagram of the structure of a board card according to an embodiment of this disclosure is shown. Detailed Implementation

[0022] The technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, not all of them. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0023] It should be understood that the terms "first," "second," "third," and "fourth," etc., that may appear in the claims, specification, and drawings of this disclosure are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" as used in the specification and claims of this disclosure indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0024] It should also be understood that the terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0025] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection."

[0026] The embodiments disclosed herein are described in detail below with reference to the accompanying drawings.

[0027] In the programming framework of intelligent computing systems, data is typically modeled as tensors. A tensor can be viewed as an N-dimensional array, and the dimension of the array is the order of the tensor. Therefore, a 0th-order tensor corresponds to scalar data; a 1st-order tensor corresponds to a one-dimensional array, i.e., a vector; a 2nd-order tensor corresponds to a two-dimensional array, i.e., a matrix; and so on, an Nth-order tensor corresponds to an N-dimensional array. For example, an RGB image can be represented as a 3rd-order tensor, while a dataset consisting of multiple RGB images can be represented as a 4th-order tensor.

[0028] Each tensor has some common properties, including data type and shape. The shape of a tensor represents the length of each order. For example, a 0th-order tensor corresponds to a scalar data, and its shape is empty; a 1st-order tensor corresponds to a one-dimensional vector, and its shape contains one element, the value of which is the length of the vector; a 2nd-order tensor corresponds to a matrix, and its shape contains two elements, corresponding to the lengths of the rows and columns respectively; a 3rd-order tensor corresponds to three-dimensional data, and its shape contains three elements, each corresponding to the length of its order.

[0029] Although multidimensional arrays have multiple dimensions, because the layout of memory (e.g., DRAM and cache RAM) is always one-dimensional, there is a correspondence between multidimensional arrays and their storage order in memory. Multidimensional arrays are typically allocated in contiguous storage space, meaning they can be unfolded one-dimensionally and stored sequentially in memory.

[0030] Figure 1 Examples illustrate different shapes of multidimensional arrays and their storage order in memory, where a one-dimensional array of a contiguous block of memory is used to implement the storage of the multidimensional array.

[0031] Figure 1 Figure (a) shows the first data, namely the three-dimensional array X, which has three dimensions: dimension 0 (dim0), dimension 1 (dim1), and dimension 2 (dim2). The size of dimension 0 is 2, the size of dimension 1 is 2, and the size of dimension 2 is 3. Therefore, its shape (size) can be represented as: X3 = (2, 2, 3).

[0032] Figure 1Figure (c) illustrates the storage order of the three-dimensional array X in memory. Data with the same background in the figure represent data located in the same dimension. Assuming storage follows a low-dimensional priority order (e.g., from left to right in the shape representation corresponding to high to low dimensions), expanding the first data in one dimension yields:

[0033] X=[1,2,3,4,5,6,7,8,9,10,11,12].

[0034] More specifically, the data in the lowest dimension (within the same row) is continuous, while the data in higher dimensions are spaced at different intervals. For example, in the storage configuration shown in Figure (c), accessing the physical structure of adjacent elements in dimension dim2 requires an offset of 1 position (e.g., from data 1 to data 2, data 5 to data 6, etc.); accessing the physical structure of adjacent elements in dimension dim1 requires an offset of 3 positions (e.g., from data 1 to data 4, data 2 to data 5, ..., data 9 to data 12, etc.); while accessing the physical structure of adjacent elements in dimension dim0 requires an offset of 6 positions (e.g., from data 1 to data 7, data 2 to data 8, ..., data 6 to data 12, etc.). This offset is called the stride. The stride of each dimension of the three-dimensional array X can be represented as S. X = (6,3,1).

[0035] In the programming framework of intelligent computing systems, there exist view-class operators that manipulate the external representation of tensors, such as transpose, slice, and split. Taking transpose as an example, it transforms tensors according to a certain dimensional transformation rule (perm). N = (p1, p2, ..., p i ,…,p N ), to obtain the data arrangement after dimensional transformation, where p i The values ​​of (i∈1,2,…,N) represent the original dimensions of the array, p i In perm N The position in the array represents the target dimension of the transformation. For example, given the dimension transformation rule perm3 = (0, 2, 1), it means that dimension 1 and dimension 2 should be swapped, that is, the original dimension 1 should be transformed into dimension 2 of the new array, and the original dimension 2 should be transformed into dimension 1 of the new array.

[0036] Figure 1Figure (b) shows the transformed array Y obtained after performing the transpose operator on the 3D array X shown in Figure (a). In this example, the exemplary dimension transformation rule perm3 = (0, 2, 1) is applied. As can be seen from the figure, compared to array X, dimensions 1 and 2 of array Y have been swapped. At this time, the dimension information of the 3D array Y can be represented as: Y3 = (2, 3, 2).

[0037] However, since view class operators do not change the storage location of data in memory, the array Y obtained after the transpose operation is still stored in memory in the same order as before. Figure 1 As shown in (c). At this point, according to the storage order in (c), the step size of each dimension of array Y becomes S. Y = (6,1,3). It can be seen that if storing data in order of lower dimension priority is called continuity, then the current storage order of array Y is discontinuous. That is, after the transpose operator, because the dimensional order of the array is changed, but the storage location in memory is not changed, the storage order of the array in memory becomes discontinuous.

[0038] If we want the array Y to be contiguous in memory, then following the principle of prioritizing lower dimensions, its one-dimensional expansion should be: Figure 1 As shown in Figure (d):

[0039] Y=[1,4,2,5,3,6,7,10,8,11,9,12].

[0040] The shape of a tensor helps programmers develop an intuitive understanding of it. In programming frameworks such as PyTorch, View class operators can change the tensor's shape (size), stride (the span between the first indices of adjacent dimensions), and storage offset (the offset of the tensor's first element relative to its initial storage position), but they do not change the tensor's actual storage location. In this case, the tensor uses size, stride, and storage offset to calculate the data's memory location on the device.

[0041] Assume the size of the tensor is (s0, s1, s2, ..., s). i ), where the stride is (y0, y1, y2, ..., y i If storage_offset is b, then the tensor at point (x0, x1, x2, ..., x...) is at the point (x0, x1, x2, ..., x...). i The basic formula for calculating the memory location corresponding to point ) is:

[0042]

[0043] Where dptr is the starting position of the tensor in memory, and dtype is the data type of the tensor.

[0044] Based on the above Figure 1 As the description shows, tensors in the computation graph, after being processed by view-class operators, produce discontinuous corresponding data. When accessing these tensors, traditional CPUs and GPUs need to read and access this discontinuous data according to the formula described above, which leads to low memory access efficiency and high time consumption in hardware devices. Another approach is to call the contiguous() operator, which first moves the data into contiguous storage according to the formula described above before performing subsequent access and operations. However, this method is very time-consuming; when dealing with large amounts of data, moving the data one by one will result in significant time consumption.

[0045] In view of this, considering that in computation graph operations such as neural networks, the generation of discontinuous memory data is often caused by view-class operators, this disclosure proposes a scheme for constructing view-class operator subgraphs for view-class operators in computation graphs. This view-class operator subgraph can then support the efficient subsequent process of moving memory data from discontinuous to continuous.

[0046] Regarding the terms "node" and "operator" mentioned in this disclosure, it should be noted that the term "operator" refers to the computational aspect of computing (or the software or algorithmic aspect); while the term "node" is a more descriptive term (from a graphical or more intuitive perspective). In terms of their meaning, "operator" and "node" are essentially the same. That is, in this disclosure, the terms "operator" and "node" can be considered to have the same meaning and can be used interchangeably, simply describing things from different perspectives.

[0047] Figure 2 An exemplary flowchart of an optimization method for a computation graph according to an embodiment of this disclosure is shown. In this optimization method, subsequent in-memory data continuity processing is supported by constructing a view-class operator subgraph.

[0048] As shown in the figure, in step 210, for the tensor data in the computation graph, the operators associated with the tensor data are traversed.

[0049] A computation graph is a directed graph consisting of nodes and edges, where tensors are passed between nodes. Execution in a computation graph follows the order of the directed graph; each time a tensor passes through a node, it is used as input to the operation performed at that node, and the result flows along the output edge of that node to the next node. Therefore, when constructing an operator subgraph for a view class, you can traverse the nodes or operators that need to be processed on the tensor data, following the order of the directed graph.

[0050] Next, in step 220, when the operator encountered during traversal is a view-type operator, the operator is extracted to construct a view-type operator subgraph.

[0051] In some embodiments, extracting view-class operators to construct a view-class operator subgraph may include: associatively caching the operator information and operator index of the operator; and adding the operator index to the view-class operator subgraph. In these embodiments, by storing the operator information and the view-class operator subgraph separately and establishing the relationship between the two through the operator index, the structure of the view-class operator subgraph can be simplified, facilitating subsequent memory data continuity processing.

[0052] Each operator has attributes to identify relevant information during operation execution. Common attributes include: operator name, operator type, operator input data, operator output data, and operation parameters. In some embodiments, the cached operator information may include at least one of the following: description information of the operator's input data, description information of the output data, and operation parameters. It can be understood that the input and output data of the operator are both tensor data, and the description information of tensor data mainly includes the shape, stride, and storage offset mentioned earlier.

[0053] The operational parameters of an operator are related to the function it performs. For example, the transpose operator's operational parameters may include the two dimensions to be swapped (dim0, dim1). Similarly, the chunk operator's function is to divide a tensor into equal parts along dimension dim; its operational parameters may include the number of chunks to be divided (chunks) and the dimension to be divided (dim).

[0054] The above describes extracting view-class operators that cause discontinuities in memory data to construct view-class operator subgraphs to support subsequent processing of memory data continuity. It can be understood that a view-class operator subgraph can be constructed for each tensor data. Furthermore, it can be understood that each tensor data can include multiple view-class operator subgraphs based on the continuity of view-class operators in the computation graph.

[0055] Figure 3 An exemplary flowchart of an optimization method for a computation graph according to another embodiment of this disclosure is shown. In this embodiment, the construction of the view class operator subgraph can be further optimized to simplify information storage.

[0056] As shown in the figure, when extracting operators to construct view-class operator subgraphs, for each encountered view-class operator, step 310 first checks whether the operator information has been cached in memory. Operator information may include, for example, the description information of the operator's input data, the description information of its output data, and the operation parameters mentioned above.

[0057] If the operator information is not cached, it means that the operator is a new operator relative to the operators in memory. The process proceeds to step 320, where an operator sequence number is generated for the operator and the previously described associated caching of the operator information and operator sequence number is performed. Then, in step 330, the operator sequence number is added to the view class operator subgraph.

[0058] If the operator information is already cached, there is no need to cache the same information again. Instead, the process proceeds directly to step 330, where only the operator index of the cached operator is added to the view class operator subgraph.

[0059] By using the above processing methods, the amount of cached information can be effectively reduced, and the construction of view class operator subgraphs can be simplified.

[0060] Figures 4a-4c Several exemplary computation graph structures and the corresponding constructed view class operator subgraph structures are shown.

[0061] Figure 4a A unidirectional computation graph is shown, in which the input tensor A 410 sequentially passes through the following nodes according to the flow of the computation graph: transpose operator 411, slice operator 412, slice operator 413, and Matmul (matrix multiplication) operator 414. Among these operators, transpose operator 411, slice operator 412, and slice operator 413 are all view-type operators, while Matmul (matrix multiplication) operator 414 is a computation-type operator.

[0062] According to the view class operator subgraph construction scheme of this disclosure embodiment, these view class operators are extracted to form a view class operator subgraph. For example... Figure 4a As shown on the right, for the input tensor A, the view class operator subgraphs sequentially include the transpose operator 411, the slice operator 412, and the slice operator 413.

[0063] In some embodiments, assuming the operator number generated by the transpose operator 411 is 1, the operator number of the slice operator 412 is 2, and the operator number of the slice operator 413 is 3, the constructed view class operator subgraph can be represented as 1->2->3 using these operator numbers. The operator information of the corresponding operator can be extracted from the cached information using the operator numbers.

[0064] In other embodiments, assuming that slice operator 412 and slice operator 413 have the same operator information, they can store only one copy of the operator information and share the same operator index. In this embodiment, when processing slice operator 413, if it is found that the operator information of slice operator 413 is the same as the operator information cached for the previous slice operator 412, then there is no need to perform the caching step. Instead, the operator index 2 of the cached slice operator 412 is directly assigned to slice operator 413 and added to the view class operator subgraph. At this time, the constructed view class operator is represented by the operator index 1->2->2.

[0065] Figure 4b A computational graph of a residual structure is shown, where the input tensor B 420 flows through the following nodes according to the computational graph: view operator 421, Conv (convolution) operator 422, Act (activation) operator 423, and Add (addition) operator 424. The output of view operator 421 is also input to Add operator 424 as its other addend. Among these operators, only view operator 421 belongs to the view class, while the rest are computation class operators.

[0066] According to the view-class operator subgraph construction scheme of this disclosure embodiment, view-class operators included in the computation graph are extracted to form a view-class operator subgraph. For example... Figure 4b As shown on the right, for the input tensor B, the view class operator subgraph only includes the view operator 421.

[0067] Figure 4cA multi-branch computation graph is shown, in which the input tensor C 430 flows through the following nodes according to the direction of the computation graph: split operator 431, transpose1 operator 432, transpose2 operator 433, transpose3 operator 434 located on the three branches respectively, BMM1 operator 435, Softmax operator 436, which operates on the outputs of the first and second branches, and BMM2 operator 437, which operates on the results of the first two branches and the output of the third branch. Among these operators, split operator 431 and the three transpose operators 432-434 are view operators, and the rest are computation operators.

[0068] According to the view-class operator subgraph construction scheme of this disclosure embodiment, view-class operators included in the computation graph are extracted to form a view-class operator subgraph. For example... Figure 4c As shown on the right, for the input tensor C, the view-class operator subgraph can be divided into three branches based on the operation parameters of the split operator 431, such as the number of data blocks it is divided into. Each branch includes the split operator 431 and one of the corresponding transpose operators 432 to 434. It can be seen that when the view-class operator is a multi-branch operator, a view-class operator subgraph including the corresponding number of branches can be constructed based on the multi-branch operator.

[0069] The above description, using several examples, illustrates the construction scheme of the view class operator subgraph provided in this disclosure embodiment. As can be seen from the constructed view class operator subgraph, the current subgraph only extracts consecutive view class operators without further processing. When there are many view class operators, performing continuous memory data processing for each operator one by one will lead to frequent operator calls and data movement, resulting in repeated memory accesses, low memory access efficiency, and increased network latency.

[0070] A typical optimization for computation graphs is operator fusion, which involves computing multiple operators together in a single kernel without saving intermediate results back to global memory.

[0071] To better understand operator fusion, Figures 5a-5b A simple example of operator fusion is shown.

[0072] Suppose there are two operators in the graph that are executed sequentially: the first operator and the second operator, which will be represented by ① and ② below. Figure 5a To avoid using the operator fusion operation, the operation process is as follows:

[0073] 1) Read the input of the entire computation graph (that is, the input of ①) from DRAM (Dynamic Random Access Memory) to on-chip memory, such as PNM (Parallel Neuron Memory), and read the weights of ① to on-chip memory, such as PWM (Parallel Weight Memory);

[0074] 2) The PFU (parallel functional unit) arithmetic unit fetches data from PNM and PWM to complete the calculation and writes the result of ① back to PNM;

[0075] 3) Write the result of ① back from PNM to DRAM as the input of ②.

[0076] Then, the second operator ② is executed.

[0077] 4) Read the input of ② from DRAM into PNM, and the weights of ② into PWM;

[0078] 5) The PFU arithmetic unit retrieves data from PNM and PWM to complete the calculation, and writes the result of ② back to PNM.

[0079] 6) Write the result of ② back into DRAM as the output of the entire computation graph.

[0080] Figure 5b The operation flow after operator fusion is shown. The operation process is as follows:

[0081] A) Read the input of the entire computation graph (i.e., the input of ①) from DRAM into PNM, and the weights of ① and ② into PWM;

[0082] B) The PFU arithmetic unit retrieves data from PNM and PWM to complete the calculation, and writes the result of ① back to PNM;

[0083] C) The PFU arithmetic unit retrieves data from PNM and PWM to complete the calculation, and writes the result of ② back to PNM;

[0084] D) Write the result of ② back into DRAM as the output of the entire computation graph.

[0085] The comparison between the two processes above shows that operator fusion can reduce steps 3) and 4) in the operation process before fusion, that is, reduce the redundant data transfer of the same piece of data (the result of ① in this example, which serves as the input of ②) from PNM to DRAM and DRAM to PNM, that is, reduce the data access steps of intermediate results, thereby improving the operation speed.

[0086] In practice, the merged operator employs compilation optimization techniques such as memory reuse, memory access optimization, instruction pipelining, and data type optimization (e.g., selecting different applicable data types) during compilation, thereby significantly improving the overall performance of the merged operator.

[0087] Therefore, in this disclosed embodiment, a scheme is provided to perform operator fusion on the view class operator subgraph constructed by the above method to optimize the operator subgraph, thereby optimizing the subsequent memory data continuity processing.

[0088] Figure 6 An exemplary method flowchart for operator fusion according to some embodiments of this disclosure is shown. In this embodiment, operator fusion strategy selection is performed by scanning a pre-built view class operator subgraph.

[0089] As shown in the figure, in step 610, the view class operator subgraph of the tensor data in the computation graph is obtained, wherein the view class operator subgraph includes the source operators of the view class associated with the tensor data.

[0090] The View class operator subgraph is constructed based on the methods described above, such as... Figures 4a-4c Here are a few examples. It can be seen that the operators in the unoptimized operator subgraph are the original view-class operators in the computation graph, referred to here as source operators to distinguish them from the optimized operators.

[0091] Next, in step 620, based on the function of the source operator in the view class operator subgraph, it is replaced with a specified target operator whose function can be mutually substituted.

[0092] In programming frameworks such as PyTorch, there are various view operators to implement different functionalities. These operators include, but are not limited to: transpose, permute, select, chunk, narrow, slice, expand, view, and so on.

[0093] While these operators implement various specific functions, they can also be categorized. In some embodiments, they can be classified into three categories based on their impact on the data size: size reduction, size expansion, and size invariance. For example, among the operators listed above, transpose, permute, and view do not change the size of the tensor data and belong to the size invariance category; select, chunk, narrow, and slice reduce the size of the tensor data and belong to the size reduction category; while expand increases the size of the tensor data and belong to the size expansion category.

[0094] For each category of functionality, an operator can be selected to represent that category's functionality. This operator can implement the functionality of all operators under the corresponding functional category. That is, this operator can functionally replace all operators under the corresponding functional category. In this document, the replaced operator is called the "target operator," and the original operator is called the "source operator." Table 1 below provides several exemplary functional classifications, along with the source and target operators included in each category. It is understood that the operators here are merely exemplary and not exhaustive. Those skilled in the art can construct similar functional classifications and functionally substitutable target operators based on the principles of the embodiments disclosed herein.

[0095] Serial Number Source operator name Target operator name 1. Scale remains unchanged Transpose, permute, view permute 2. Scale reduction Select, chunk, narrow, slice slice 3. Scale expansion expand expand

[0096] Table 1

[0097] It can be seen that the source operators included in each category implement a subset of the corresponding target operators. By classifying these source operators according to their functions and replacing them with the specified target operators, the number of operator types in the operator subgraph can be reduced, facilitating subsequent fusion operations.

[0098] continue Figure 6 Finally, in step 630, multiple consecutive identical target operators in the replaced operator subgraph are merged into a single target operator to generate a merged view class operator subgraph.

[0099] Through the replacement in the previous step, view class operators with similar or related functions are replaced with the same target operator. When multiple identical target operators are consecutive in position, these target operators can be merged into a single target operator, thereby reducing the number of operators and consequently reducing the number of operators that need to be called subsequently.

[0100] In some embodiments, fusing multiple consecutive identical target operators into a single target operator may include merging the dimensional operations of the multiple target operators such that the fused single target operator is equivalent to the multiple target operators before fusion.

[0101] Under normal circumstances, the dimensionality operations of multiple consecutive target operators are executed sequentially, with each target operator performing dimensionality operations on its input tensor data in turn. Since the target operators are consecutive and identical, these dimensionality operations can be merged, thus achieving the effect of multiple dimensionality operations through a single target operator.

[0102] For example, suppose there are two consecutive operators in the view-class operator subgraph: the chunk operator and the split operator. Based on functional classification, both the chunk and split operators belong to the scaling type, so they are both replaced by the slice operator here. According to the embodiments disclosed herein, these two slice operators can be merged into one slice operator, and the dimension operations of these two slice operators also need to be merged into one.

[0103] The first slice operator corresponds to the original chunk operator. Assuming its dimension operation is to split the input tensor data D along dimension dim0 into two parts, then executing the chunk operator will split the tensor data D along dimension dim0 into two parts as evenly as possible.

[0104] The second slice operator corresponds to the original split operator. Assuming its dimension operation is to divide the input tensor data D along its dim1 dimension, with each block being approximately 4 units in size, then executing the split operator will divide the tensor data D along its dim1 dimension into blocks of approximately 4 units in size.

[0105] When these two slice operators are combined into one slice operator, the dimension operation it performs is to split the input tensor data D into two blocks along dimension dim0, and simultaneously divide dimension dim1 into blocks of size 4 as much as possible. This operation can be achieved by configuring the slice operator's computational parameters.

[0106] Optionally or additionally, in some embodiments, the position of a specific type of target operator in the view class operator subgraph can be adjusted to optimize processing.

[0107] In one example, the position of expansion operators (such as the expand operator) that cause an increase in memory data can be moved later. This postponement avoids adding memory data early on, which would increase the data movement of subsequent I / O operators. Preferably, the expand operator should be moved to the last stage of processing.

[0108] When placing an expand-type operator after the view operator, the position before and after the expand-type operator needs to be adjusted according to the expand-type operator. The parameters of the target operator in the view operator subgraph between the two need to be modified to adapt to the position adjustment.

[0109] For example, suppose the operator subgraph includes the expand operator, the permute operator, and the slice operator (assuming they have all been replaced with the target operator). The expand operator performs a dimension operation by expanding the dimensions of tensor data E (e.g., size1 = (1,3), representing a 1x3 matrix) to a new shape (e.g., size2 = (2,3), representing a 2x3 matrix, obtained by copying and expanding tensor data E) to obtain tensor data E'. The permute operator performs a dimension operation by swapping the two dimensions of the expanded tensor data E' to obtain tensor data E”. The slice operator performs a dimension operation by dividing the tensor data E” into 2x2 blocks as much as possible and taking the first block.

[0110] According to the embodiments disclosed herein, the expand operator can be moved to the end, which requires modifying the parameters of the permute and slice operators. Analysis shows that the expand operator only increases the size of one dimension (e.g., dim0) of the tensor data E, without increasing the dimension itself. Therefore, the parameters of the permute operator can remain unchanged, for example, still (1,0), indicating that dim0 and dim1 dimensions are swapped. Since the expand operator changes the size of the dim0 dimension, the parameters of the slice operator need to be adjusted. For example, for dimensions whose size has not changed, the original parameters can be maintained, while for dimensions whose size has changed, the parameters need to be reduced accordingly, for example, to half of the original (depending on the expansion factor of expand). That is, the dimension operation of the slice operator is modified to divide the tensor data output by the permute operator into 2×1 blocks as much as possible, and take the first data block. Correspondingly, the final expand operator also adjusts its parameters as needed, for example, adjusting the expanded dimension size to size3 = (2,2), thus ensuring that the dimension operation after adjustment is equivalent to the dimension operation before adjustment.

[0111] After the above processing, the merged view class operator subgraph can be returned.

[0112] As mentioned earlier, when tensor data becomes discontinuous in memory after passing through view-class operators, traditional CPUs and GPUs need to access and read this discontinuous data using the formula described earlier. This leads to low memory access efficiency and high time consumption in hardware devices. In neural network computing libraries, most operators require the input tensors to be contiguous in memory; otherwise, errors will occur. In this case, operators such as `contiguous()` need to be called. These operators also move the data one by one into contiguous storage according to the formula described above. This method of moving data one by one is very time-consuming, bringing a significant time overhead to the computation of the entire computation graph.

[0113] In this disclosed embodiment, after the view-type operator subgraph is constructed and fused and optimized, when encountering operators (such as computation-type operators) that require tensor data to be continuous in memory, memory data continuity processing can be performed based on these pre-constructed view-type operator subgraphs, and the corresponding kernel can be called to perform data transfer processing, thereby reducing data transfer time and improving computational efficiency.

[0114] Figure 7 An exemplary flowchart of a data processing method according to some embodiments of this disclosure is shown.

[0115] As shown in the figure, in step 710, in response to the fact that the tensor data to be processed is not contiguous in memory, the view class operator subgraph of the tensor data is obtained. The view class operator subgraph of the tensor data is constructed and optimized according to the method described above.

[0116] In some embodiments, the `is_contiguous` function can be used to determine whether the tensor data is contiguous in memory. If the tensor data is contiguous, no additional processing is required. If the tensor data is discontiguous, the view-class operator subgraph associated with that tensor data can be obtained.

[0117] It is understandable that if there is no view class operator subgraph associated with the tensor data, then the tensor data can only be made continuous by moving the data one by one in the existing way, such as calling the contiguous function.

[0118] Next, in step 720, based on the information of the obtained view class operator subgraph, the corresponding kernel is called to perform data transfer processing to convert the tensor data into tensor data that is continuous in memory.

[0119] Specifically, to avoid the time overhead of moving data one by one, we can analyze the operator types in the view class operator subgraph and call the kernel that matches the operator type to perform data moving processing. These kernels move data in blocks according to the operator type.

[0120] As mentioned in the operator fusion process above, the fused view-class operator subgraph can essentially only contain three types of view-class operators: permute, slice, and expand. For each view-class operator, a suitable kernel can be selected from a high-performance computing library to perform the corresponding data manipulation. This kernel can implement the functionality of the corresponding operator. For example, for the permute operator, the transpose kernel in a high-performance computing library (such as CNNL) can be called to implement the data rearrangement function. Similarly, for the expand operator, the expand kernel in CNNL can be called to implement the data expansion function.

[0121] Therefore, by traversing each view class operator and calling the kernel according to the order of the view class operator subgraph, the tensor data can be transformed from a discontinuous memory state to a contiguous memory state.

[0122] Compared to moving data one by one, calling the kernel to move data in blocks can greatly shorten processing time and improve memory access efficiency.

[0123] The foregoing description, in conjunction with the accompanying drawings, outlines a method for constructing a view-class operator subgraph, an optimization method for operator fusion, and a method for memory data continuity processing based on a view-class operator subgraph. This disclosure also provides a computing device that can be used to construct a view-class operator subgraph, optimize an operator subgraph, or perform memory data continuity processing.

[0124] Figure 8 A block diagram illustrating the hardware configuration of a computing device 800 that can implement various embodiments of the present disclosure is shown. As shown, the computing device 800 may include a processor 810 and a memory 820. Figure 8 In the computing device 800, only the components relevant to this embodiment are shown. Therefore, it will be apparent to those skilled in the art that the computing device 800 may also include components related to... Figure 8 The following are common components with different constituent elements, such as: display.

[0125] The computing device 800 can correspond to a computing device with various processing functions, such as a function for compiling computation graphs. For example, the computing device 800 can be implemented as various types of devices, such as personal computers (PCs), server devices, mobile devices, etc.

[0126] Processor 810 is configured to execute program instructions to control all functions of computing device 800. For example, processor 810 controls all functions of computing device 800 by executing programs stored in memory 820 on computing device 800. Processor 810 may be implemented by a central processing unit (CPU), graphics processing unit (GPU), application processor (AP), artificial intelligence processor chip (IPU), etc., provided in computing device 800. However, this disclosure is not limited thereto.

[0127] Memory 820 is hardware used to store various data processed in computing device 800. For example, memory 820 can store processed data and data to be processed in computing device 800. Memory 820 can store data that processor 810 has processed or is to process, such as computational graphs before compilation, computational graphs after compilation, etc. Furthermore, memory 820 can store program instructions such as applications and drivers to be driven by computing device 800. For example, memory 820 can store various programs related to optimization algorithms for computational graphs to be executed by processor 810. Memory 820 can be DRAM, but this disclosure is not limited thereto. Memory 820 can include at least one of volatile memory or non-volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, phase-change RAM (PRAM), magnetic RAM (MRAM), resistive RAM (RRAM), ferroelectric RAM (FRAM), etc. Volatile memory may include dynamic RAM (DRAM), static RAM (SRAM), synchronous DRAM (SDRAM), PRAM, MRAM, RRAM, ferroelectric RAM (FeRAM), etc. In an embodiment, memory 820 may include at least one of hard disk drive (HDD), solid-state drive (SSD), high-density flash memory (CF), secure digital card (SD), micro-secure digital card (Micro-SD), mini-secure digital card (Mini-SD), extreme digital card (xD), cache, or memory stick.

[0128] In summary, the specific functions implemented by the memory 820 and processor 810 of the computing device 800 provided in the embodiments of this specification can be explained in comparison with the foregoing embodiments in this specification, and can achieve the technical effects of the foregoing embodiments. Therefore, they will not be repeated here.

[0129] In this disclosure embodiment, a computer-readable storage medium is also provided, wherein program instructions are stored, which, when loaded and executed by a processor, cause the processor to perform the computation graph optimization method or data processing method described in this disclosure embodiment.

[0130] In this disclosure embodiment, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement an optimization method or data processing method for a computational graph as described in this disclosure embodiment.

[0131] Figure 9 This is a structural diagram illustrating a combined processing apparatus 900 according to an embodiment of the present disclosure. As shown, the combined processing apparatus 900 includes a computing device 902, an interface device 904, other processing devices 906, and a storage device 908. Depending on different application scenarios, the computing processing device may include one or more computing devices 910, which can be configured to... Figure 8 The computing device 800 shown is used to perform the operations described herein in conjunction with the accompanying drawings.

[0132] In different embodiments, the computing processing apparatus disclosed herein can be configured to perform user-specified operations. In exemplary applications, the computing processing apparatus can be implemented as a single-core artificial intelligence processor or a multi-core artificial intelligence processor. Similarly, one or more computing devices included within the computing processing apparatus can be implemented as an artificial intelligence processor core or a portion of the hardware structure of an artificial intelligence processor core. When multiple computing devices are implemented as artificial intelligence processor cores or portions of the hardware structure of artificial intelligence processor cores, the computing processing apparatus disclosed herein can be considered to have a single-core structure or a homogeneous multi-core structure.

[0133] In exemplary operation, the computing processing device disclosed herein can interact with other processing devices through an interface device to jointly complete user-specified operations. Depending on the implementation, the other processing devices disclosed herein may include one or more types of processors such as a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and an artificial intelligence processor, both general-purpose and / or special-purpose processors. These processors may include, but are not limited to, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, the computing processing device disclosed herein can be considered to have a single-core structure or a homogeneous multi-core structure. However, when the computing processing device and other processing devices are considered together, they can be considered to form a heterogeneous multi-core structure.

[0134] In one or more embodiments, the other processing device may serve as an interface between the computing processing device disclosed herein (which may be specifically embodied in artificial intelligence, such as neural network operations) and external data and control, performing basic controls including but not limited to data transfer, starting and / or stopping the computing device. In another embodiment, the other processing device may also cooperate with the computing processing device to jointly complete computational tasks.

[0135] In one or more embodiments, the interface device can be used to transfer data and control commands between a computing processing device and other processing devices. For example, the computing processing device can obtain input data from other processing devices via the interface device and write it to on-chip storage (or memory) of the computing processing device. Further, the computing processing device can obtain control commands from other processing devices via the interface device and write them to on-chip control cache of the computing processing device. Alternatively or optionally, the interface device can also read data from the storage device of the computing processing device and transmit it to other processing devices.

[0136] Additionally or optionally, the combined processing apparatus disclosed herein may further include a storage device. As shown in the figures, the storage device is connected to both the computing processing device and the other processing device. In one or more embodiments, the storage device may be used to store data from the computing processing device and / or the other processing device. For example, the data may be data that cannot be fully stored in the internal or on-chip storage of the computing processing device or other processing device.

[0137] In some embodiments, this disclosure also discloses a chip (e.g. Figure 10 The chip shown is 1002. In one implementation, the chip is a system-on-a-chip (SoC) and integrates one or more such... Figure 9 The combined processing unit shown is illustrated. This chip can be connected to external interface devices (such as...). Figure 10 The external interface device 1006 shown is connected to other related components. These related components may be, for example, a camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. In some applications, the chip may integrate other processing units (e.g., video codecs) and / or interface modules (e.g., DRAM interfaces). In some embodiments, this disclosure also discloses a chip package structure that includes the aforementioned chip. In some embodiments, this disclosure also discloses a board that includes the aforementioned chip package structure. The following will be combined with... Figure 10 This board is described in detail.

[0138] Figure 10 This is a schematic diagram illustrating the structure of a board 1000 according to an embodiment of this disclosure. As shown, the board includes a storage device 1004 for storing data, which includes one or more storage cells 1010. This storage device can be connected and transmit data with a controller 1008 and the aforementioned chip 1002 via, for example, a bus. Furthermore, the board also includes an external interface device 1006, configured for data relay or switching between the chip (or a chip in a chip package structure) and an external device 1012 (e.g., a server or computer). For example, data to be processed can be transmitted from the external device to the chip via the external interface device. Alternatively, the calculation results of the chip can be transmitted back to the external device via the external interface device. Depending on the application scenario, the external interface device can have different interface forms; for example, it can adopt a standard PCIe interface.

[0139] In one or more embodiments, the controller in the disclosed board can be configured to regulate the state of the chip. Therefore, in one application scenario, the controller may include a microcontroller (MCU) for regulating the operating state of the chip.

[0140] Based on the above combination Figure 9 and Figure 10 Based on the description, those skilled in the art will understand that this disclosure also discloses an electronic device or apparatus that may include one or more of the aforementioned boards, one or more of the aforementioned chips, and / or one or more of the aforementioned combined processing apparatus.

[0141] Depending on the application scenario, the electronic devices or apparatus disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablets, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, dashcams, navigators, sensors, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, autonomous driving terminals, vehicles, home appliances, and / or medical devices. The vehicles include airplanes, ships, and / or vehicles; the home appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical devices include MRI scanners, ultrasound machines, and / or electrocardiographs. The electronic devices or apparatus disclosed herein can also be applied in fields such as the Internet, IoT, data centers, energy, transportation, public management, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and healthcare. Furthermore, the electronic devices or apparatus disclosed herein can also be used in application scenarios related to artificial intelligence, big data, and / or cloud computing, such as cloud computing, edge computing, and terminal applications. In one or more embodiments, the high-computing-power electronic devices or apparatuses according to the present disclosure can be applied to cloud devices (e.g., cloud servers), while the low-power electronic devices or apparatuses can be applied to terminal devices and / or edge devices (e.g., smartphones or cameras). In one or more embodiments, the hardware information of the cloud devices and the hardware information of the terminal devices and / or edge devices are compatible with each other, so that suitable hardware resources can be matched from the hardware resources of the cloud devices to simulate the hardware resources of the terminal devices and / or edge devices based on the hardware information of the terminal devices and / or edge devices, so as to complete the unified management, scheduling and collaborative work of end-to-cloud or cloud-edge-end integration.

[0142] It should be noted that, for the sake of brevity, this disclosure describes some methods and their embodiments as a series of actions and combinations thereof. However, those skilled in the art will understand that the solutions disclosed herein are not limited by the order of the described actions. Therefore, based on the disclosure or teachings of this document, those skilled in the art will understand that some steps can be performed in a different order or simultaneously. Furthermore, those skilled in the art will understand that the embodiments described in this disclosure can be considered optional embodiments, that is, the actions or modules involved are not necessarily essential for the implementation of one or more solutions disclosed herein. In addition, depending on the solution, the description of some embodiments in this disclosure may have different emphases. In view of this, those skilled in the art will understand that parts not described in detail in a certain embodiment of this disclosure can also be referred to the relevant descriptions of other embodiments.

[0143] In terms of specific implementation, based on the disclosure and teachings of this document, those skilled in the art will understand that several embodiments disclosed herein can also be implemented in other ways not disclosed herein. For example, regarding the various units in the electronic device or apparatus embodiments described above, this document divides them based on logical functions, but in actual implementation, there may be other division methods. As another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. Regarding the connection relationships between different units or components, the connections discussed above in conjunction with the accompanying drawings can be direct or indirect couplings between units or components. In some scenarios, the aforementioned direct or indirect couplings involve communication connections utilizing interfaces, where the communication interface can support electrical, optical, acoustic, magnetic, or other forms of signal transmission.

[0144] In this disclosure, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network units. Furthermore, depending on actual needs, some or all of the units can be selected to achieve the purpose of the solution described in the embodiments of this disclosure. Additionally, in some scenarios, multiple units in the embodiments of this disclosure may be integrated into one unit or each unit may exist physically independently.

[0145] In other implementation scenarios, the integrated units described above can also be implemented in hardware, i.e., as specific hardware circuits, which may include digital circuits and / or analog circuits. The physical implementation of the circuit's hardware structure may include, but is not limited to, physical devices, which may include, but are not limited to, transistors or memristors. Therefore, the various devices described herein (e.g., computing devices or other processing devices) can be implemented using appropriate hardware processors, such as CPUs, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage units or storage devices can be any suitable storage medium (including magnetic storage media or magneto-optical storage media), such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), ROM, and RAM.

[0146] The foregoing can be better understood in accordance with the following terms:

[0147] Clause 1. An optimization method for a computational graph, comprising:

[0148] Obtain the view class operator subgraph of the tensor data in the computation graph, wherein the view class operator subgraph includes the source operators of the view class associated with the tensor data;

[0149] Based on the functionality of the source operators in the view class operator subgraph, replace them with specified target operators whose functions are mutually substitutable; and

[0150] Multiple consecutive identical target operators are merged into a single target operator to generate a fused view-class operator subgraph.

[0151] Clause 2. The method according to Clause 1, wherein fusing multiple consecutive identical objective operators into a single objective operator includes:

[0152] The dimensional operations of the multiple target operators are merged so that the merged single target operator is equivalent to the multiple target operators before fusion.

[0153] Clause 3. The method described under any of Clauses 1-2 further includes:

[0154] After the fusion is performed, the position of a specific type of target operator is adjusted for post-processing. The specific type of target operator is an extended class operator that causes an increase in memory data.

[0155] Clause 4. The method described in Clause 3, wherein adjusting the position of a specific type of target operator for post-processing includes:

[0156] Based on the position of the target operator before and after adjustment according to the specific type of target operator, modify the parameters of the target operator in the view class operator subgraph that is between the two to adapt to the adjustment.

[0157] Clause 5. According to any of the methods described in Clauses 1-4, the functions of the source operators are divided into three categories based on their impact on the size of the memory data: size reduction, size expansion, and size invariance.

[0158] Clause 6. According to the method described in Clause 5, the target operators corresponding to the three types of functions of scaling down, scaling up, and scaling remaining unchanged are the slice operator, the expand operator, and the permute operator, respectively.

[0159] Clause 7. The method described under any one of Clauses 1-6 further includes:

[0160] Based on the information of the fused view class operator subgraph, the corresponding kernel is invoked to perform data transfer processing, so as to convert the tensor data into tensor data that is continuous in memory.

[0161] Clause 8. A computing device for optimizing a computational graph, comprising:

[0162] A processor, configured to execute program instructions; and

[0163] A memory configured to store the program instructions, which, when loaded and executed by the processor, cause the processor to perform an optimization method of the computation graph according to any one of clauses 1-7.

[0164] Clause 9. A computer-readable storage medium storing program instructions that, when loaded and executed by a processor, cause the processor to perform an optimization method of a computation graph according to any one of Clauses 1-7.

[0165] Clause 10. A computer program product comprising a computer program or instructions that, when executed by a processor, implement the optimization method of the computation graph described in any one of Clauses 1-7.

[0166] While numerous embodiments of this disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and intent of this disclosure. It should be understood that various alternatives to the embodiments of this disclosure described herein may be employed in the practice of this disclosure. The appended claims are intended to define the scope of this disclosure and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A method for optimizing a computational graph, comprising: Obtain the view class operator subgraph of the tensor data in the computation graph, wherein the view class operator subgraph includes source operators of the view class associated with the tensor data, and the view class operators do not change the actual arrangement of the tensor data in memory; The source operators in the view class operator subgraph are classified according to their functions, and each category is assigned a target operator to represent the function of that category. The source operators are replaced with the target operators whose functions can be mutually substituted. The functions of the source operators are divided into three categories based on their impact on the size of the memory data: size reduction, size expansion, and size unchanged. as well as Multiple consecutive identical target operators are merged into a single target operator to generate a fused view-class operator subgraph.

2. The method according to claim 1, wherein fusing multiple consecutive identical target operators into a single target operator comprises: The dimensional operations of the multiple target operators are merged so that the merged single target operator is equivalent to the multiple target operators before fusion.

3. The method according to claim 1, further comprising: After the fusion is performed, the position of a specific type of target operator is adjusted for post-processing. The specific type of target operator is an extended class operator that causes an increase in memory data.

4. The method of claim 3, wherein adjusting the position of the specific type of target operator for post-processing includes: Based on the position of the target operator before and after adjustment according to the specific type of target operator, modify the parameters of the target operator in the view class operator subgraph that is between the two to adapt to the adjustment.

5. The method according to claim 1, wherein the target operators corresponding to the three types of functions of scaling down, scaling up, and scaling remaining unchanged are the slice operator, the expand operator, and the permute operator, respectively.

6. The method according to any one of claims 1-5, further comprising: Based on the information of the fused view class operator subgraph, the corresponding kernel is invoked to perform data transfer processing, so as to convert the tensor data into tensor data that is continuous in memory.

7. A computing device for optimizing a computational graph, comprising: A processor, configured to execute program instructions; as well as A memory configured to store the program instructions, which, when loaded and executed by the processor, cause the processor to perform the optimization method for the computation graph according to any one of claims 1-6.

8. A computer-readable storage medium storing program instructions that, when loaded and executed by a processor, cause the processor to perform an optimization method for a computational graph according to any one of claims 1-6.

9. A computer program product comprising a computer program or instructions which, when executed by a processor, implement the optimization method of the computation graph according to any one of claims 1-6.

Images