Calculation graph optimization method, calculation device and related product

Abstract

The invention discloses a calculation graph optimization method, a calculation device, a computer readable storage medium and a computer program product. A computing device that performs an optimization method of a computational graph may be included in a combination processing device that may further include an interface device and other processing devices. The computing device interacts with other processing devices to jointly complete computing operation specified by a user. The combined processing device can further comprise a storage device, and the storage device is connected with the computing device and the other processing devices and used for storing data of the computing device and the other processing devices. According to the scheme disclosed by the invention, by optimizing the view-class operator subgraph, the carrying of a memory at an equipment end can be reduced, so that the processing efficiency of a machine is improved.

Description

Technical Field

[0001] The present disclosure generally relates to the field of intelligent computing technology, and more particularly to the field of compilation. More specifically, the present disclosure relates to a method for optimizing a computational graph, a computing device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In intelligent computing systems, programming frameworks provide programmers with an interface for using hardware and systems, and are a critical core hub in intelligent computing systems. TensorFlow, PyTorch, etc. are currently popular deep learning frameworks. In these programming frameworks, computational graphs are usually used to describe the computational process of machine learning algorithms, tensors are used to represent all data in the computational graphs, and operators are used to represent various operations.

[0003] There is a class of operators, such as slicing operators that slice memory data (such as narrow, select, unbind, chunk, slice, and split), transposition operators that swap data dimensions (such as permute, transpose, T, etc.), and deformation operators that change the size and value of dimensions (such as unsqueeze, squeeze, view, view_as, reshape, reshape_as, etc.), which only change the appearance or appearance of tensor data, but do not change the actual arrangement of tensor data in memory, that is, they will not carry out real memory data transfer, but will change the addressing properties of tensor memory data, such as size, stride, and storage_offset properties. This type of operator can be called a view operator.

[0004] Due to this characteristic of view operators, tensor data is usually discontinuous in memory. When reading discontinuous data, it will cause problems such as low memory access efficiency and high time consumption of hardware devices. Through the implementation of view operator subgraphs, the device-side IO operators can be called in sequence according to the recorded information to change discontinuous memory data into continuous memory data, thereby improving the memory access efficiency of the device. However, when there are deformation operators in the view operator subgraph, and the deformation operators are interspersed among other types of operators, it will lead to problems such as multiple operator calls or difficulty in optimizing the view operator subgraph, thereby affecting the memory access efficiency and network time consumption of the hardware device. Summary of the invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present disclosure proposes a computational graph optimization method, a computing device, a computer-readable storage medium, and a computer program product in multiple aspects.

[0006] In a first aspect, the present disclosure provides a method for optimizing a computational graph, comprising: obtaining a view-class operator subgraph of tensor data in the computational graph; determining whether there is an operator sequence in a specific order in the view-class operator subgraph, wherein the specific order includes that a deformation-class operator is located before a slice-class operator, and there are other types of operators before the deformation-class operator except the deformation-class operator; and in response to the existence of the operator sequence in the specific order, adjusting the position of the slice-class operator and / or the deformation-class operator in the operator sequence so that the slice-class operator is pre-processed.

[0007] In a second aspect, the present disclosure provides a computing device for optimizing a computational graph, comprising: a processor configured to execute program instructions; and a memory configured to store the program instructions, wherein when the program instructions are loaded and executed by the processor, the processor executes the computational graph optimization method described in the first aspect of the present disclosure.

[0008] In a third aspect, the present disclosure provides a computer-readable storage medium having computer-readable instructions stored thereon, which, when executed by one or more processors, implement the computational graph optimization method as described in the first aspect of the present disclosure.

[0009] In a fourth aspect, the present disclosure provides a computer program product, comprising a computer program or instructions, which, when executed by a processor, implements the computational graph optimization method described in the first aspect of the present disclosure.

[0010] According to the calculation graph optimization method provided above, when there is an operator sequence in which a deformation operator is located between slice operators and other types of operators in the view operator subgraph, the slice operator can be pre-processed, thereby facilitating further optimization of the deformation operator and other types of operators, thereby achieving the purposes of reducing memory data transfer, improving data memory access efficiency, and reducing network latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] By reading the detailed description below with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0012] Figure 1 The different shapes of multidimensional arrays and their storage order in memory are shown by way of example;

[0013] Figure 2 An exemplary flow chart of a method for optimizing a computational graph according to an embodiment of the present disclosure is shown;

[0014] Figure 3 A flow chart of a method for optimizing a computational graph according to another embodiment of the present disclosure is shown;

[0015] Figure 4 An example of determining a dimension mapping relationship according to an embodiment of the present disclosure is shown;

[0016] Figure 5a-5c An exemplary flow chart of an optimization method including operator fusion according to multiple embodiments of the present disclosure is shown;

[0017] Figure 6 An example of multiple slice-like target operators operating on the same dimension is shown;

[0018] Figure 7 An example diagram of an optimization process in which a transposition operator exists before a deformation operator in an operator sequence according to an embodiment of the present disclosure is shown;

[0019] Figure 8 A block diagram showing a hardware configuration of a computing device 800 that may implement embodiments of the present disclosure;

[0020] Fig. 9 is a structural diagram showing a combined processing device 900 according to an embodiment of the present disclosure;

[0021] Fig.10 is a schematic diagram showing the structure of a board 1000 according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0023] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0024] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0025] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0026] The specific implementation of the present disclosure is described in detail below with reference to the accompanying drawings.

[0027] In the programming framework of intelligent computing systems, data is usually modeled as tensors. A tensor can be viewed as an N-dimensional array, and the dimension of the array is the order of the tensor. Therefore, a 0-order tensor corresponds to scalar data; a 1-order tensor corresponds to a one-dimensional array, that is, a vector; a 2-order tensor corresponds to a two-dimensional array, that is, a matrix; and so on, an N-order tensor corresponds to an N-dimensional array. For example, an RGB image can be represented as a 3-order tensor, and a data set consisting of multiple RGB images can be represented as a 4-order tensor.

[0028] Each tensor has some common properties, including data type, shape, etc. The shape of a tensor indicates the length of each order of the tensor. For example, a 0-order tensor corresponds to a scalar data, and its shape is empty; a 1-order tensor corresponds to a one-dimensional vector, and its shape contains one element, and the element value is the length of the vector; a 2-order tensor corresponds to a matrix, and its shape contains two elements, corresponding to the length of the row and column respectively; a 3-order tensor corresponds to a three-dimensional data, and its shape contains three elements, corresponding to the length of each order.

[0029] Although multidimensional arrays have multiple dimensions, because the layout of memory (for example, memory DRAM and cache RAM) is always one-dimensional, there is a corresponding relationship between multidimensional arrays and the storage order on the memory. Multidimensional arrays are usually allocated in continuous storage space, that is, multidimensional arrays can be expanded one-dimensionally and stored in order on the memory.

[0030] Figure 1Different shapes of multi-dimensional arrays and their storage order in the memory are exemplified, wherein a one-dimensional array of a continuous memory is used to implement the storage of the multi-dimensional array.

[0031] Figure 1 Figure (a) shows the first data, that is, the three-dimensional array X, which has three dimensions, namely dimension 0 (dim0), dimension 1 (dim1) and dimension 2 (dim2). The size of dimension 0 is 2, the size of dimension 1 is 2, and the size of dimension 2 is 3. Therefore, its shape (size) can be expressed as: X 3 =(2,2,3).

[0032] Figure 1 Figure (c) shows the storage order of the three-dimensional array X in the memory. In the figure, data with the same background are located in the same dimension. When storing, assuming that the storage is in the order of low dimension priority (for example, from left to right in shape representation corresponds to high dimension to low dimension), the first data is expanded in one dimension, and the following can be obtained:

[0033] X=[1,2,3,4,5,6,7,8,9,10,11,12].

[0034] More specifically, the data in the lowest dimension (the same row) is continuous, while the data in higher dimensions are spaced at different distances. For example, in the storage method shown in Figure (c), accessing the physical structure of adjacent elements on dimension dim2 requires a shift of 1 position (for example, from data 1 to data 2, data 5 to data 6, and so on); accessing the physical structure of adjacent elements on dimension dim1 requires a shift of 3 positions (for example, from data 1 to data 4, data 2 to data 5, ..., data 9 to data 12, and so on); and accessing the physical structure of adjacent elements on dimension dim0 requires a shift of 6 positions (for example, from data 1 to data 7, data 2 to data 8, ..., data 6 to data 12, and so on). This offset is called the stride. The stride of each dimension of the three-dimensional array X can be expressed as S X =(6,3,1).

[0035] In the programming framework of intelligent computing systems, there are view operators that operate on the external representation of tensors, such as transpose, slice, split, etc. Taking transpose as an example, it converts a dimension according to a certain perm N =(p 1 , p 2 ,…,p i ,…,p N ), and get the data arrangement after dimension conversion, where p i The value of (i∈1,2,…,N) represents the original dimension of the array, [i In perm N The position in represents the target dimension of the transformation. For example, given the dimension transformation rule perm 3 =(0,2,1), which means that dimension 1 and dimension 2 are to be swapped, that is, the original dimension 1 is to be converted into dimension 2 of the new array, and the original dimension 2 is to be converted into dimension 1 of the new array.

[0036] Figure 1 Figure (b) shows the transformed array Y obtained by performing the transpose operator on the three-dimensional array X shown in Figure (a). In this example, the above exemplary dimension conversion rule perm 3 =(0,2,1). As can be seen from the figure, compared with array X, the dimension 1 and dimension 2 of array Y are swapped. At this time, the dimension information of the three-dimensional array Y can be expressed as: 3 =(2,3,2).

[0037] However, since the view operator does not change the storage location of the data in memory, the storage order of the array Y obtained after the transposition operation in memory remains the same. Figure 1 As shown in (c) in the figure. At this time, according to the storage order in (c), the step size of each dimension of array Y becomes S Y =(6,1,3). It can be seen that if the order of storing each data in the principle of low dimensionality first is called continuity, then the current storage order of array Y is discontinuous. That is, after the transpose operator, the order of the array dimensions is changed, but the storage position on the memory is not changed, so the storage order of the array on the memory becomes discontinuous.

[0038] If you want array Y to be continuous in memory, then according to the principle of low dimension first, its one-dimensional expansion should be Figure 1 As shown in Figure (d):

[0039] Y=[1,4,2,5,3,6,7,10,8,11,9,12].

[0040] The shape of a tensor can help programmers form an intuitive feeling about tensors. In programming frameworks such as Pytorch, the View class operator can change the shape (size), stride (the span of the first index between adjacent dimensions of a tensor), storage offset (storage_offset, the offset of the first element of a tensor relative to the storage start position) and other properties of a tensor, but does not change the actual storage location of the tensor. At this time, the tensor uses size, stride and storage_offset to calculate the memory location of the data on the device side.

[0041] Assume that the size of the tensor is (s 0 ,s 1 ,s 2 ,…,s i ), stride is (y 0 ,y 1 ,y 2 ,…,y i ), storage_offset is b, then the tensor at point (x 0 ,x 1 ,x 2 ,…,x i ) is calculated as follows:

[0042]

[0043] Where dptr is the starting position of the memory storage corresponding to the tensor, and dtype is the data type of the tensor.

[0044] Combine from the front Figure 1 As can be seen from the description, the tensors in the computational graph generate discontinuous corresponding data after being processed by the view operator. When accessing these tensors, traditional CPUs and GPUs need to access and read discontinuous data according to the above formula, which will lead to problems such as low memory access efficiency and high time consumption of hardware devices. Another way is to call the contiguous() operator to move the data one by one into continuous storage according to the above formula before performing subsequent access and calculation. However, this method is very time-consuming. When the amount of data is large, moving the data one by one will consume a lot of time.

[0045] In view of this, considering that in the operation of computational graphs such as neural networks, the discontinuity of memory data is often caused by view-type operators, the present disclosure proposes a solution for optimizing the view-type operator subgraph in the computational graph. The optimized view-type operator subgraph can then support the subsequent efficient completion of the process of transferring memory data from discontinuous to continuous.

[0046] Regarding the terms "node" and "operator" mentioned in this disclosure, it should be noted that the term "operator" is from the computing level of the computer (or from the software level or algorithm level); and the term "node" is a more figurative term (from the graphical level or a more intuitive level). In terms of the content referred to, the terms "operator" and "node" actually refer to the same thing. That is, in this disclosure, it can be considered that the terms "operator" and "node" have the same meaning and can be used interchangeably, but are described from different aspects.

[0047] As mentioned earlier, when tensor data becomes discontinuous in memory after passing through view operators, traditional CPUs and GPUs need to access and read discontinuous data, which leads to low memory access efficiency and high time consumption of hardware devices. In the neural network computing library, most operators require the input tensors to be continuous in memory, otherwise an error will occur. In this case, it is necessary to call operators such as contiguous(), which move data one by one into continuous storage. This method of moving data one by one is very time-consuming and brings great time overhead to the calculation of the entire computational graph.

[0048] The inventors have found that by constructing a view-class operator subgraph, when encountering operators (such as computing operators) that require tensor data to be continuous in memory in the future, memory data continuity processing can be performed based on these pre-constructed view-class operator subgraphs, and the corresponding device-side IO operator (such as kernel) can be called to perform data handling processing, and discontinuous memory data can be changed into continuous memory data, thereby improving the memory access efficiency of subsequent computing operators. However, the slice operator itself has the characteristic of reducing the amount of memory data. When there is an operator sequence in the view-class operator subgraph where the deformation operator is located before the slice operator, and there are other types of operators before the deformation operator, in the case of continuous operator calls, not only will more IO data handling be generated, but the slice operator will also be unable to be optimized with other types of operators except the deformation operator, which will lead to problems such as reduced memory access efficiency of hardware devices and increased network latency.

[0049] Figure 2 An exemplary flow chart of a method for optimizing a computational graph according to an embodiment of the present disclosure is shown. Figure 2 As shown in , the optimization method 200 may include: in step 202, a view class operator subgraph of tensor data in the calculation graph may be obtained. The calculation graph is a directed graph including nodes and edges, and tensors are passed between nodes of the calculation graph. The execution of the calculation graph follows the order of the directed graph. Each time a tensor passes through a node, it will be calculated as the input of the node operation, and the result of the calculation will flow to the subsequent node along the output edge of the node. Therefore, obtaining the view class operator subgraph of tensor data in the calculation graph can be for tensor data, according to the order of the directed graph, traversing the nodes or operators to be processed on the tensor data, and when the traversed operator is a view class operator, extracting the view class operator subgraph constructed by the view class operator. It can be understood that for each tensor data, one or more segments of view class operator subgraphs can be included according to the continuity of the view class operator in the calculation graph. The optimization method of the disclosed embodiment can optimize the one or more segments of the view class operator subgraph.

[0050] Next, in step 204, it can be determined whether there is a sequence of operators in a specific order in the view class operator subgraph, wherein the specific order may include that the deformation class operator is located before the slice class operator, and there are other types of operators before the deformation class operator except the deformation class operator. In programming frameworks such as Pytorch, there are various view class operators to achieve different functions. The transposition class operator does not change the scale of the tensor data. Its main function is to exchange the data dimension and does not change the position of the memory block data. It may include operators such as permute, transpose, t(), T, etc. The slice class operator will reduce the scale of the tensor data. Its main function is to slice the memory data. It may include operators such as narrow, select, unbind, chunk, slice, and split. Deformation operators do not change the scale of tensor data. Their main function is to change the dimension size (or the number of dimensions) and the dimension value (or the dimension size) without changing the location of memory data. They may include operators such as unsqueeze, squeeze, view, view_as, reshape, and reshape_as.

[0051] In some embodiments, the deformation type operator is located before the slice type operator, which may be one deformation type operator before one slice type operator, or multiple deformation type operators before one slice type operator, or one deformation type operator before multiple slice type operators, or multiple deformation type operators before multiple slice type operators. In other embodiments, when multiple deformation type operators are located before a slice type operator, the multiple deformation type operators may be the same operator or different operators whose functions can replace each other; when there are multiple slice type operators after a deformation type operator, the multiple slice type operators may be the same operator or different operators whose functions can replace each other.

[0052] In other embodiments, when there is a deformation type operator located before a slice type operator or before multiple consecutive slice type operators, and there are other types of operators except the deformation type operator before the deformation type operator, the operator sequence of the specific order described in the disclosed embodiment is formed. In some other embodiments, when there are multiple consecutive deformation type operators located before the slice type operator, and there are other types of operators except the deformation type operator before the multiple consecutive deformation type operators, the operator sequence of the specific order described in the disclosed embodiment is formed. In some embodiments, other types of operators except the deformation type operator may include, for example, at least one of a slice type operator, a transposition type operator, and an expansion type operator. The main function of the expansion type operator is to expand the scale of data, and may include operators such as expand.

[0053] For example, in the view class operator subgraph, there may be an operator sequence of permute+view+slice, an operator sequence of select+transpose+squeeze+narrow, an operator sequence of slice+expand+reshape+select, or an operator sequence of slice+reshape+slice, etc. These exemplary operator sequences may be operator sequences of a specific order in the embodiments of the present disclosure, but are not limited thereto.

[0054] For the view class operator subgraph, the view class nodes or operators to be processed on the tensor data are traversed in the order of the directed graph to determine whether there is an operator sequence of a specific order in the view class operator subgraph. In other embodiments, the operator sequence of a specific order may include a continuous sequence of deformation class operators and slice class operators. The continuous sequence here can be understood as the absence of other types of operators between the deformation class operator and the slice class operator. Exemplarily, when there are multiple continuous deformation class operators located before the slice class operator, there are no other types of operators between the last deformation class operator of the multiple deformation class operators and the slice class operator, and such an operator sequence is an operator sequence of a specific order. Similarly, when there is a deformation class operator located before multiple continuous slice class operators, there are no other types of operators between the first slice class operator of the multiple slice class operators and the deformation class operator, and such an operator sequence is an operator sequence of a specific order.

[0055] like Figure 2As further shown in, in step 206, in response to the existence of an operator sequence in a specific order, the position of the slice-type operator and / or the deformation-type operator in the operator sequence is adjusted so that the slice-type operator is processed in advance. In some embodiments, adjusting the position of the slice-type operator and / or the deformation-type operator in the operator sequence may include: moving the slice-type operator before the deformation-type operator; and / or moving the deformation-type operator after the slice-type operator. By adjusting the front-to-back position relationship between the slice-type operator and the deformation-type operator, the slice-type operator can process the tensor data before the deformation-type operator. In other embodiments, when adjusting the position of the slice-type operator and / or the deformation-type operator in the operator sequence, the parameters of the slice-type operator and / or the deformation-type operator also need to be modified to adapt to the position adjustment.

[0056] Combination of the above Figure 2 An exemplary description is given of the optimization method of the computational graph according to the embodiment of the present disclosure. It can be understood that, according to the specific order of the operator sequence of the embodiment of the present disclosure, the position of the operators in the view subgraph is optimized by adjusting the positions of the slice operators and / or deformation operators in the operator sequence, so that the slice operators are pre-processed to facilitate the optimization and expansion of more view subgraphs.

[0057] For example, in some embodiments, assuming that there is a slice operator before the deformation operator in the operator sequence, adjusting the position of the slice operator and / or the deformation operator in the operator sequence will be beneficial to form multiple continuous slice operators before the deformation operator, so that multiple slice operators can be fused to reduce the number of operator calls in subsequent calculations, which is beneficial to reduce the amount of memory data carried on the device side and improve memory access efficiency.

[0058] For example, in some other embodiments, assuming that there is a transposition operator or an extension operator before the deformation operator, since the slicing processing performed by the slicing operator can naturally reduce the amount of data, pre-processing the slicing operator can facilitate further optimization between the subsequent slicing operator and the transposition operator or the extension operator, such as moving the slicing operator before the transposition operator, etc., which will help reduce the amount of memory data moved and the amount of calculation during subsequent continuity processing, and thus help improve the memory access efficiency of hardware devices.

[0059] In order to further understand the implementation method of adjusting the position of the slicing operator and / or the deformation operator in the embodiment of the present disclosure, the following will be combined with Figure 3 Provide further explanation.

[0060] Figure 3 A flowchart of a method for optimizing a computational graph according to another embodiment of the present disclosure is shown. As can be seen from the following description, Figure 3 The optimization method 300 shown can be combined with Figure 2 The optimization method 200 described above is a manifestation of Figure 2 The description of the optimization method 200 can also be applied to the following description of the optimization method 300. Figure 3 Provide explanation.

[0061] like Figure 3 As shown in , the optimization method 300 may include: in step 302, a view class operator subgraph of tensor data in the computation graph may be obtained; in step 304, it may be determined whether there is a sequence of operators in a specific order in the view class operator subgraph, wherein the specific order may include that the deformation class operator is located before the slice class operator, and there are other types of operators before the deformation class operator except the deformation class operator. Step 302 and step 304 are combined with the above Figure 2 The described steps 202 and 204 are the same or similar and will not be described again here.

[0062] Next, in step 306, in response to the existence of a sequence of operators in a specific order, the dimension mapping relationship between the input tensor and the output tensor of the deformation operator can be determined according to the relationship between the dimension values ​​of the input tensor and the dimension values ​​of the output tensor of the deformation operator, where the dimension value represents the number of data (or size) in each dimension. Figure 4 An exemplary description is given.

[0063] Figure 4 An example of determining a dimension mapping relationship according to an embodiment of the present disclosure is shown. Figure 4 As shown in, in an example, assume that the dimensional values ​​of the input tensor X1 of the deformation operator include [4, 6, 5, 1, 7, 1], where the position of each value represents the dimension, and the size of each value represents the number of data in the dimension, that is, [4, 6, 5, 1, 7, 1] respectively represent the number of data in the dimensions [dim0, dim1, dim2, dim3, dim4, dim5], [4] means that the dim0 dimension includes 4 data, [6] means that the dim1 dimension includes 6 data, and so on, which will not be repeated here.

[0064] The parameters of the deformation operator can include the dimension information of the input tensor and the dimension information of the output tensor. The dimension information of the output tensor is generated according to the deformation rule of the deformation operator. The deformation operator needs to meet the basic characteristics of Pytorch native safe_view, that is, the characteristic of not changing the actual data memory location. According to the deformation rule of the deformation operator, the dimension value of the output tensor Y1 can be obtained, such as [2, 2, 5, 6, 1, 7, 1] in the figure, where the position of each value represents the dimension, and the size of each value represents the number of data in the dimension, that is, [2, 2, 5, 6, 1, 7, 1] respectively represents the number of data in the dimensions [dim0, dim1, dim2, dim3, dim4, dim5, dim6], [2, 2] respectively represents 2 data in the dim0 dimension and 2 data in the dim1 dimension, [5] represents 5 data in the dim2 dimension, and so on, which will not be repeated here.

[0065] According to the correspondence between the input tensor X1 dimension value and the output tensor Y1 dimension value of the deformation operator, the dimension mapping relationship between the input tensor and the output tensor of the deformation operator can be determined. In some embodiments, in response to the existence of the same dimension value in the input tensor dimension value and the output tensor dimension value, the dimensions to which the same dimension value belongs are mapped in pairs; in response to the existence of the same dimension value in the input tensor dimension value and the output tensor dimension value, the dimensions to which the product of the same dimension value belongs are mapped in association.

[0066] by Figure 4 Taking the example in as an example, the dimension values ​​[1, 7, 1] of the input tensor X1 correspond to the dimension values ​​[1, 7, 1] of the output tensor Y1, and the dimensions to which the corresponding dimension values ​​[1, 7, 1] belong are mapped in pairs (indicated by solid arrows in the figure), that is, the dim3 dimension of the input tensor X1 is mapped in pairs with the dim4 dimension of the output tensor Y1, the dim4 dimension of the input tensor X1 is mapped in pairs with the dim5 dimension of the output tensor Y1, and the dim5 dimension of the input tensor X1 is mapped in pairs with the dim6 dimension of the output tensor Y1.

[0067] In addition, the values ​​of [4, 6, 5] in the input tensor X1 dimension are not the same as the values ​​of [2, 2, 5, 6] in the output tensor Y1 dimension. However, the product of [2, 2] in the output tensor Y1 dimension is the same as [4] in the input tensor X1 dimension, and the product of [6, 5] in the output tensor Y1 dimension is the same as the product of [5, 6] in the input tensor X1 dimension. Therefore, the dim0 dimension of the input tensor X1 is associated with the dim0 and dim1 dimensions of the output tensor Y1, and the dim1 and dim2 dimensions of the input tensor X1 are associated with the dim2 and dim3 dimensions of the output tensor Y1. An associated mapping indicates that there is an association between the mapped dimensions.

[0068] After determining the dimension mapping relationship, the following returns Figure 3 Continue to describe. Figure 3 As further shown in , the optimization method 300 may also include step 308, which can determine whether to adjust the position of the slice type operator and / or the deformation type operator according to the dimension mapping relationship corresponding to the slice dimension of the slice type operator, so that the slice type operator is pre-processed. Since in a specific order of operator sequences, the slice type operator is operated after the deformation type operator, the input tensor of the slice type operator is the output tensor of the deformation type operator, and the slice dimension of the slice type operator corresponds to the dimension in the output tensor of the deformation type operator, the dimension of the input tensor of the corresponding deformation type operator can be determined according to the dimension in the output tensor into which the slice dimension falls, that is, the dimension mapping relationship corresponding to the slice dimension of the slice type operator.

[0069] by Figure 4 For example, assuming that the slice dimension of the slice operator corresponds to dim0 or dim1 in the output tensor Y1, the dimension mapping relationship corresponding to the slice dimension of the slice operator is an associative mapping. Assuming that the slice dimension of the slice operator corresponds to dim2 or dim3 in the output tensor Y1, the dimension mapping relationship corresponding to the slice dimension of the slice operator is also an associative mapping. If the slice dimension of the slice operator corresponds to dim4, dim5 or dim6 in the output tensor Y1, the dimension mapping relationship corresponding to the slice dimension of the slice operator is a paired mapping.

[0070] return Figure 3 ,like Figure 3As further shown in , in some embodiments, the process can proceed to step 310, and in response to the dimensional mapping relationship corresponding to the slice dimension being a paired mapping, the position of the slice operator and / or the deformation operator can be adjusted to allow the slice operator to be pre-processed. Since the input tensor dimension value of the paired mapping is the same as the output tensor dimension value, the slice dimension can directly correspond to the dimension to which the input tensor dimension value belongs, and the slice dimension can be updated according to the dimensional mapping relationship, so that after adjusting the position of the slice operator and / or the deformation operator, the slice operator that updates the slice dimension can directly operate on the input tensor. For example, Figure 4 As shown in , assuming that the slice dimension of the slice-type operator corresponds to the dim6 dimension in the output tensor Y1, according to the dimension mapping relationship of the pairwise mapping, the slice dimension can be updated to the dim5 dimension of the corresponding input tensor X1 in order to perform position adjustment and pre-processing.

[0071] In some other embodiments, the process may proceed to step 312, and in response to the dimension mapping relationship corresponding to the slice dimension being an associative mapping, it is determined whether to perform pre-processing on the slice class operator according to the mapping type of the associative mapping corresponding to the slice dimension. The mapping type of the associative mapping may include, for example, a slicing mapping (e.g. Figure 4 The dotted arrows in the figure show the mapping relationship), exchange mapping (e.g. Figure 4 In some embodiments, in response to the mapping type of the associated mapping corresponding to the slice dimension being an exchange mapping, it may be selected not to perform pre-processing on the slice class operator as needed; or, according to the corresponding relationship of the slice dimension in the exchange mapping, the parameters of the slice class operator may be adjusted for pre-processing.

[0072] Further, in step 314, in response to the mapping type being a split mapping and the parameters of the slice operator satisfying the first preset condition, the parameters of the slice operator are adjusted to perform pre-processing, wherein the split mapping includes splitting the dimension value of the input tensor into multiple dimension values ​​of the output tensor. Figure 4 As shown in the example, the segmentation mapping can be Figure 4 The dimension value [4] in the input tensor X1 shown in the figure is split into multiple dimension values ​​[2, 2] of the output tensor Y1, that is, the number of dimensions of the input tensor in the split mapping is less than the number of dimensions of the corresponding output tensor.

[0073] The slicing parameters of the slicing operator can include the slicing dimension, the start position of the slice, the end position of the slice, and the fetching stride. Here, the slicing dimension includes the dimension for slicing, the start position and the end position of the slice are used to represent the slice range on the slicing dimension, and the fetching stride can represent the fetching step length in this slice range. According to these slicing parameters, the tensor input to the slicing operator can be sliced. When the slicing dimension falls into the partitioning mapping, the parameters of the slicing operator can be adjusted according to the first preset condition. Multiple embodiments for adjusting the parameters of the slicing operator will be described below.

[0074] In some embodiments, the first preset condition may include: start + step ≥ end; and the adjusted parameters of the slicing operator may include:

[0075] step new = value 2 ,

[0076] start new = start,

[0077] end new = (value 1 - 1) * value 2 + end;

[0078] where start represents the original slice start position of the slicing operator, step represents the original fetching stride of the slicing operator, end represents the original slice end position of the slicing operator, step new represents the new fetching stride of the adjusted slicing operator, start new represents the new slice start position of the adjusted slicing operator, end new represents the new slice end position of the adjusted slicing operator, value 1 represents the numerical value of the first output tensor dimension sliced out in the partitioning mapping, value 2 represents the numerical value of the second output tensor dimension sliced out in the partitioning mapping. In the Figure 4 example shown, the numerical value of the first output tensor dimension value 1 = 2, and the numerical value of the second output tensor dimension value 2 = 2. start and end are the slice ranges before the slicing operator adjusts the parameters. The original fetching stride step is the fetching stride before the slicing operator adjusts the parameters.

[0079] In some other embodiments, the first preset condition may include:

[0080] start + step < end, and

[0081] value 2 -(end-start-1+step)=0;

[0082] Adjusted parameters may include:

[0083] step new =step,

[0084] srart new =srart,

[0085] end new =(calue 1 -1)*value 2 +end;

[0086] Among them, start represents the original slice starting position of the slice operator, step represents the original data jump step of the slice operator, end represents the original slice ending position of the slice operator, step new Indicates the new data jump step of the adjusted slice operator, start new Indicates the new slice starting position of the adjusted slice operator, end new Indicates the new slice end position of the adjusted slice operator, value 1 Represents the first output tensor dimension value split in the split mapping, value 2 Represents the value of the second output tensor dimension split in the split mapping.

[0087] Combination of the above Figure 3 and Figure 4 How to determine whether to adjust the position of the slice operator and / or the deformation operator to make the slice operator pre-processed is described. It can be understood that the above description is exemplary and not restrictive. For example, the optimization method for the computation graph may not be limited to adjusting the position of the slice operator and / or the deformation operator, but may also include other optimization implementation methods to further improve the memory access efficiency of the device end. Figure 5a-Figure 7 An exemplary description is given.

[0088] Figure 5a-5c An exemplary flow chart of an optimization method including operator fusion according to multiple embodiments of the present disclosure is shown. Figure 5a FIG. 1 shows a schematic diagram of an optimization process including slicing operator fusion. Figure 5aAs shown in FIG. 1 , in an exemplary computational graph, the input tensor A511 will sequentially pass through the following nodes according to the flow of the computational graph: select operator 512, squeeze operator 513, slice operator 514, and Matmul (matrix multiplication) operator 515. Among these operators, select operator 512, squeeze operator 513, and slice operator 514 are all view type operators, and Matmul (matrix multiplication) operator 515 is a computation type operator.

[0089] like Figure 5a In the exemplary calculation graph shown in , in the optimization method of the disclosed embodiment, a view class operator subgraph can be obtained according to the view class operators in the calculation graph. For example, the select operator 512, the squeeze operator 513 and the slice operator 514 in the calculation order in the diagram constitute a view class operator subgraph. The view class operator subgraph can be optimized subsequently, and the data output by the optimized view class operator subgraph can be used for subsequent calculation of calculation class operators.

[0090] In some embodiments, deformation operators can be normalized to deformation target operators whose specified functions can replace each other; and slice operators can be normalized to slice target operators whose specified functions can replace each other. Although the specific functions implemented by view operators vary, they can also be classified. The main functions of unsqueeze operator, squeeze operator, view operator, view_as operator, reshape operator, reshape_as operator, etc. are to change the size of data dimensions and the size of dimension values. Such operators can be collectively referred to as deformation operators, and these operators are operators whose functions can replace each other. The main functions of narrow operator, select operator, unbind operator, chunk operator, slice operator and split operator are to slice memory data. Such operators can be collectively referred to as slice operators, and these operators are operators whose functions can replace each other.

[0091] For each category of functions, an operator can be selected to represent the functions of the category. The operator can implement the functions of all operators under the corresponding function category. That is, the operator can functionally replace all operators under the corresponding function category. In this article, the replaced operator is called a "target operator". In some embodiments, the deformation class target operator can be, for example, a reshape operator. In other embodiments, the slice class target operator can be, for example, a slice operator. In some embodiments, the deformation class target operator can be, for example, a view operator. In yet other embodiments, the slice class target operator can be, for example, a split operator. It can be understood that these operators are merely exemplary and not exhaustive, and those skilled in the art can construct similar functional classifications and functionally replaceable target operators based on the principles of the embodiments disclosed herein.

[0092] In some embodiments, the deformation class operator is normalized to the deformation class target operator, and the slice class operator is normalized to the slice class target operator, and the normalization can be achieved by deformation mapping. For example, in one example, the select operator can be obtained by deformation mapping of the slice operator and the squeeze operator. In another example, the squeeze operator can be deformed and mapped to the reshape operator by calculating the corresponding dimension. In other embodiments, the deformation class operator is normalized to the deformation class target operator, and the slice class operator is normalized to the slice class target operator, which can also be achieved by direct replacement.

[0093] like Figure 5a As shown in , assuming that the specified deformation class target operator is reshape, by normalizing the deformation class operator squeeze operator 513 in the view class operator subgraph, the reshape operator 517 can be obtained. In an example, assuming that the specified slice class target operator is slice, the slice operator 514 can be directly used as the normalized slice class target operator, and the select operator 512 is replaced by the slice operator 516 after normalization.

[0094] By classifying these operators according to their functions and replacing them with specified target operators, the types of operators in the operator subgraph can be reduced, which also facilitates subsequent optimization operations.

[0095] According to the normalized result, the position of the slice type target operator and / or the deformation type target operator in the normalized operator sequence can be adjusted to make the slice type target operator pre-processed. In one example, the slice operator 514 can be moved before the reshape operator 517, or the reshape operator 517 can be moved after the slice operator 514, and the parameters of the slice operator 514 are updated.

[0096] like Figure 5a As further shown in FIG. 1 , after the position adjustment is performed, multiple consecutive identical target operators appear, such as the slice operator 516 and the slice operator 514 (shown in dashed boxes in the figure). The multiple consecutive identical target operators can be fused into a single target operator (such as the slice operator 518 in the figure) to further optimize the view class operator subgraph. By fusion of multiple consecutive identical target operators, the number of operators can be reduced, thereby reducing the number of operators that need to be called later.

[0097] Figure 5a The optimization process of merging multiple slice operators in the operator sequence is shown. It can be understood that the operator sequence in a specific order may not be limited to only one deformation operator, but may also include multiple deformation operators. Figure 5b An exemplary description is given.

[0098] Figure 5b FIG. 1 shows a schematic diagram of an optimization process of an operator sequence including multiple deformation operators. Figure 5b As shown in FIG. 1 , in an exemplary computational graph, the input tensor B521 will sequentially pass through the following nodes according to the flow of the computational graph: slice operator 522, unsqueeze operator 523, reshape operator 524, chunk operator 525, and Matmul (matrix multiplication) operator 526. Among these operators, slice operator 522, unsqueeze operator 523, reshape operator 524, and chunk operator 525 are all view type operators, and Matmul (matrix multiplication) operator 526 is a computation type operator.

[0099] In an example, taking the deformation type target operator as the reshape operator and the slice type target operator as the slice operator as an example, the unsqueeze operator 523 can be normalized to the reshape operator 527, and the chunk operator 523 can be normalized to the slice operator 528, and the slice operator 522 and the reshape operator 524 can continue to be retained in the normalization operation.

[0100] According to the normalized result, the position of the slice target operator and / or the deformation target operator in the normalized operator sequence can be adjusted to make the slice target operator pre-processed. Figure 5bIn the example shown, the slice operator 528 may be moved before the reshape operators 527 and 524, or the reshape operators 527 and 524 may be moved after the slice operator 528. In some other embodiments, when there is an operator sequence of a specific order including a plurality of continuous deformation operators in the view operator subgraph, the slice operator may be moved forward or the deformation operator may be moved backward in multiple steps.

[0101] For example, in one example, the adjustment can be performed in two steps: the slice operator 528 is moved before the reshape operator 524, and then the slice operator 528 is moved before the reshape operator 527. Each move can be performed according to the above description, for example Figure 3 In another example, the adjustment can be made in two steps, where the reshape operator 524 is moved after the slice operator 528, and then the reshape operator 527 is moved after the slice operator 528. Each move can be made according to the above description, for example Figure 3 The described rules determine the parameters of the slice operator.

[0102] like Figure 5b As further shown in the figure, after the position adjustment is performed, multiple consecutive identical target operators appear, such as the slice operators 522 and 528 (shown in dotted boxes in the figure) and the reshape operators 527 and 524 (shown in dotted boxes in the figure) in the figure. The multiple consecutive identical target operators can be fused into a single target operator, for example, the slice operators 522 and 528 in the figure are fused into the slice operator 529, and the reshape operators 527 and 524 are fused into the reshape operator 524 to achieve further optimization of the view class operator subgraph.

[0103] In one example, when multiple deformable target operators are fused, since the parameters of the deformable operators usually include information about the dimension change, that is, the rule for changing the data dimension, the parameters of the last deformable target operator among multiple continuous deformable target operators can be equivalent to the parameters of the continuous multiple deformable target operators. Figure 5b In the example of , reshape operator 527 and reshape operator 524 can be fused into a single target operator, and the parameters of the fused single target operator can be the same as those of reshape operator 524, so Figure 5b After the reshape operators 527 and 524 are merged, they can be replaced by the reshape operator 524.

[0104] Combination of the above Figure 5bThe optimization process of an operator sequence including a plurality of continuous deformation operators is described. It is understandable that the above description is exemplary and not restrictive. For example, fusing a plurality of continuous identical target operators into a single target operator may not be limited to being performed after adjusting the position, but may also be performed before adjusting the position, which helps to reduce the complexity of subsequent operations of adjusting the operator position. In other embodiments, Figure 5b After the normalization operation shown in , there are multiple continuous and identical deformation target operators. The multiple deformation target operators (i.e., reshape operators 527 and 524) can be fused into a single deformation target operator, and then the position adjustment and subsequent fusion of multiple slice operators are performed. For another example, the normalization operation can also be performed before the position adjustment, and can also be performed after the position adjustment. Figure 5c An exemplary description is given.

[0105] Figure 5c Figure 2 shows a schematic diagram of the optimization process for adjusting the operator position before normalization. Figure 5c As shown in FIG. 5 , in an exemplary computational graph, the input tensor C531 will sequentially pass through the following nodes according to the flow of the computational graph: select operator 532, squeeze operator 533, slice operator 534, and Matmul (matrix multiplication) operator 535. Among these operators, select operator 532, squeeze operator 533, and slice operator 534 are all view type operators, and Matmul (matrix multiplication) operator 535 is a computation type operator.

[0106] By traversing the computational graph, a view operator subgraph including select operator 532, squeeze operator 533 and slice operator 534 can be obtained. Since squeeze operator 533 belongs to the deformation operator, select operator 532 and slice operator 534 belong to the slice operator, there exists an operator sequence in a specific order in which the deformation operator is located before the slice operator: select operator 532+squeeze operator 533+slice operator 534.

[0107] In some embodiments, the slice operator 534 may be moved before the squeeze operator 533 , or the squeeze operator 533 may be moved after the slice operator 534 , so that the slice operator 534 may be processed in advance.

[0108] After the position adjustment is performed, the deformation type operator can be normalized to the specified transposition type target operator whose functions can replace each other; and the slice type operator can be normalized to the specified slice type target operator whose functions can replace each other. Taking the deformation type target operator as the reshape operator and the slice type target operator as the slice operator as an example, the squeeze operator 533 can be normalized to the reshape operator 537, and the select operator 532 can be normalized to the slice operator 536, and the slice operator 534 can continue to be retained in the normalization operation.

[0109] According to the normalized result, multiple consecutive identical target operators appear, and the multiple consecutive identical target operators can be fused into a single target operator. Figure 5c As further shown in the figure, the slice operators 536 and 534 (shown in dashed boxes in the figure) in the figure can be fused into a single target operator (such as the slice operator 538 in the figure) to achieve further optimization of the view class operator subgraph.

[0110] The above describes the optimization method flow including normalization and fusion operations according to the embodiment of the present disclosure in combination with multiple accompanying drawings. It can be understood that although the figure only shows the case where the operator sequence includes multiple deformation operators or multiple slice operators, when there is an operator sequence composed of multiple continuous deformation operators and multiple continuous slice operators, the method of optimizing the view operator subgraph by combining the above-mentioned normalization, position adjustment, operator fusion and other operations is still applicable, and will not be repeated here.

[0111] In the foregoing, the embodiment of fusing multiple consecutive identical target operators when they are slice-type operators has been mentioned many times. In order to make it easier to understand the implementation method of fusing slice-type target operators, it will be further explained below.

[0112] In some embodiments, when multiple consecutive identical target operators are slice-type operators, fusing the multiple consecutive identical target operators into a single target operator may include: in response to the multiple target operators operating in the same dimension, the equivalent parameters of the fused single target operator can be determined according to the corresponding slice ranges and data access steps of the multiple target operators in the same dimension; in response to the multiple target operators operating in different dimensions, the dimensional operations of the multiple target operators in different dimensions can be merged.

[0113] When multiple target operators operate in the same dimension, the target operator that operates later will further slice the data obtained by the target operator that operates earlier. Therefore, the equivalent parameters of the single target operator after fusion can be determined according to parameters such as the slicing range and data skipping step of each target operator, so that the single target operator can be equivalent to the multiple target operators before fusion, and the equivalent parameters can be equivalent to the parameters of the multiple target operators before fusion.

[0114] In other embodiments, determining the equivalent parameters of the fused single target operator may include: determining the equivalent slice starting position of the fused single target operator based on the first data access jump step of the target operator that performs the operation first among the multiple target operators and the slice starting position of each target operator; determining the equivalent slice ending position of the fused single target operator based on the first data access jump step and the first slice starting position of the target operator that performs the operation first among the multiple target operators, and based on the slice ending position of the target operator that performs the operation later among the multiple target operators; and determining the equivalent data access jump step of the fused single target operator based on the data access jump step of each target operator among the multiple target operators.

[0115] In some embodiments, the determined equivalent parameters may include:

[0116] dim new =dim,

[0117] start new =start1+start2*step1,

[0118] end new =start1+end2*step1,

[0119] step new =step1*step2,

[0120] Among them, dim new Indicates the equivalent slice dimension after fusion, dim indicates the same dimension on which multiple target operators operate, start new Indicates the starting position of the equivalent slice after fusion. start1 indicates the starting position of the first slice of the target operator that performs the operation first among multiple target operators. start2 indicates the starting position of the second slice of the target operator that performs the operation later among multiple target operators. end new Indicates the end position of the equivalent slice after fusion, end1 indicates the end position of the first slice of the target operator that performs the operation first among multiple target operators, end2 indicates the end position of the second slice of the target operator that performs the operation later among multiple target operators, step newIndicates the equivalent data fetching jump step after fusion, step1 indicates the first data fetching jump step of the target operator that performs the operation first among multiple target operators, and step2 indicates the second data fetching jump step of the target operator that performs the operation later among multiple target operators.

[0121] Figure 6 An example of multiple slice-like target operators operating on the same dimension is shown. Figure 6 As shown in, in this example, taking the fusion of two adjacent slice-type target operators as an example, it is assumed that the slice dimensions of the slice-type target operator slice1 that performs the operation first and the slice-type target operator slice2 that performs the operation later are both dim, and the data included in the slice dimension dim is [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], and other parameters of slice1 also include the first slice starting position start1=0, the first slice ending position end1=6, and the first data access step step1=2, and start1 and end1 constitute the slice range of slice1; other parameters of slice2 also include the second slice starting position start2=0, the second slice ending position end2=2, and the second data access step step2=1, and start2 and end2 constitute the slice range of slice2.

[0122] like Figure 6 As shown in , under normal circumstances, according to the slicing range of slice1 and the first access step, the access result of slice1 on dim is [0, 2, 4]; according to the slicing range of slice2 and the second access step, the access result of slice2 on [0, 2, 4] is [0, 2].

[0123] According to the method for determining equivalent parameters of the disclosed embodiment, since slice1 and slice2 operate in the same dimension, the equivalent slice dimension dim in the equivalent parameters after the two are fused is new Same as dim, that is, the slicing operation after merging slice1 and slice2 is still performed on the slice dimension of the target operator. In addition, the starting position of the equivalent slice after merging slice1 and slice2 is start new =start1+start2*step1=0+0*2=0, equivalent slice end position end new =start1+end2*step1=0+2*2=4, equivalent to data jump step new=step1*step2=2*1=2. Based on the determined equivalent parameters, when equivalent slicing is performed on [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], the equivalent number-taking result is [0, 2]. This equivalent number-taking result is the same as the final number-taking result obtained when slice1 and slice2 are operated in sequence. Therefore, a single target operator represented by the equivalent parameters can be used to replace multiple consecutive identical target operators.

[0124] It can be understood that when there are more than two consecutive identical target operators, the equivalent parameters of two consecutive target operators can be calculated in sequence based on the above-mentioned equivalent parameter calculation formula until all consecutive identical target operators are fused into a single target operator, or iterative calculation can be performed based on the above-mentioned equivalent parameter calculation formula to obtain the equivalent parameters of the fused single target operator.

[0125] In other embodiments, when multiple target operators operate in different dimensions, the dimensional operations of multiple consecutive target operators are executed sequentially, each target operator performs dimensional operations on its input tensor data in turn, and the target operators are continuous and identical. These dimensional operations can be merged to achieve the effect of multiple dimensional operations through a single target operator.

[0126] In one embodiment, equivalent parameters after fusion of multiple consecutive target operators operating in different dimensions may include:

[0127] dim new =(dim1, dim2, … dimN),

[0128] start new =(start1,start2,…startN),

[0129] end new =(end1,end2,…endN),

[0130] step new =(step1,step2,…stepN),

[0131] Among them, dim new Indicates the equivalent slice dimension after fusion, dim1 indicates the slice dimension of the first target operator to perform the operation, dim2 indicates the slice dimension of the second target operator to perform the operation, N indicates the number of consecutive identical target operators operating in different dimensions, start new Indicates the starting position of the equivalent slice after fusion. start1 indicates the starting position of the first slice in the first target operator that performs the operation. start2 indicates the starting position of the second slice in the second target operator that performs the operation. endnew Indicates the end position of the equivalent slice after fusion, end1 indicates the end position of the first slice of the first target operator performing the operation, end2 indicates the end position of the second slice of the second target operator performing the operation, step new Indicates the equivalent data fetching jump step after fusion, step1 indicates the first data fetching jump step of the first target operator performing the operation, and step2 indicates the second data fetching jump step of the second target operator performing the operation.

[0132] In an example, suppose there are two consecutive slice-type target operators slice in the operator sequence. The dimension operations of these two slice operators can be merged into one. Assume that the first slice performs a slice operation on the dim1 dimension, and the second slice performs a slice operation on the dim2 dimension. Since the two slices operate on different dimensions, the parameters of the two slices can be directly merged. When these two slice operators are merged into one slice operator, what it wants to achieve is to perform the operation of the first slice on the dim1 dimension and the operation of the second slice on the dim2 dimension at the same time.

[0133] In the above, in combination with multiple figures, embodiments such as the presence of a slice operator before a deformation operator in a specific order of operator sequence, the fusion of multiple slice operators, etc. are described. It can be understood that the presence of a slice operator before a deformation operator in a specific order of operator sequence is not limited to only a slice operator, so as to facilitate the fusion optimization of slice operators, and other types of operators may also exist to perform other optimization methods. Figure 7 An example is shown for description.

[0134] Figure 7 An example diagram of an optimization process in which a transposition operator exists before a deformation operator in an operator sequence according to an embodiment of the present disclosure is shown. Figure 7 As shown in FIG. 1( a ), in an exemplary computation graph, the input tensor D711 will sequentially pass through the following nodes according to the flow of the computation graph: transpose operator 712, reshape operator 713, slice operator 714, and Matmul (matrix multiplication) operator 715. Among these operators, transpose operator 712, reshape operator 713, and slice operator 714 are all view operators, and Matmul (matrix multiplication) operator 715 is a computation operator.

[0135] In the optimization method of the disclosed embodiment, a view class operator subgraph can be obtained according to the view class operators in the calculation graph. For example, the transpose operator 712, the reshape operator 713 and the slice operator 714 in the diagram in the calculation order constitute a view class operator subgraph. The view class operator subgraph can be optimized subsequently, and the data output by the optimized view class operator subgraph can be used for subsequent calculations of calculation class operators.

[0136] In the view class operator subgraph in this example, the reshape operator 713 is located before the slice operator 714, and the transpose operator 712 is located before the reshape operator 713, so there is a specific order of operator sequence transpose operator 712+reshape operator 713+slice operator 714. Then, as Figure 7 As shown in FIG. 7( b), the position of the slice target operator and / or the deformation target operator in the operator sequence can be adjusted so that the slice target operator is pre-processed. That is, the slice operator 714 can be moved before the reshape operator 713, or the reshape operator 713 can be moved after the slice operator 714, and the parameters of the slice operator 714 can be updated.

[0137] In such Figure 7 In the view operator subgraph obtained after adjusting the position in Figure (b), the slice operator 714 is located after the transpose operator 712. Since the slice operator itself has the characteristic of reducing the amount of memory data, and the transpose operator does not change the amount of memory data, when there is an operator sequence in the view operator subgraph where the transpose operator is located before the slice operator, further optimization can be performed to further reduce the IO data handling amount and further improve the memory access efficiency of the hardware device in the case of continuous operator calls.

[0138] Furthermore, in some embodiments, the optimization method of the disclosed embodiment may further include: in response to the presence of a transposition operator before the deformation operator, after adjusting the position of the slicing operator and / or the deformation operator, the position of the slicing operator and / or the transposition operator may be further adjusted to make the slicing operator be processed in advance. Figure 7 As shown in Figure (c), the slice operator (slice operator 714) can be moved before the transpose operator (transpose operator 712), and / or the transpose operator (transpose operator 712) can be moved after the slice operator (slice operator 714) to achieve pre-processing of the slice operator (slice operator 714).

[0139] In some other embodiments, adjusting the position of the slice operator and / or the transposition operator in the operator sequence so that the pre-processing of the slice operator may include: updating the parameters of the slice operator according to the dimension mapping relationship between the dimension information of the input tensor of the transposition operator and the slice dimension parameters of the slice operator, so as to adjust the slice operator to be processed before the transposition operator. In some other embodiments, a first mapping relationship between the dimension information of the input tensor and the transformation dimension parameters of the transposition operator may be determined; a second mapping relationship between the slice dimension parameters of the slice operator and the transformation dimension parameters of the transposition operator may be determined; and a dimension mapping relationship may be determined according to the first mapping relationship and the second mapping relationship.

[0140] Since the slice dimension parameter represents the dimension of the slice processing, the data acquisition operations of the slice processing are all performed on this dimension. Therefore, the parameters of the slice operator can be updated according to the dimensional correspondence between the slice dimension parameter and the dimensional information of the input tensor of the transposition operator. This is so that when the slice operator is adjusted to be processed before the transposition operator, the input tensor of the transposition operator can be sliced ​​first, thereby reducing the amount of data when performing subsequent transposition operator calculations, and reducing the amount of data carried by the IO operator.

[0141] In one example, assuming that the dimension information of the input tensor of the transposition operator is X=[dim1, dim2, dim3, dim4, dim5, dim6, dim7, dim8], where dim1, dim2, etc. respectively represent dimension 1, dimension 2, etc., a total of 8 dimensions; assuming that the transformation dimension parameters of the transposition operator include Dims=[dim7, dim8, dim1, dim2, dim5, dim4, dim3, dim6], indicating that the dimensions of X are rearranged, where dimension 7 and dimension 8 are transformed to dimension 1 and dimension 2 before dimension 3 and dimension 5 are swapped, such a mapping relationship can be the first mapping relationship. Assuming that the slice dimension parameter of the slice operator is dim4', it means that the fourth dimension of the tensor of the input slice operator is sliced, and the fourth dimension after transformation in the transformation dimension parameter of the transposition operator is dim2, then the slice dimension parameter dim4' of the slice operator actually corresponds to dim2 in the transformation dimension parameter for slicing, and this mapping relationship can be the second mapping relationship.

[0142] According to the first mapping relationship and the second mapping relationship, it can be determined that the dimension mapping relationship between the dimension information X of the input tensor of the transposition operator and the slice dimension parameter dim4' of the slice operator is that dim4' is mapped to the dimension position of dim2 in X. Therefore, when the slice operator is pre-processed, the slice dimension parameter can be updated to dim2', indicating that the second dimension of the input tensor is sliced, that is, the corresponding dim2 of X is sliced.

[0143] Understandably, Figure 7 Before or after each position adjustment as shown in , operators of the same type can be normalized. When multiple consecutive identical target operators appear, the subgraph can still be further optimized by combining the fusion operation.

[0144] The above describes the optimization method of the computational graph of the disclosed embodiment in conjunction with the accompanying drawings. By adjusting the position of the slice operator and / or deformation operator in the operator sequence of a specific order, the slice operator is pre-processed, which can facilitate the further optimization of the subsequent view subgraph, and is conducive to reducing the amount of memory data carried, improving the memory access efficiency of the hardware device and reducing network time consumption. In some embodiments, by fusing multiple consecutive identical target operators, the number of operator calls can be effectively reduced, and continuous calls of the same operator can be avoided, thereby improving the operator call efficiency, and further improving the memory access efficiency of the hardware device and the end-to-end operation efficiency of the network.

[0145] The present disclosure also provides a computing device, which can be used to optimize a computing graph. Figure 8 800 is a block diagram showing a hardware configuration of a computing device 800 that can implement the disclosed embodiments. As shown in the figure, the computing device 800 may include a processor 810 and a memory 820. Figure 8 In the computing device 800, only the components related to this embodiment are shown. Therefore, it is obvious to those skilled in the art that the computing device 800 may also include Figure 8 The components shown in the figure are different from common components, such as: display.

[0146] The computing device 800 may correspond to a computing device having various processing functions, such as a function for compiling a computational graph. For example, the computing device 800 may be implemented as various types of devices, such as a personal computer (PC), a server device, a mobile device, etc.

[0147] The processor 810 is configured to execute program instructions to control all functions of the computing device 800. For example, the processor 810 controls all functions of the computing device 800 by executing a program stored in the memory 820 on the computing device 800. The processor 810 may be implemented by a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), an artificial intelligence processor chip (IPU), etc. provided in the computing device 800. However, the present disclosure is not limited thereto.

[0148] The memory 820 is hardware for storing various data processed in the computing device 800. For example, the memory 820 can store processed data and data to be processed in the computing device 800. The memory 820 can store data that has been processed or to be processed by the processor 810, such as a calculation graph before compilation, a calculation graph after compilation, etc. In addition, the memory 820 can store program instructions such as applications, drivers, etc. to be driven by the computing device 800. For example: the memory 820 can store various programs related to the optimization algorithm of the calculation graph to be executed by the processor 810. The memory 820 can be a DRAM, but the present disclosure is not limited thereto. The memory 820 can include at least one of a volatile memory or a non-volatile memory. The non-volatile memory can include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a phase change RAM (PRAM), a magnetic RAM (MRAM), a resistive RAM (RRAM), a ferroelectric RAM (FRAM), etc. The volatile memory may include a dynamic RAM (DRAM), a static RAM (SRAM), a synchronous DRAM (SDRAM), a PRAM, an MRAM, an RRAM, a ferroelectric RAM (FeRAM), etc. In an embodiment, the memory 820 may include at least one of a hard disk drive (HDD), a solid state drive (SSD), a high-density flash memory (CF), a secure digital (SD) card, a micro secure digital (Micro-SD) card, a mini secure digital (Mini-SD) card, an extreme digital (xD) card, caches, or a memory stick.

[0149] In summary, the specific functions implemented by the memory 820 and the processor 810 of the computing device 800 provided in the embodiments of this specification can be explained in comparison with the aforementioned embodiments in this specification, and can achieve the technical effects of the aforementioned embodiments, and will not be repeated here.

[0150] In the embodiments of the present disclosure, a computer-readable storage medium is also provided, in which program instructions are stored. When the program instructions are loaded and executed by a processor, the processor executes the calculation graph optimization method described in the embodiments of the present disclosure.

[0151] In the embodiments of the present disclosure, a computer program product is also provided, including a computer program or instructions, which, when executed by a processor, implements the optimization method of the computational graph described in the embodiments of the present disclosure.

[0152] Fig. 9900 according to an embodiment of the present disclosure. As shown in the figure, the combined processing device 900 includes a computing processing device 902, an interface device 904, other processing devices 906 and a storage device 908. According to different application scenarios, the computing processing device may include one or more computing devices 910, which may be configured as Figure 8 The computing device 800 shown is used to perform the operations described herein in conjunction with the accompanying drawings.

[0153] In different embodiments, the computing and processing device disclosed herein may be configured to perform user-specified operations. In an exemplary application, the computing and processing device may be implemented as a single-core artificial intelligence processor or a multi-core artificial intelligence processor. Similarly, one or more computing devices included in the computing and processing device may be implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core. When multiple computing devices are implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core, with respect to the computing and processing device disclosed herein, it may be regarded as having a single-core structure or a homogeneous multi-core structure.

[0154] In an exemplary operation, the computing processing device of the present disclosure can interact with other processing devices through an interface device to jointly complete the operation specified by the user. Depending on the implementation, other processing devices of the present disclosure may include one or more types of processors in general and / or special processors such as a central processing unit (CPU), a graphics processing unit (GPU), an artificial intelligence processor, etc. These processors may include but are not limited to a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, only with respect to the computing processing device of the present disclosure, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing processing device and other processing devices are considered together, the two can be regarded as forming a heterogeneous multi-core structure.

[0155] In one or more embodiments, the other processing device can serve as an interface between the computing device disclosed herein (which can be embodied as an artificial intelligence such as a computing device related to neural network computing) and external data and control, and perform basic controls including but not limited to data handling, starting and / or stopping the computing device, etc. In other embodiments, the other processing device can also cooperate with the computing device to jointly complete computing tasks.

[0156] In one or more embodiments, the interface device can be used to transmit data and control instructions between the computing and processing device and other processing devices. For example, the computing and processing device can obtain input data from other processing devices via the interface device and write it into a storage device (or memory) on the computing and processing device chip. Further, the computing and processing device can obtain control instructions from other processing devices via the interface device and write them into a control cache on the computing and processing device chip. Alternatively or optionally, the interface device can also read data from the storage device of the computing and processing device and transmit it to other processing devices.

[0157] Additionally or optionally, the combined processing device of the present disclosure may further include a storage device. As shown in the figure, the storage device is connected to the computing processing device and the other processing device, respectively. In one or more embodiments, the storage device may be used to store data of the computing processing device and / or the other processing device. For example, the data may be data that cannot be fully stored in the internal or on-chip storage device of the computing processing device or other processing device.

[0158] In some embodiments, the present disclosure also discloses a chip (e.g. Fig.10 In one implementation, the chip is a system on chip (SoC) and integrates one or more Fig. 9 The chip can be connected to an external interface device (such as Fig.10 The external interface device 1006 shown in the figure is connected to other related components. The related components may be, for example, a camera, a display, a mouse, a keyboard, a network card or a wifi interface. In some application scenarios, other processing units (such as a video codec) and / or interface modules (such as a DRAM interface) may be integrated on the chip. In some embodiments, the present disclosure further discloses a chip packaging structure, which includes the above-mentioned chip. In some embodiments, the present disclosure further discloses a board card, which includes the above-mentioned chip packaging structure. The following will be combined with Fig.10 The board is described in detail.

[0159] Fig.101 is a schematic diagram showing the structure of a board 1000 according to an embodiment of the present disclosure. As shown in the figure, the board includes a storage device 1004 for storing data, which includes one or more storage units 1010. The storage device can be connected and transmit data with the control device 1008 and the chip 1002 described above by means such as a bus. Further, the board also includes an external interface device 1006, which is configured for data relay or switching between the chip (or the chip in the chip packaging structure) and the external device 1012 (such as a server or computer, etc.). For example, the data to be processed can be transmitted to the chip by the external device through the external interface device. For another example, the calculation result of the chip can be transmitted back to the external device via the external interface device. According to different application scenarios, the external interface device can have different interface forms, for example, it can adopt a standard PCIE interface, etc.

[0160] In one or more embodiments, the control device in the disclosed board may be configured to regulate the state of the chip. To this end, in an application scenario, the control device may include a microcontroller unit (MCU) to regulate the working state of the chip.

[0161] According to the above combination Fig. 9 and Fig.10 Based on the description, those skilled in the art can understand that the present disclosure also discloses an electronic device or apparatus, which may include one or more of the above-mentioned boards, one or more of the above-mentioned chips and / or one or more of the above-mentioned combined processing devices.

[0162] According to different application scenarios, the electronic equipment or device disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, driving recorders, navigators, sensors, cameras, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, automatic driving terminals, transportation, household appliances, and / or medical equipment. The transportation includes airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; the medical equipment includes magnetic resonance imaging, ultrasound machines and / or electrocardiographs. The electronic equipment or device disclosed herein may also be applied to the Internet, IoT, data centers, energy, transportation, public administration, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and medical fields. Further, the electronic equipment or device disclosed herein may also be used in cloud, edge, and terminal applications related to artificial intelligence, big data, and / or cloud computing. In one or more embodiments, electronic devices or devices with high computing power according to the disclosed solution can be applied to cloud devices (such as cloud servers), while electronic devices or devices with low power consumption can be applied to terminal devices and / or edge devices (such as smart phones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-to-end.

[0163] It should be noted that, for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art will appreciate that the scheme of the present disclosure is not limited by the order of the actions described. Therefore, based on the disclosure or teaching of the present disclosure, those skilled in the art will appreciate that some of the steps therein may be performed in other orders or simultaneously. Further, those skilled in the art will appreciate that the embodiments described in the present disclosure may be regarded as optional embodiments, i.e., the actions or modules involved therein may not necessarily be necessary for the implementation of one or some of the schemes of the present disclosure. In addition, depending on the different schemes, the present disclosure also has different focuses on the description of some embodiments. In view of this, those skilled in the art may appreciate the parts that are not described in detail in a certain embodiment of the present disclosure, and may also refer to the relevant descriptions of other embodiments.

[0164] In terms of specific implementation, based on the disclosure and teaching of this disclosure, those skilled in the art can understand that several embodiments disclosed in this disclosure can also be implemented by other methods not disclosed herein. For example, with respect to the various units in the electronic device or device embodiments described above, this article divides them on the basis of considering logical functions, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. In terms of the connection relationship between different units or components, the connection discussed in the above text in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0165] In the present disclosure, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed on multiple network units. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the scheme described in the embodiments of the present disclosure. In addition, in some scenarios, multiple units in the embodiments of the present disclosure may be integrated into one unit or each unit may exist physically separately.

[0166] In some other implementation scenarios, the above-mentioned integrated unit can also be implemented in the form of hardware, that is, a specific hardware circuit, which may include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit may include but is not limited to physical devices, and the physical devices may include but are not limited to devices such as transistors or memristors. In view of this, the various devices described herein (such as computing devices or other processing devices) can be implemented by appropriate hardware processors, such as CPU, GPU, FPGA, DSP and ASIC, etc. Further, the aforementioned storage unit or storage device can be any appropriate storage medium (including magnetic storage medium or magneto-optical storage medium, etc.), which can be, for example, a variable resistive memory (Resistive Random Access Memory, RRAM), a dynamic random access memory (Dynamic Random Access Memory, DRAM), a static random access memory (Static Random Access Memory, SRAM), an enhanced dynamic random access memory (Enhanced Dynamic Random Access Memory, EDRAM), a high bandwidth memory (High Bandwidth Memory, HBM), a hybrid memory cube (Hybrid Memory Cube, HMC), ROM and RAM, etc.

[0167] The foregoing content can be better understood in accordance with the following terms:

[0168] Clause 1. A method for optimizing a computational graph, comprising:

[0169] Get the view operator subgraph of the tensor data in the computation graph;

[0170] Determine whether there is an operator sequence in a specific order in the view operator subgraph, wherein the specific order includes that the deformation operator is located before the slice operator, and there are other types of operators before the deformation operator except the deformation operator; and

[0171] In response to the existence of the operator sequence in the specific order, the position of the slicing operator and / or the deformation operator in the operator sequence is adjusted so that the slicing operator is processed in advance.

[0172] Clause 2. The optimization method according to clause 1, further comprising:

[0173] Normalizing the deformation class operators to specified deformation class target operators whose functions can replace each other;

[0174] Normalizing the slice-type operators to designated slice-type target operators whose functions can replace each other; and

[0175] According to the normalized result, the position of the slice-type target operator and / or the deformation-type target operator in the normalized operator sequence is adjusted so that the slice-type target operator is pre-processed.

[0176] Clause 3. An optimization method according to clause 1 or 2, wherein the operator sequence in a specific order comprises a continuous sequence of deformation operators and slicing operators.

[0177] Clause 4. The optimization method according to any one of clauses 1 to 3, wherein before adjusting the position of the slicing operator and / or the deformation operator in the operator sequence so that the slicing operator is pre-processed, the optimization method further comprises:

[0178] Determine a dimensional mapping relationship between an input tensor and an output tensor of the deformation operator according to a relationship between a dimensional value of the input tensor and a dimensional value of the output tensor of the deformation operator, wherein the dimensional value represents the amount of data in each dimension; and

[0179] According to the dimension mapping relationship corresponding to the slice dimension of the slice operator, it is determined whether to adjust the position of the slice operator and / or the deformation operator so that the slice operator is pre-processed.

[0180] Clause 5. The optimization method according to clause 4, wherein determining the dimensional mapping relationship between the input tensor and the output tensor of the deformation operator comprises:

[0181] In response to the presence of corresponding identical dimension values ​​in the input tensor dimension values ​​and the output tensor dimension values, mapping the dimensions to which the corresponding identical dimension values ​​belong in pairs;

[0182] In response to the presence of non-corresponding dimension values ​​in the input tensor dimension values ​​and the output tensor dimension values, the dimensions to which the products of the non-corresponding dimension values ​​correspond to the same dimension values ​​are associated and mapped.

[0183] Clause 6. The optimization method according to clause 5, wherein determining whether to adjust the position of the slicing operator and / or the deformation operator so that the slicing operator pre-processing comprises:

[0184] In response to the dimension mapping relationship corresponding to the slice dimension being a paired mapping, adjusting the position of the slice operator and / or the deformation operator so that the slice operator is pre-processed;

[0185] In response to the dimension mapping relationship corresponding to the slice dimension being an associative mapping, it is determined whether to perform pre-processing on the slice class operator according to a mapping type of the associative mapping corresponding to the slice dimension.

[0186] Clause 7. The optimization method according to Clause 6, wherein determining whether to perform preprocessing on the slice-like operator includes:

[0187] In response to the mapping type being a split mapping and the parameters of the slice-like operator satisfying a first preset condition, adjusting the parameters of the slice-like operator for preprocessing, wherein the split mapping includes splitting the dimension value of the input tensor into multiple dimension values of the output tensor.

[0188] Clause 8. The optimization method according to Clause 7, wherein

[0189] the first preset condition includes: start + step ≥ end; and

[0190] the adjusted parameters include:

[0191] step new = value 2 ,

[0192] start new = start,

[0193] end new = (value 1 - 1)*value 2 + end;

[0194] wherein, start represents the original slice start position of the slice-like operator, step represents the original fetching skip of the slice-like operator, end represents the original slice end position of the slice-like operator, step new represents the new fetching skip of the adjusted slice-like operator, start new represents the new slice start position of the adjusted slice-like operator, end new represents the new slice end position of the adjusted slice-like operator, value 1 represents the first output tensor dimension value sliced out in the split mapping, value 2 represents the second output tensor dimension value sliced out in the split mapping.

[0195] Clause 9. The optimization method according to Clause 7, wherein

[0196] the first preset condition includes:

[0197] start + step < end, and

[0198] value 2 - (end - start - 1 + step) = 0; and

[0199] The adjusted parameters include:

[0200] step new =step

[0201] start new =start

[0202] end new =(value 1 -1)*value 2 +end;

[0203] Among them, start represents the original slice starting position of the slice operator, step represents the original data jump step of the slice operator, end represents the original slice ending position of the slice operator, step new Indicates the new data jump step of the adjusted slice operator, start new Indicates the new slice starting position of the adjusted slice operator, end new Indicates the new slice end position of the adjusted slice operator, value 1 Indicates the first output tensor dimension value split out in the split mapping, value 2 Represents the value of the second output tensor dimension split in the split mapping.

[0204] Clause 10. The optimization method according to clause 2, further comprising:

[0205] Before performing the position adjustment, multiple consecutive identical target operators are fused into a single target operator; and / or

[0206] After performing the adjustment position, multiple consecutive identical target operators are fused into a single target operator.

[0207] Clause 11. The optimization method according to clause 1, further comprising: after performing the position adjustment,

[0208] Normalizing the deformation class operators to specified deformation class target operators whose functions can replace each other;

[0209] Normalizing the slice-type operators to designated slice-type target operators whose functions can replace each other; and

[0210] According to the normalized results, multiple consecutive identical target operators are fused into a single target operator.

[0211] Clause 12. The optimization method according to clause 10 or 11, wherein when the multiple consecutive identical target operators are slice operators, fusing the multiple consecutive identical target operators into a single target operator comprises:

[0212] In response to the multiple target operators operating in the same dimension, determining equivalent parameters of a fused single target operator according to corresponding slicing ranges and data fetching steps of the multiple target operators in the same dimension;

[0213] In response to the multiple target operators operating in different dimensions, the dimensional operations of the multiple target operators in different dimensions are merged.

[0214] Clause 13. The optimization method according to clause 12, wherein determining the equivalent parameters of the fused single target operator comprises:

[0215] Determine an equivalent slice starting position of a fused single target operator according to a first data fetching jump step of a target operator that first performs operation among the multiple target operators and a slice starting position of each target operator;

[0216] Determine an equivalent slice end position of a fused single target operator according to a first data fetch jump step and a first slice start position of a target operator that performs operations first among the multiple target operators, and according to a slice end position of a target operator that performs operations later among the multiple target operators; and

[0217] According to the data acquisition jump step of each target operator in the multiple target operators, an equivalent data acquisition jump step of a fused single target operator is determined.

[0218] Clause 14. The optimization method according to clause 12 or 13, wherein determining the equivalent parameter comprises:

[0219] dim new =dim,

[0220] start new =start1+start2*step1,

[0221] end new =start1+end2*step1,

[0222] step new =step1*step2,

[0223] Among them, dim new represents the equivalent slice dimension after fusion, dim represents the same dimension on which the multiple target operators operate, start new represents the starting position of the equivalent slice after fusion, start1 represents the starting position of the first slice of the target operator that performs the operation first among the multiple target operators, start2 represents the starting position of the second slice of the target operator that performs the operation later among the multiple target operators, and end newrepresents the end position of the equivalent slice after fusion, end1 represents the end position of the first slice of the target operator that performs the operation first among the multiple target operators, end2 represents the end position of the second slice of the target operator that performs the operation later among the multiple target operators, step new It represents the equivalent data fetching jump step after fusion, step1 represents the first data fetching jump step of the target operator that performs the operation first among the multiple target operators, and step2 represents the second data fetching jump step of the target operator that performs the operation later among the multiple target operators.

[0224] Clause 15. The optimization method according to clause 2, wherein:

[0225] The deformation target operator is a reshape operator;

[0226] The slice type target operator is a slice operator.

[0227] Clause 16. An optimization method according to Clause 1, wherein the other types of operators include at least one of slicing type operators, transposition type operators and extension type operators.

[0228] Clause 17. A computing device for optimizing a computational graph, comprising:

[0229] a processor configured to execute program instructions; and

[0230] A memory configured to store the program instructions, which, when loaded and executed by the processor, enables the processor to perform the calculation graph optimization method according to any one of clauses 1-16.

[0231] Clause 18. A computer-readable storage medium having computer-readable instructions stored thereon, which, when executed by one or more processors, implements the computational graph optimization method described in any one of Clauses 1-16.

[0232] Clause 19. A computer program product, comprising a computer program or instructions, which, when executed by a processor, implements the method for optimizing a computational graph as described in any one of Clauses 1-16.

[0233] Although multiple embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art may think of many changes, modifications, and alternatives without departing from the thought and spirit of the present disclosure. It should be understood that in the process of practicing the present disclosure, various alternatives to the embodiments of the present disclosure described herein may be adopted. The attached claims are intended to define the scope of protection of the present disclosure, and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A method for optimizing a computational graph, comprising: Get the view operator subgraph of the tensor data in the computation graph; Determine whether there is an operator sequence in a specific order in the view operator subgraph, wherein the specific order includes that the deformation operator is located before the slice operator, and there are other types of operators before the deformation operator except the deformation operator; as well as In response to the existence of the operator sequence in the specific order, the position of the slicing operator and / or the deformation operator in the operator sequence is adjusted so that the slicing operator is processed in advance.

2. The optimization method according to claim 1, further comprising: Normalizing the deformation class operators to specified deformation class target operators whose functions can replace each other; Normalizing the slice-type operators to designated slice-type target operators whose functions can replace each other; as well as According to the normalized result, the position of the slice-type target operator and / or the deformation-type target operator in the normalized operator sequence is adjusted so that the slice-type target operator is pre-processed.

3. The optimization method according to claim 1 or 2, wherein the operator sequence in a specific order comprises a continuous sequence of deformation operators and slicing operators.

4. The optimization method according to any one of claims 1 to 3, wherein before adjusting the position of the slicing operator and / or the deformation operator in the operator sequence so that the slicing operator is pre-processed, the optimization method further comprises: Determine a dimensional mapping relationship between an input tensor and an output tensor of the deformation operator according to a relationship between a dimensional value of the input tensor and a dimensional value of the output tensor of the deformation operator, wherein the dimensional value represents the amount of data in each dimension; and According to the dimension mapping relationship corresponding to the slice dimension of the slice operator, it is determined whether to adjust the position of the slice operator and / or the deformation operator so that the slice operator is pre-processed.

5. The optimization method according to claim 4, wherein determining the dimensional mapping relationship between the input tensor and the output tensor of the deformation operator comprises: In response to the presence of corresponding identical dimension values ​​in the input tensor dimension values ​​and the output tensor dimension values, mapping the dimensions to which the corresponding identical dimension values ​​belong in pairs; In response to the presence of non-corresponding dimension values ​​in the input tensor dimension values ​​and the output tensor dimension values, the dimensions to which the products of the non-corresponding dimension values ​​correspond to the same dimension values ​​are associated and mapped.

6. The optimization method according to claim 5, wherein determining whether to adjust the position of the slicing operator and / or the deformation operator so that the slicing operator pre-processing comprises: In response to the dimension mapping relationship corresponding to the slice dimension being a paired mapping, adjusting the position of the slice operator and / or the deformation operator so that the slice operator is pre-processed; In response to the dimension mapping relationship corresponding to the slice dimension being an associative mapping, it is determined whether to perform pre-processing on the slice class operator according to a mapping type of the associative mapping corresponding to the slice dimension.

7. The optimization method according to claim 6, wherein determining whether to perform pre-processing on the slice type operator comprises: In response to the mapping type being a split mapping and the parameters of the slice-type operator satisfying a first preset condition, the parameters of the slice-type operator are adjusted for pre-processing, wherein the split mapping includes splitting the dimensional values ​​of the input tensor into multiple dimensional values ​​of the output tensor.

8. The optimization method according to claim 7, wherein The first preset condition includes: start+step≥end; as well as The adjusted parameters include: step new =value2, start new =start, end new =(value1-1)*value2+end; Among them, start represents the original slice starting position of the slice operator, step represents the original data jump step of the slice operator, end represents the original slice ending position of the slice operator, step new Indicates the new data jump step of the adjusted slice operator, start new Indicates the new slice starting position of the adjusted slice operator, end new It indicates the new slice end position of the adjusted slice operator, value1 indicates the first output tensor dimension value obtained by slicing in the slicing map, and value2 indicates the second output tensor dimension value obtained by slicing in the slicing map.

9. The optimization method according to claim 7, wherein The first preset condition includes: start+step<end, and value2-(end-start-1+step)=0; as well as The adjusted parameters include: step new =step start new =start end new =(value1-1)*value2+end; Among them, start represents the original slice starting position of the slice operator, step represents the original data jump step of the slice operator, end represents the original slice ending position of the slice operator, step new Indicates the new data jump step of the adjusted slice operator, start new Indicates the new slice starting position of the adjusted slice operator, end new It indicates the new slice end position of the adjusted slice operator, value1 indicates the first output tensor dimension value obtained by slicing in the slicing map, and value2 indicates the second output tensor dimension value obtained by slicing in the slicing map.

10. The optimization method according to claim 2, further comprising: Before performing the position adjustment, multiple consecutive identical target operators are fused into a single target operator; and / or After performing the adjustment position, multiple consecutive identical target operators are fused into a single target operator.

11. The optimization method according to claim 1, further comprising: After performing the repositioning, Normalizing the deformation class operators to specified deformation class target operators whose functions can replace each other; Normalizing the slice-type operators to designated slice-type target operators whose functions can replace each other; as well as According to the normalized results, multiple consecutive identical target operators are fused into a single target operator.

12. The optimization method according to claim 10 or 11, wherein when the multiple consecutive identical target operators are slice operators, fusing the multiple consecutive identical target operators into a single target operator comprises: In response to the multiple target operators operating in the same dimension, determining equivalent parameters of a fused single target operator according to corresponding slicing ranges and data fetching steps of the multiple target operators in the same dimension; In response to the multiple target operators operating in different dimensions, the dimensional operations of the multiple target operators in different dimensions are merged.

13. The optimization method according to claim 12, wherein determining equivalent parameters of the fused single target operator comprises: Determine an equivalent slice starting position of a fused single target operator according to a first data fetching jump step of a target operator that first performs operation among the multiple target operators and a slice starting position of each target operator; Determine an equivalent slice end position of a fused single target operator according to a first data fetch jump step and a first slice start position of a target operator that performs operations first among the multiple target operators, and according to a slice end position of a target operator that performs operations later among the multiple target operators; as well as According to the data acquisition jump step of each target operator in the multiple target operators, an equivalent data acquisition jump step of a fused single target operator is determined.

14. The optimization method according to claim 12 or 13, wherein determining the equivalent parameters comprises: dim new =dim, start new =start1+start2*step1, end new =start1+end2*step1, step new =step1*step2, Among them, dim new represents the equivalent slice dimension after fusion, dim represents the same dimension on which the multiple target operators operate, start new represents the starting position of the equivalent slice after fusion, start1 represents the starting position of the first slice of the target operator that performs the operation first among the multiple target operators, start2 represents the starting position of the second slice of the target operator that performs the operation later among the multiple target operators, and end new represents the end position of the equivalent slice after fusion, end1 represents the end position of the first slice of the target operator that performs the operation first among the multiple target operators, end2 represents the end position of the second slice of the target operator that performs the operation later among the multiple target operators, step new It represents the equivalent data fetching jump step after fusion, step1 represents the first data fetching jump step of the target operator that performs the operation first among the multiple target operators, and step2 represents the second data fetching jump step of the target operator that performs the operation later among the multiple target operators.

15. The optimization method according to claim 2, wherein: The deformation target operator is a reshape operator; The slice type target operator is a slice operator.

16. The optimization method according to claim 1, wherein: The other types of operators include at least one of slicing operators, transposition operators and extension operators.

17. A computing device for optimizing a computing graph, comprising: a processor configured to execute program instructions; as well as A memory configured to store the program instructions, and when the program instructions are loaded and executed by the processor, the processor executes the calculation graph optimization method according to any one of claims 1-16.

18. A computer-readable storage medium having computer-readable instructions stored thereon, which, when executed by one or more processors, implement the calculation graph optimization method as described in any one of claims 1-16.

19. A computer program product, comprising a computer program or instructions, which, when executed by a processor, implements the method for optimizing the computational graph described in any one of claims 1 to 16.

Images