Method and computing device for optimizing memory operation of data flow chip

Abstract

The present invention provides a method and computing device for optimizing memory operations of a data stream chip, which are applied to the field of data processing technology. The method comprises obtaining a compiled model graph; optimizing the model graph by performing padding operations and cutting operations to obtain an optimized model graph; adjusting the order of nodes in the optimized model graph, moving the transposition operation nodes forward; and merging adjacent transposition operation nodes. The present invention can improve the overall reasoning efficiency of the model.

Description

Technical Field

[0001] The present invention relates to the field of neural network and chip technology, and in particular to a method and computing device for optimizing memory operations of data stream chips. Background Art

[0002] In deep learning and neural network models, the transpose operation is often used to adjust the order of tensor dimensions. This operation is very important to ensure that data is input into subsequent layers in the correct format. Especially when processing multidimensional data such as images or videos, the transpose operation can help change the shape of the data to make it more suitable for specific algorithms or hardware optimizations.

[0003] Data alignment is an important consideration when it comes to hardware implementation, especially for data flow chips designed specifically to accelerate deep learning tasks. Chips usually require data to be organized in a specific way to perform calculations efficiently. However, the continuity of the transposition operation may be interrupted during the data padding and removal operations of the model nodes.

[0004] Therefore, a method for optimizing data flow chip memory operations is needed to improve the overall reasoning performance of the model. Summary of the invention

[0005] The present invention aims to provide a method and computing device for optimizing memory operations of data flow chips, which helps to simplify the structure of model graphs and improve overall performance.

[0006] According to one aspect of the present invention, a method for optimizing memory operations of a data stream chip is provided, comprising:

[0007] Get the compiled model graph;

[0008] Performing padding and cutting operations on the model graph to optimize the model graph to obtain an optimized model graph;

[0009] Adjusting the order of nodes in the optimized model graph and moving the transposition operation node forward;

[0010] Merge adjacent transpose operation nodes.

[0011] According to some embodiments, adjusting the order of transposition operation nodes in the optimized model graph includes:

[0012] Traverse the nodes of the model graph, collect all transposition operation nodes, and add them to the node list;

[0013] The node list is traversed, and if the input node of the transposition operation node is a cutting operation node or a padding operation node and the transposition operation node is the only successor node of the input node, the order of the input node and the transposition operation node is exchanged.

[0014] According to some embodiments, exchanging the order of the input node and the transposition operation node includes:

[0015] Obtaining input data and output data dimension information of the input node;

[0016] Obtaining a transposition attribute of the transposition operation node;

[0017] Creating a second transposition operation node using the input data and the transposition attribute, wherein the second transposition operation node is located after the input data in the optimized model graph;

[0018] According to the type of the input node, using the attribute of the input node and the second transposition operation node to create a second node of a corresponding type, wherein the second node is located after the second transposition operation node in the optimized model graph;

[0019] The input node and the transpose operation node are removed from the optimized model graph.

[0020] According to some embodiments, creating a second transposition operation node using the input data and the transposition attribute includes:

[0021] The input data is used as input data of the second transposition operation node, and axis mapping is performed according to the transposition attribute to create the second transposition operation node.

[0022] According to some embodiments, after obtaining the transposition attribute of the transposition operation node, the method further includes:

[0023] An axis mapping relationship from the input node to the transposition operation node is established according to the output data dimension information and the transposition attribute of the transposition operation node.

[0024] According to some embodiments, creating a second node of a corresponding type using an attribute of the input node and the second transposition operation node according to the type of the input node includes:

[0025] If the type of the input node is a completion node, then the original completion attribute is retrieved;

[0026] Calculating a second padding attribute according to the original padding attribute and the axis mapping relationship;

[0027] A second padding node is created according to the second padding attribute, wherein an input of the second padding node comes from an output of the second transposition operation node.

[0028] According to some embodiments, calculating a second padding attribute according to the original padding attribute and the axis mapping relationship includes:

[0029] According to the axis mapping relationship, the padding axis in the original padding attribute is transformed into the corresponding axis of the second transposition operation node, and the padding information in the original padding attribute remains unchanged.

[0030] According to some embodiments, creating a second node of a corresponding type using an attribute of the input node and the second transposition operation node according to the type of the input node includes:

[0031] If the type of the input node is a cut node, then the original cut attribute is retrieved;

[0032] Calculating a second resection attribute according to the original resection attribute and the axis mapping relationship;

[0033] A second resection node is created according to the second resection attribute, wherein an input of the second resection node comes from an output of the second transpose operation node.

[0034] According to some embodiments, calculating a second resection attribute according to the original resection attribute and the axis mapping relationship comprises:

[0035] According to the axis mapping relationship, the resection axis in the original resection attribute is transformed into the corresponding axis of the second transposition operation node, and the resection information in the original resection attribute remains unchanged.

[0036] According to another aspect of the present invention, there is provided a computing device, comprising:

[0037] Processor; and

[0038] A memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform any of the aforementioned methods.

[0039] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, any of the above-mentioned methods is implemented.

[0040] According to an example embodiment of the present invention, by moving the transposition operation forward, multiple transposition operations between intermediate nodes can be reduced, thereby reducing the amount of calculation and the number of memory accesses. By merging adjacent transposition operation nodes, the model graph structure can be simplified, the number of nodes can be reduced, and the model can be easier to understand and maintain. The optimized model graph is more concise and easy to debug and optimize.

[0041] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below.

[0043] Figure 1 A flow chart of a method for optimizing memory operations of a data stream chip according to an example embodiment is shown.

[0044] Figure 2 A flow chart of a method for changing the order of nodes in a data flow chip according to an example embodiment is shown.

[0045] Figure 3 A schematic diagram showing a change in the order of nodes of a data flow chip according to an example embodiment is shown.

[0046] Figure 4 A flow chart of a method for input node sequence exchange according to an example embodiment is shown.

[0047] Figure 5 A block diagram of a computing device according to an example embodiment of the invention is shown. DETAILED DESCRIPTION

[0048] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0049] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0050] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0051] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0052] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the present inventive concept. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0053] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0054] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present invention, and therefore cannot be used to limit the protection scope of the present invention.

[0055] In the data padding and cropping operations of the model nodes, usually only the memory-aligned cropping operation of a single node is considered, which will disrupt the continuity of the original transposition operation. That is, the two original consecutive transposition operations are fused into one transposition operation to reduce memory operations. However, after the data padding operation (Pad) and the data cropping operation (Crop), the continuity of the transposition operation may be interrupted, thereby affecting the fusion operation and ultimately the overall reasoning performance.

[0056] After optimizing the padding and cutting operations on the model graph, the present invention solves the problem that the padding and cutting operations destroy the continuity of the transpose operation by adjusting the order of nodes in the model graph, ensuring that the padding and cutting operations do not affect the fusion of continuous Transpose, thereby achieving the purpose of computing acceleration.

[0057] Before describing the embodiments of the present application, some terms or concepts involved in the embodiments of the present application are explained.

[0058] Pad (padding operation) adds additional elements on the specified axis to meet the hardware's requirements for data alignment.

[0059] Crop (resection operation) removes some elements on the specified axis in order to trim away unnecessary data.

[0060] Transpose changes the order of the dimensions of a tensor.

[0061] Exemplary embodiments of the present invention are described below with reference to the accompanying drawings.

[0062] Figure 1 A flow chart of a method for optimizing memory operations of a data stream chip according to an example embodiment is shown.

[0063] See also Figure 1 The advantages of the original model graph after the insertion of padding and cutting operations are flexibility, robustness and data preprocessing capabilities. Adjusting the order of nodes in the optimized model graph and moving the transpose operation node forward can effectively solve the problem that the padding and cutting operations affect the continuity of the transpose operation.

[0064] In S101, a graph after model compilation is obtained.

[0065] In order to optimize the graph after model compilation, first, it is necessary to obtain the graph structure after model compilation. According to some embodiments, this can usually be achieved through an API provided by a model compilation tool or framework.

[0066] In S103, the model graph is optimized by performing a filling operation and a cutting operation to obtain an optimized model graph.

[0067] For application scenarios that need to process input data of different sizes or have a high tolerance for abnormal data, padding and cropping operations can be performed on the basis of the original model graph. The padding and cropping operations themselves need to specify the padding and cropping sizes on a specific axis, but do not change the meaning of the corresponding dimensions or the number of dimensions.

[0068] According to some embodiments, all nodes of the model graph are traversed, all the completion operation nodes are collected and added to a set, the first node set is traversed, and if the predecessor node of the completion operation node is a removal operation node and the completion operation node is the only successor node of the removal operation node, the removal operation node and the completion operation node are merged. The compiled model graph is optimized by the completion operation and the removal operation to obtain an optimized model graph.

[0069] In S105, the order of the nodes in the optimized model graph is adjusted to move the transposition operation node forward.

[0070] According to an example embodiment, adjusting the order of transposition operation nodes in the optimized model graph includes traversing the nodes of the model graph, collecting all transposition operation nodes, and adding them to a node list; traversing the node list, if the input node of the transposition operation node is a cut operation node or a padding operation node and the device operation node is the only successor node of the input node, then exchanging the order of the input node and the transposition operation node.

[0071] The node order of the transposition operation nodes in the model graph that has undergone the padding operation and the cutting operation is adjusted. Each transposition operation involves a large amount of memory access. By adjusting the node order, the number of memory accesses can be reduced, thereby improving memory access efficiency.

[0072] In S107 , adjacent transposition operation nodes are merged.

[0073] According to the example embodiment, merging adjacent transposition operation nodes can reduce unnecessary calculation steps, improve calculation efficiency, reduce the number of nodes in the model graph, make the model graph more concise, and improve readability and maintainability.

[0074] In this example embodiment, the order of the nodes in the optimized model graph is adjusted by this method, and the transposition operation node is moved forward to ensure that Pad and Crop do not affect the fusion of continuous Transpose, thereby retaining data processing capabilities and achieving the purpose of accelerating calculations. By merging adjacent transposition operation nodes, the execution efficiency and resource utilization of the model can be significantly improved while maintaining the model function.

[0075] Figure 2 A flow chart of a method for changing the order of nodes in a data flow chip according to an example embodiment is shown.

[0076] See also Figure 2 According to an example embodiment, the node list is traversed, and if the input node of the transposition operation node is a resection operation node or a padding operation node and the device operation node is the only successor node of the input node, the order of the input node and the transposition operation node is exchanged, and the order of the input node and the transposition operation node is exchanged.

[0077] In S201, the input data and output data dimension information of the input node are obtained.

[0078] According to some embodiments, the input data and output data dimension information of the input node are obtained, it is determined that the input node to be optimized is a Pad or Crop node, and the output data dimension information of the node is obtained.

[0079] In S203, the transposition attribute of the transposition operation node is obtained.

[0080] According to an example embodiment, a transposition operation node directly connected to an input node is obtained, and after obtaining the transposition attribute of the transposition operation node, an axis mapping relationship from the input node to the transposition operation node is established according to the output data dimension information and the transposition attribute of the transposition operation node.

[0081] In S205, a second transposition operation node is created using the input data and the transposition attribute, wherein the second transposition operation node is located after the input data in the optimized model graph.

[0082] The second transposition operation node is created by using the input data and the transposition attribute, the input data is used as the input data of the second transposition operation node, and the axis mapping is performed according to the transposition attribute to create the second transposition operation node. The original model diagram and the schematic diagram after exchanging the order of the input node and the transposition operation node are shown in Figure 3 .

[0083] In S207, according to the type of the input node, a second node of a corresponding type is created using the attributes of the input node and the second transposition operation node, and the second node is located after the second transposition operation node in the optimized model graph.

[0084] According to the type of the input node, a second node of the corresponding type is created using the attributes of the input node and the second transposition operation node; if the type of the input node is a padding node, the original padding attribute is taken out, and a second padding attribute is calculated according to the original padding attribute and the axis mapping relationship; a second padding node is created according to the second padding attribute, the input of the second padding node comes from the output of the second transposition operation node, and according to the axis mapping relationship, the padding axis in the original padding attribute is transformed into the corresponding axis of the second transposition operation node, and the padding information in the original padding attribute remains unchanged.

[0085] According to the type of the input node, a second node of the corresponding type is created using the attributes of the input node and the second transposition operation node. If the type of the input node is a resection node, the original resection attribute is taken out; a second resection attribute is calculated according to the original resection attribute and the axis mapping relationship; a second resection node is created according to the second resection attribute, and the input of the second resection node comes from the output of the second transposition operation node.

[0086] According to the axis mapping relationship, the resection axis in the original resection attribute is transformed into the corresponding axis of the second transposition operation node, and the resection information in the original resection attribute remains unchanged.

[0087] In S209, the input node and the transposition operation node are removed from the optimized model graph.

[0088] The connection relationship in the model graph is updated, and the input node and the transposition operation node are removed from the optimized model graph.

[0089] The present invention can significantly improve the performance and efficiency of the model by acquiring a model graph that has been subjected to a padding operation and a cutting operation, and performing a fusion of a transposition operation on the basis of the model graph that has been subjected to a padding operation and a cutting operation. By moving the order of the subsequent transposition operation nodes forward, the preceding transposition operation node and the following transposition operation node become two adjacent transposition operation nodes, and then fusing the two adjacent transposition operation nodes into one transposition operation node, the merged transposition operation can better utilize the cache and memory layout, and improve the memory access speed and model execution efficiency.

[0090] This example embodiment can better handle input data of different sizes and enhance the flexibility of the model by adjusting the node order. Adjusting the node order can facilitate the subsequent reduction of the number of nodes in the model graph, making the model graph more concise. The simplified model graph is easier to understand and maintain, and is convenient for debugging and optimization.

[0091] By adjusting the order of Pad and Crop operations and adjusting the parameters of these operations accordingly, two originally non-adjacent Transpose operations can be made adjacent, thereby achieving fusion, reducing memory operations, and improving overall reasoning performance. This method is very effective in practical applications, especially in the optimization process of deep learning models.

[0092] According to some embodiments, the transposition operation nodes are merged, and after the model is compiled and necessary padding operations (Pad) and cropping operations (Crop) are performed on each node, the Pad and Crop in the model are merged and eliminated.

[0093] According to the embodiment, the entire model graph structure is traversed, all transposition operation nodes are collected, and added to the node list, all nodes in the model graph are traversed, all transposition operation nodes are collected, and for each node, its operation type is checked to see if it is a transposition operation node "Transpose", and if it is a transposition operation node, it is added to the node list (AllTransposes).

[0094] For each transposition operation node in the node list, determine the input node type of the transposition operation node, and determine whether the transposition operation node is the only successor node of the input node. For each transposition operation node in the node list AllTransposes, determine the input node type of the transposition operation node, for example, obtain the input node input node in_A of the transposition operation node transpose_A, check whether the operation type of the input node in_A is the padding operation node Pad or the cutting operation node Crop, and determine whether the transposition operation node transpose_A is the only successor node of the input node in_A. If the input node in_A has multiple successor nodes, skip the transposition operation node transpose_A. If the transposition operation node transpose_A is the only successor node of the input node in_A, the order of the transposition operation node transpose_A and the input node in_A can be exchanged, and this step is repeated until the node list AllTransposes is empty.

[0095] All the transposition operation nodes that can be exchanged are moved forward, and the adjacent transposition operation nodes are merged into one transposition operation node. Each transposition operation node transpose_A in the AllTransposes node list is traversed, and all the transposition operation nodes that can be exchanged are moved forward, and the adjacent transposition operation nodes are merged into one transposition operation node.

[0096] During the optimization process, it is ensured that the transpose operation nodes are correctly moved forward and adjacent transpose operation nodes are merged into one transpose operation node, thereby simplifying the model graph structure and improving efficiency.

[0097] Figure 4 A flow chart of a method for swapping the order of input nodes according to an example embodiment is shown.

[0098] Traverse the entire model graph structure, collect all Transpose nodes, put them into the node list AllTransposes, and perform the following operations for each node transpose_A in AllTransposes.

[0099] In S301, output data dimension information of an input node is obtained, and input data information of an input node is obtained.

[0100] According to an example embodiment, the output data dimension information rank of the input node in_A is obtained. If the output shape of the input node in_A is [2, 4, 4, 3], rank=4, and the input data in_A_input of the input node in_A is obtained at the same time.

[0101] In S303, the transposition attribute of the transposition operation node is obtained, and the axis mapping relationship is calculated.

[0102] Get the transposition attribute perm of the transposition operation node transpose_A, for example, perm = [0, 2, 3, 1], calculate the axis mapping relationship axis_mapping, initialize an axis_mapping with a length of rank, where rank is the output data dimension of the input node in_A, and use pseudo code to calculate the axis mapping relationship axis_mapping. According to some embodiments, the pseudo code of the calculation is as follows:

[0103] for(int64_t ii = 0; ii <rank; ii++) axis_mapping[perm[ii]] = ii;

[0104] Specific calculation process:

[0105] When ii = 0, perm[0] = 0, so axis_mapping[0] = 0

[0106] When ii = 1, perm[1] = 2, so axis_mapping[2] = 1

[0107] When ii = 2, perm[2] = 3, so axis_mapping[3] = 2

[0108] When ii = 3, perm[3] = 1, so axis_mapping[1] = 3

[0109] The axis mapping relationship axis_mapping is obtained as [0, 3, 1, 2].

[0110] In S305, a second transposition operation node is created.

[0111] According to an example embodiment, a second transposition operation node is created, and if the node type of the input node is a padding operation, the padding attribute is retrieved and a second padding attribute is calculated; if the node type of the input node is a cut attribute, the cut attribute is retrieved and a second cut attribute is calculated.

[0112] For example, a second transpose operation node new_transpose_A is created, the input of the second transpose operation node new_transpose_A is in_A_input, the transposition attribute is perm, and the second transpose operation node new_transpose_A= Transpose(in_A_input, perm); the node type is determined to be a Pad or Crop node according to the type of the input node in_A.

[0113] If the node type of the input node in_A is Pad, the padding attributes p_axes and p_sizes are retrieved, and the second padding attribute new_p_axes is calculated. According to some embodiments, the calculation method is as follows:

[0114] for(int64_t ii=0;ii <p_axes.size();ii++)

[0115] new_p_axes[ii]= axis_mapping[p_axes[ii]];

[0116] Create a second Pad node pad_or_crop, with input as in_A_input and transposition attribute as perm, pad_or_crop = Pad(new_ transpose_A, new_p_axes, p_sizes); The input of the second Pad node is new_ transpose_A, and the padding attributes are new_p_axes and p_sizes.

[0117] If the node type of the input node in_A is Crop, take out the cut attributes c_axes and c_sizes, and calculate the second cut attribute new_c_axes, the calculation method is as follows:

[0118] for (int64_t ii = 0; ii <c_axes.size(); ii++)

[0119] new_c_axes[ii] = axis_mapping[c_axes[ii]];

[0120] Create a second Crop node pad_or_crop, pad_or_crop = Crop(new_ transpose_A, new_c_axes, c_sizes); the input of the second Crop node is new_ transpose_A, and the cutting attributes are new_c_axes and c_sizes.

[0121] In S307, the original transposition operation node and its input nodes are removed from the model graph.

[0122] Update the connection relationship in the model graph, replace the input of the nodes that use the transpose operation node transpose_A as input with pad_or_crop, and remove the transpose operation node transpose_A from the model graph. Update the output node of in_A_input to point to the second transpose operation node new_transpose_A, update the output node of the second transpose operation node new_ transpose_A to point to pad_or_crop, and update the output node of pad_or_crop to point to the output node of the transpose operation node transpose_A.

[0123] This embodiment creates a new node according to the type of input node, and adjusts its attributes to adapt to the new transposition order; by exchanging the positions of the transposition operation nodes and the input nodes, the order of the transposition operation nodes and the input nodes is exchanged to ensure the logical correctness of the model graph, and then the connection relationship in the model graph is updated, and the old transposition operation nodes are removed to ensure that the input and output relationships of all nodes are correct, thereby ensuring that the model graph structure is more efficient and concise during the data optimization process.

[0124] Another example of the technical solution according to the present invention is described below.

[0125] For example, the original node order is Input([1, 4, 6, 3])->Transpose(input, perm=[0,2, 3, 1])->Pad(input, p_axes=[3], p_sizes=[2, 3])->Transpose(input, [0, 3, 1,2])->Output([1, 9, 6, 3]).

[0126] In this exemplary embodiment, the input represents the output of the previous node, that is, the input of the current node, which is optimized according to the method of changing the order of data flow chip nodes proposed by the present invention:

[0127] The first step is to collect all the transpose operation nodes.

[0128] Transpose(input, perm=[0, 2, 3, 1]);

[0129] Transpose(input, perm=[0, 3, 1, 2]).

[0130] The second step is to determine the input node type of the transposition operation node and whether the transposition operation node is the only successor node of the input node.

[0131] The first transpose operation node Transpose(input, perm=[0, 2, 3, 1]), the input node Input([1, 4, 6, 3]), is not a Pad or Crop node, so it is skipped; the second transpose operation node Transpose(input,perm=[0, 3, 1, 2]), the input node Pad(input, p_axes=[3], p_sizes=[2, 3]) is a Pad node, and Transpose(input, perm=[0, 3, 1, 2]) is the only successor node of Pad(input, p_axes=[3], p_sizes=[2, 3]), so the order can be exchanged;

[0132] After swapping the order of the transpose operation nodes and the input nodes, the optimized model graph becomes:

[0133] Input([1, 4, 6, 3])->Transpose(input, perm=[0, 2, 3, 1])->Transpose(input, 0, 3, 1, 2])->Pad(input, p_axes=[1], p_sizes=[2, 3])->Output([1, 9,6, 3]).

[0134] In the third step, adjacent Tranposes can be further fused by merging adjacent transpose operation nodes and calculating the combined transpose attributes, perm1 = [0, 2, 3, 1], perm2 = [0, 3, 1, 2], combined_perm[ii] = perm1[perm2[ii]], and combined_perm is [0, 1, 2, 3].

[0135] Update the connection relationship in the model diagram to obtain:

[0136] Input([1, 4, 6, 3])->Transpose(input, perm=[0, 1, 2, 3])>Pad(input,p_axes=[1], p_sizes=[2, 3])->Output([1, 9, 6, 3]).

[0137] According to some embodiments, because Transpose(input, [0, 1, 2, 3]) is equivalent to not transposing the input, the input can be directly connected to the output, and the final optimized model graph is:

[0138] Input([1, 4, 6, 3])->Pad(input, p_axes=[1], p_sizes=[2, 3])->Output[1, 9, 6, 3]).

[0139] This example embodiment successfully moves the transposition operation nodes in the original model graph forward by changing the order of the data flow chip nodes, and merges adjacent transposition operation nodes, thereby ultimately simplifying the model graph structure.

[0140] Figure 5 A block diagram of a computing device according to an example embodiment of the invention is shown.

[0141] like Figure 5 As shown, computing device 30 includes processor 12 and memory 14. Computing device 30 may also include bus 22, network interface 16, and I / O interface 18. Processor 12, memory 14, network interface 16, and I / O interface 18 may communicate with each other via bus 22.

[0142] The processor 12 may include one or more general-purpose CPUs (Central Processing Units, processors), microprocessors, or application-specific integrated circuits, etc., for executing relevant program instructions.

[0143] The memory 14 may include a machine system readable medium in the form of a volatile memory, such as a random access memory (RAM), a read-only memory (ROM) and / or a cache memory. The memory 14 is used to store one or more programs including instructions and data. The processor 12 may read the instructions stored in the memory 14 to execute the above-mentioned method according to the embodiment of the present invention.

[0144] The computing device 30 may also communicate with one or more networks via the network interface 16. The network interface 16 may be a wireless network interface.

[0145] The bus 22 may include an address bus, a data bus, a control bus, etc. The bus 22 provides a path for exchanging information between components.

[0146] It should be noted that, in the specific implementation process, the computing device 30 may also include other components necessary for normal operation. In addition, those skilled in the art may understand that the above device may only include components necessary for implementing the embodiments of this specification, and need not include all components shown in the figure.

[0147] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the program is executed by a processor. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), a network storage device, a cloud storage device, or any type of medium or device suitable for storing instructions and / or data.

[0148] An embodiment of the present invention also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any method recorded in the above method embodiments.

[0149] Those skilled in the art can clearly understand that the technical solution of the present invention can be implemented with the help of software and / or hardware. "Unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array, an integrated circuit, etc.

[0150] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0151] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0152] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0153] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0154] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0155] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.

[0156] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0157] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended clauses.

[0158] Those skilled in the art can clearly understand that the technical solution of the present invention can be implemented with the help of software and / or hardware. "Unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array, an integrated circuit, etc.

[0159] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0160] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0161] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0162] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0163] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0164] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.

[0165] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0166] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended clauses.

Claims

1. A method for optimizing memory operations of a data stream chip, characterized in that: include: Get the compiled model graph; Performing padding and cutting operations on the model graph to optimize the model graph to obtain an optimized model graph; Adjusting the order of nodes in the optimized model graph and moving the transposition operation node forward; Merge adjacent transpose operation nodes, Wherein, adjusting the order of transposition operation nodes in the optimized model graph includes: Traverse the nodes of the model graph, collect all transposition operation nodes, and add them to the node list; Traverse the node list, if the input node of the transposition operation node is a cutting operation node or a padding operation node and the transposition operation node is the only successor node of the input node, then swap the order of the input node and the transposition operation node: Obtaining input data and output data dimension information of the input node; Obtaining a transposition attribute of the transposition operation node; Creating a second transposition operation node using the input data and the transposition attribute, wherein the second transposition operation node is located after the input data in the optimized model graph; According to the type of the input node, using the attribute of the input node and the second transposition operation node to create a second node of a corresponding type, wherein the second node is located after the second transposition operation node in the optimized model graph; The input node and the transpose operation node are removed from the optimized model graph.

2. The method according to claim 1, characterized in that Creating a second transposition operation node using the input data and the transposition attribute includes: The input data is used as input data of the second transposition operation node, and axis mapping is performed according to the transposition attribute to create the second transposition operation node.

3. The method according to claim 1, characterized in that After obtaining the transposition attribute of the transposition operation node, the method further includes: An axis mapping relationship from the input node to the transposition operation node is established according to the output data dimension information and the transposition attribute of the transposition operation node.

4. The method according to claim 3, characterized in that According to the type of the input node, using the attribute of the input node and the second transposition operation node to create a second node of a corresponding type, comprising: If the type of the input node is a completion node, then the original completion attribute is retrieved; Calculating a second padding attribute according to the original padding attribute and the axis mapping relationship; A second padding node is created according to the second padding attribute, wherein an input of the second padding node comes from an output of the second transposition operation node.

5. The method according to claim 4, characterized in that Calculating a second padding attribute according to the original padding attribute and the axis mapping relationship includes: According to the axis mapping relationship, the padding axis in the original padding attribute is transformed into the corresponding axis of the second transposition operation node, and the padding information in the original padding attribute remains unchanged.

6. The method according to claim 3, characterized in that According to the type of the input node, using the attribute of the input node and the second transposition operation node to create a second node of a corresponding type, comprising: If the type of the input node is a cut node, then the original cut attribute is retrieved; Calculating a second resection attribute according to the original resection attribute and the axis mapping relationship; A second resection node is created according to the second resection attribute, wherein an input of the second resection node comes from an output of the second transpose operation node.

7. The method according to claim 6, characterized in that Calculating a second resection attribute according to the original resection attribute and the axis mapping relationship includes: According to the axis mapping relationship, the resection axis in the original resection attribute is transformed into the corresponding axis of the second transposition operation node, and the resection information in the original resection attribute remains unchanged.

8. A computing device, characterized in that include: processor; as well as A memory storing a computer program, which, when executed by the processor, enables the processor to perform the method according to any one of claims 1 to 7.

9. A computer program product, characterized in that The method comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images