Method, system and computing device for optimizing memory operation of data flow chip

Abstract

The present invention provides a method and computing device for optimizing memory operations of a data stream chip, which are applied to the field of data processing technology. The method comprises obtaining a graph compiled from a model; traversing the nodes of the graph, collecting completion operation nodes, and adding them to a first node set; traversing the first node set, if the predecessor node of the completion operation node is a removal operation node and the completion operation node is the only successor node of the removal operation node, then merging the removal operation node with the completion operation node; determining a new node obtained after merging according to the result of the merging operation, wherein the new node is an empty node, a removal operation node, a completion operation node, or a node having both removal and completion operations. The present invention can improve the overall reasoning efficiency of the model.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically to a method and computing device for optimizing memory operations of data stream chips. Background Technology

[0002] In artificial intelligence model computation, in order to match the data alignment requirements of the data flow chip, some nodes running on the chip are usually padded with data. After the computation, the excess data is removed, so as to meet the hardware's data alignment requirements and ensure the correctness of the model computation results.

[0003] In the data completion and cut-off operations of model nodes, only the memory alignment of a single node is usually considered for the cut-off operation, without optimizing the relationship between the preceding and following nodes. Therefore, frequent completion and cut-off operations may occur, affecting the overall inference efficiency of the model.

[0004] Therefore, a method is needed to optimize dataflow chip memory operations to improve the overall inference efficiency of the model. Summary of the Invention

[0005] The present invention aims to provide a method and computing device for optimizing memory operations of data stream chips, which helps to improve the inference efficiency of models, reduce memory consumption, and improve overall performance.

[0006] According to one aspect of the present invention, a method for optimizing memory operations in a data stream chip is provided, comprising:

[0007] Obtain the graph after the model is compiled;

[0008] Traverse the nodes of the graph, collect the nodes to be completed, and add them to the first node set;

[0009] Traverse the first set of nodes. If the predecessor of the completion operation node is the cut operation node and the completion operation node is the only successor of the cut operation node, then merge the cut operation node and the completion operation node.

[0010] The new node obtained after merging is determined based on the result of the merge operation. The new node can be an empty node, a node that has undergone a cut operation, a node that has undergone a completion operation, or a node that has undergone both cut and completion operations.

[0011] According to some embodiments, merging the cut operation node and the fill operation node includes:

[0012] The padding information and cut information on the corresponding axes of the padding operation node and the cut operation node are merged to obtain a mapping dictionary of axis information and merged information as key-value pairs;

[0013] Process the key-value pairs in the mapping dictionary to obtain the padding axis information and the corresponding padding information and / or the cut axis information and the corresponding cut information based on the merging information.

[0014] According to some embodiments, the padding information and cut information on the corresponding axes of the padding operation node and the cut operation node are merged to obtain a mapping dictionary of axis information and merged information as key-value pairs, including:

[0015] Extract all the completion axis information and completion information from the completion operation node, and extract all the cut axis information and cut information from the cut operation node;

[0016] Each value in the padding axis information is used as a key, and the corresponding padding information is used as data to create a mapping, which is then stored in the mapping dictionary.

[0017] Each value in the cut axis information is used as a key. If the key in the cut axis information exists in the mapping dictionary, the value of the corresponding key in the mapping dictionary is updated using the corresponding cut information.

[0018] According to some embodiments, processing key-value pairs in a mapping dictionary includes:

[0019] When all merged information is less than 0, the corresponding axis is filled in, the corresponding axis is added to the fill axis, and the merged information is added to the fill information.

[0020] According to some embodiments, processing key-value pairs in a mapping dictionary includes:

[0021] When all merged information is less than 0, the corresponding axis is a cut operation, the corresponding axis is added to the cut axis, and the positive merged information is added to the cut information.

[0022] According to some embodiments, processing key-value pairs in a mapping dictionary further includes:

[0023] If the first part of the merged information is not greater than 0 and the second part of the merged information is not less than 0, then the corresponding axis is a cut and fill operation. The corresponding axis is added to the cut axis, the first part of the merged information is positive and the second part of the merged information is 0, and it is added to the cut information. The corresponding axis is also added to the fill axis, the first part of the merged information is 0 and the second part of the merged information remains unchanged, and it is added to the fill information.

[0024] The second part of the merged information is not greater than 0 and the first part of the merged information is not less than 0. The corresponding axis is a cut and fill operation. The corresponding axis is added to the cut axis. The first part of the merged information is set to 0 and the second part of the merged information is set to positive and added to the cut information. The corresponding axis is added to the fill axis. The first part of the merged information remains unchanged and the second part of the merged information is set to 0 and added to the fill information.

[0025] According to some embodiments, the new node obtained after merging is determined based on the result of the merging operation, including:

[0026] If both the completion information and the cut information are empty, or both have a value of 0, then the fusion result is an empty operation, the new node is an empty node, and the input of the cut operation node is directly output to the successor node of the completion operation node.

[0027] According to some embodiments, the new node obtained after merging is determined based on the result of the merging operation, including:

[0028] If the cut-off information is empty, or the values ​​of the cut-off data are all 0, then the fusion result is the completion operator, and the new node is the completion operation node; and / or

[0029] If the padding information is empty, or the values ​​of the padding data are all 0, then the fusion result is the cut-off operator, and the new node is the cut-off operation node.

[0030] According to some embodiments, the new node obtained after merging is determined based on the result of the merging operation, including:

[0031] If neither the completion information nor the cut information is empty and the value of the operation data is not 0, then the node is a composite operation, which includes both cut and completion operations. The new node is a node that combines cut and completion operations.

[0032] According to another aspect of the present invention, a computing device is provided, comprising:

[0033] Processor; and

[0034] A memory that stores a computer program, which, when executed by the processor, causes the processor to perform any of the aforementioned methods.

[0035] According to an exemplary embodiment of the present invention, the compiled graph of the model is obtained. By traversing the entire model graph, it can be ensured that all possible completion operation nodes are identified, and no potential optimization target is missed. The collected completion operation nodes are accurately collected into a set, providing a reliable foundation for subsequent optimization steps. Traversing the first node set and merging completion operation nodes and cut operation nodes can reduce intermediate data copying operations, thereby reducing unnecessary memory access and data copying. By generating new nodes, the number of nodes in the graph is reduced, making the computation graph more concise and easier to manage and debug.

[0036] According to the example embodiment, the completion information and cut information on the corresponding axes of the completion operation node and the cut operation node are merged. By merging the information, the number of times each axis is processed separately in subsequent steps is reduced, which improves the computational efficiency. The axis information and the merged information are used as a key-value pair mapping dictionary, which reduces the storage and processing of redundant information. The mapping dictionary makes the data structure more compact and easier to manage and query.

[0037] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit the invention. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0039] Figure 1 A flowchart illustrating a method for optimizing dataflow chip memory operations according to an example embodiment is shown.

[0040] Figure 2 A flowchart illustrating the method for obtaining a new node according to an example embodiment is shown.

[0041] Figure 3 A flowchart illustrating a method for merging nodes according to an example embodiment is shown.

[0042] Figure 4 A block diagram of a computing device according to an exemplary embodiment of the present invention is shown. Detailed Implementation

[0043] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that the invention will be thorough and complete, and the concept of the exemplary embodiments will be fully conveyed to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0044] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of the invention. However, those skilled in the art will recognize that the technical solutions of the invention can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of the invention.

[0045] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0046] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0047] It should be understood that although the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of the present invention. As used herein, the term "and / or" includes all combinations of any one and more of the associated listed items.

[0048] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0049] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily essential for implementing the present invention, and therefore cannot be used to limit the scope of protection of the present invention.

[0050] This invention optimizes the completion and cutting operations of the original graph structure of the artificial intelligence model. By optimizing the completion and cutting operations of adjacent nodes and eliminating redundant completion and cutting operations, the overall execution efficiency of the model is optimized, thereby achieving the purpose of accelerating computation.

[0051] Before describing the embodiments of this application, some terms or concepts involved in the embodiments of this application will be explained.

[0052] MLIR (Multi-Level Intermediate Representation) is a powerful framework for building interoperable compiler infrastructure. It provides a multi-level intermediate representation that enables developers to perform various graph-level optimizations efficiently.

[0053] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention.

[0054] Figure 1 A flowchart illustrating a method for optimizing dataflow chip memory operations according to an example embodiment is shown.

[0055] In S101, obtain the graph after the model is compiled.

[0056] To optimize the graph after model compilation, it is first necessary to obtain the graph structure after model compilation. According to some implementations, this can usually be achieved through APIs provided by model compilation tools or frameworks, such as the MLIR compilation tool in LLVM.

[0057] In S103, traverse the nodes of the graph, collect the nodes for completion operations, and add them to the first node set.

[0058] According to the example implementation, all nodes of the model graph are traversed, all completion operation nodes are collected, and they are added to a set.

[0059] In S105, traverse the first node set. If the predecessor node of the completion operation node is the cut operation node and the completion operation node is the only successor node of the cut operation node, then perform a merge operation on the cut operation node and the completion operation node.

[0060] According to the example embodiment, each completion operation node in the first node set is traversed, and for each completion operation node, its predecessor node is obtained. The predecessor node is checked to see if its type is a cut operation node. If the predecessor node of the completion operation node is a cut operation node and the completion operation node is the only successor node of the cut operation node, then a merging operation is performed on the cut operation node and the completion operation node.

[0061] According to some embodiments, when merging the cut operation node and the fill operation node, the fill information and cut information on the corresponding axes of the fill operation node and the cut operation node are merged to obtain a mapping dictionary of axis information and merged information as key-value pairs; the key-value pairs in the mapping dictionary are processed to obtain the fill axis information and corresponding fill information and / or the cut axis information and corresponding cut information based on the merged information.

[0062] In S107, the new node obtained after merging is determined based on the result of the merge operation. The new node is an empty node, a node that has undergone a cut operation, a node that has undergone a completion operation, or a node that has undergone both cut and completion operations.

[0063] According to the example implementation, the new node obtained after merging is determined based on the result of the merging operation. The type of the new node includes an empty node, a node that has undergone a cut operation, a node that has undergone a completion operation, or a node that has undergone both cut and completion operations. The new node type can be determined based on the result of the merging operation, thereby optimizing the model graph, reducing redundant operations, and improving the overall execution efficiency of the model.

[0064] Figure 2 A flowchart illustrating the method for obtaining a new node according to an example embodiment is shown.

[0065] See Figure 2 After performing necessary padding and cropping operations on each node during model compilation, the Pads and Crops in the model are merged and eliminated.

[0066] In S201, traverse the entire model graph nodes, collect all completion operation nodes, and add them to the node set.

[0067] According to some embodiments, the graph in the model compilation is based on the MLIR graph representation, where each node in the graph has a node name and a node type name. The nodes collected here are those with the node type name "Pad". Specifically, the set is first initialized by creating an empty set AllPads to store all found padding operation nodes; then, the entire model graph nodes are traversed, and all padding operation nodes are collected and added to the node set AllPads.

[0068] In S203, a completion operation node is taken from the node set, and it is determined whether the predecessor node of the completion operation node is a cut operation and whether the completion operation node is the only successor node of the cut operation node.

[0069] According to the example implementation, a completion operation node A is retrieved from AllPads. It is then determined whether its predecessor node C is a cut operation Crop and whether completion operation node A is the only successor node of cut operation node C. If the type name of node C is not Crop or completion operation node A is not the only successor node of cut operation node C, this node is skipped, and the next completion operation node is processed.

[0070] In S205, the completion operation node and the cut operation node are merged, and the resulting node can be denoted as node M.

[0071] According to the example embodiment, the padding operation node A and the cut operation node C are merged. The padding information and cut information on the corresponding axes of the padding operation node and the cut operation node are merged to obtain a mapping dictionary padcrop_info that uses axis information and merged information as key-value pairs (see [example embodiment]). Figure 2 (As specifically described). An axis is a dimension index; for example, in four-dimensional data, each axis can be 0, 1, 2, or 3.

[0072] Repeat steps S203 and S205 until the AllPads set is empty.

[0073] In S207, key-value pairs in the mapping dictionary are processed.

[0074] According to the example embodiment, the key-value pairs in the mapping dictionary padcrop_info are processed, and the padding axis information and corresponding padding information, and / or the cut axis information and corresponding cut information are obtained based on the merging information.

[0075] According to the example embodiment, the merge information (left, right) corresponding to each axis in the mapping dictionary padcrop_info can be divided into the following cases.

[0076] Case 1: When both left and right are not less than 0, the axis is a pad operation. The axis is added to the padding axis pad_axis, and the merge information (left, right) is added to the padding information pad_sizes.

[0077] Case 2: If the merge information (left, right) is less than 0, the axis is a crop operation. The axis is added to the crop axis, and the positive merge information (-left, -right) is added to the crop information crop_sizes.

[0078] Case 3: If the first part of the merged information (left) is not greater than 0 and the second part of the merged information (right) is not less than 0, then the axis is a crop and pad operation. The axis is added to the crop axis (crop_axis), the first part of the merged information is positive and the second part of the merged information is 0, i.e., (-left, 0) is added to the crop information (crop_sizes), and the axis is added to the pad axis (pad_axis). The first part of the merged information is 0 and the second part of the merged information remains unchanged, i.e., (0, right) is added to the pad information (pad_sizes).

[0079] Case 4: If the second part of the merged information (right) is not greater than 0 and the first part of the merged information (left) is not less than 0, the axis is a crop and pad operation. Add the axis to the crop axis (crop_axis), set the first part of the merged information to 0 and the second part of the merged information to positive (i.e., (0, -right)) to the crop information (crop_sizes), add the axis to the pad axis, and set the first part of the merged information to remain unchanged and the second part of the merged information to 0 (i.e., (left, 0)) to the pad information (pad_sizes).

[0080] In S209, the new node obtained after the merger is determined based on the result of the merge operation.

[0081] According to the example implementation, ensure that all eligible Pad nodes and Crop nodes have been merged, and determine the type name of the new node based on the merging result. For example, it can be an empty node, a cut operation node, a fill operation node, or a node that combines cut and fill operations.

[0082] If both the completion and removal information are empty, or both have a value of 0, then the fusion result is an empty operation, the new node is an empty node, and the input of the removal operation node C can be directly output to the successor node of the completion operation node A.

[0083] If the cut information is empty, or the values ​​of the cut data are all 0, then the fusion result is the padding operator, and the new node is the padding operation node. In this case, the corresponding Pad operator can be created: create Pad(pad_axis, pad_sizes).

[0084] If the padding information is empty, or if all padding values ​​are 0, the fusion result is a Crop operator, and the new node is the cropping operation node. A corresponding Crop operator can be created: `create Crop(crop_axis, crop_sizes)`.

[0085] If both the padding and cropping information are not empty and the value of the operation data is not 0, then the node is a composite operation, involving both cropping (Crop) and padding (Pad). The new node is a node that combines cropping and padding operations. A corresponding cropping and padding operator, CropAndPad, can be created: create CropAndPad(crop_axis, crop_sizes, pad_axis, pad_sizes).

[0086] After the fusion operation, four possible results can be obtained: a Crop node, a Pad node, a CropAndPad node, or an empty node. The input of the Crop node C can be directly fed to the output of the Pad node A. The best-case scenario through this fusion operation is that the Crop and Pad nodes are directly eliminated, saving two computational steps. The worst-case scenario is that both Crop and Pad operations exist simultaneously, and they are merged into a single CropAndPad operation. Although two operations are still required, the graph structure is simplified.

[0087] This invention is based on dataflow chips, which are characterized by high computational parallelism and higher computational efficiency than instruction-based chips. They can perform rapid computation once data is moved to the chip's storage location. However, if memory operations exist between computing nodes, requiring frequent data copying, this disrupts the pipelined advantage of dataflow chips, causing a rapid decrease in computational efficiency. This invention primarily integrates completion and cut operations, potentially eliminating intermediate memory operations and data copying, maintaining the computational continuity of the dataflow, and fully leveraging its computational performance.

[0088] Figure 3 A flowchart illustrating a method for merging nodes according to an example embodiment is shown.

[0089] In S301, retrieve all the completion axis information and completion information from completion operation node A; retrieve all the cut axis information and cut information from cut operation node C.

[0090] According to the example embodiment, all padding axis information A_axes and padding information A_pad_sizes in padding operation node A are retrieved, and all cutting axis information C_axes and cutting information C_crop_sizes in cutting operation node C are retrieved. The padding information A_pad_sizes corresponds to the padding information of each axis in the padding axis information A_axes. For example, for two-dimensional data A(6, 8), which represents 6 rows and 8 columns of data, padding axis information A_axes=[0, 1] and padding information A_pad_sizes=[2,3, 4, 6] means padding 2 rows of data at the top, 3 rows of data at the bottom, 4 columns of data on the left, and 6 columns of data on the right. After the padding operation, node pA(11, 18) becomes 11 rows and 18 columns of data. In the cropping operation, the two-dimensional data C(8, 10) represents 8 rows and 10 columns of data. C_axes=[0, 1] and C_crop_sizes=[1, 2, 3, 4] mean that 1 row is cut from the top, 2 rows are cut from the bottom, 3 columns are cut from the left, and 4 columns are cut from the right. After the cropping operation, the node C(5, 3) becomes 5 rows and 3 columns of data.

[0091] In S303, each value in the padding axis information is used as a key, and the corresponding padding information is used as data to create a mapping, which is then stored in the mapping dictionary.

[0092] According to the example implementation, each value in the padding axis information A is used as a key, and the corresponding padding operation information is used as data (value) to create a mapping, which is stored in the mapping dictionary. Each value in the padding axis information A_axes is used as a key, and the corresponding padding operation data (left, right) is used as data (value) to create a mapping, which is stored in the mapping dictionary padcrop_info. Each number in A_axes corresponds to two numbers in A_pad_sizes, representing the amount of padding data before and after this axis. For example, in the example A_axes=[0, 1], A_pad_sizes=[2, 3, 4, 6], then for axis 0, its left=2, right=3; for axis 1, its left=4, right=6.

[0093] According to some embodiments, the above two-dimensional data A(6,8) is transformed into pA(11, 18) after a padding operation of A_axes=[0, 1] and A_pad_sizes=[2, 3, 4, 6], and stored in the padcrop_info dictionary.

[0094] In S305, each value in the cut axis information is used as a key. If the key in the cut axis information exists in the mapping dictionary, the value of the corresponding key in the mapping dictionary is updated using the corresponding cut information.

[0095] According to the example embodiment, each axis information value in the cut axis information C_axes is used as the key. If axis is in the mapping dictionary padcrop_info, the corresponding value padcrop_info[axis] is retrieved. The mapping dictionary padcrop_info stores the axis as the key and the operation data (left, right) as the value. Pad and Crop are operations corresponding to the axis. Assuming the value is (left, right), the corresponding cut data (cleft, cright) in C_crop_sizes is retrieved, and new_left = left - cleft, new_right = right - cright is set. The new values ​​are then stored in the mapping dictionary padcrop_info, i.e., padcrop_info[axis] = (new_left, new_right).

[0096] According to the method of the present invention, fusing the completion operation node with the removal operation node can effectively reduce memory data copying operations and improve the overall inference efficiency of the model. The method for optimizing memory operations in dataflow chips can not only reduce unnecessary memory accesses but also simplify the computation graph, improving computation speed and energy efficiency.

[0097] According to some implementations, during training, for example, if the image has 3 channels and the data stream chip parallelism is generally 4 or 64, an input data of 1*224*224*3 will be padded to 1*224*224*64 when entering the convolution calculation. If the original weights of the convolution are 3*1*1*3, the weights will be padded to 64*1*1*64, and the new output will be 1*224*224*64. The normal output before padding should be 1*224*224*3, and the extra data needs to be clipped.

[0098] Similar to a production line, data is padded to the subsequent channel dimensions, resulting in 64 channels. In AI processing, an image data set is represented as 1*224*224*3, which represents the batch size. In this example, the batch size is 1, indicating only one image. In practical applications, the batch size is usually greater than 1 to improve computational efficiency and model stability. "224*224" represents the height and width of the image, which is 224 pixels, indicating that the image is a square image. 3 represents the number of color channels, indicating that the image is an RGB image with three color channels: red, green, and blue.

[0099] If the original image size is not 224×224, it is usually necessary to adjust its size by methods such as cropping or resizing to match the model requirements. The batch size can be adjusted according to actual needs to improve computational efficiency.

[0100] Figure 4 A block diagram of a computing device according to an exemplary embodiment of the present invention is shown.

[0101] like Figure 4 As shown, the computing device 30 includes a processor 12 and a memory 14. The computing device 30 may also include a bus 22, a network interface 16, and an I / O interface 18. The processor 12, memory 14, network interface 16, and I / O interface 18 can communicate with each other via the bus 22.

[0102] The processor 12 may include one or more general-purpose CPUs (Central Processing Units), microprocessors, or application-specific integrated circuits, etc., for executing relevant program instructions.

[0103] Memory 14 may include a machine system readable medium in the form of volatile memory, such as random access memory (RAM), read-only memory (ROM), and / or cache memory. Memory 14 is used to store one or more programs containing instructions, as well as data. Processor 12 may read the instructions stored in memory 14 to perform the methods described above according to embodiments of the present invention.

[0104] The computing device 30 can also communicate with one or more networks via the network interface 16. The network interface 16 can be a wireless network interface.

[0105] Bus 22 can include address bus, data bus, control bus, etc. Bus 22 provides a path for exchanging information between components.

[0106] It should be noted that, in specific implementations, the computing device 30 may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the device described above may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0107] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), network storage devices, cloud storage devices, or any type of medium or device suitable for storing instructions and / or data.

[0108] This invention also provides a computer program product comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments.

[0109] Those skilled in the art will clearly understand that the technical solutions of the present invention can be implemented by means of software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware capable of independently performing or cooperating with other components to perform a specific function, wherein the hardware may be, for example, a field-programmable gate array (FPGA), an integrated circuit, etc.

[0110] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0111] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0112] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0114] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0115] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention.

[0116] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0117] Exemplary embodiments of the present invention have been specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, arrangements, or implementations described herein; rather, the present invention is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended provisions.

[0118] Those skilled in the art will clearly understand that the technical solutions of the present invention can be implemented by means of software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware capable of independently performing or cooperating with other components to perform a specific function, wherein the hardware may be, for example, a field-programmable gate array (FPGA), an integrated circuit, etc.

[0119] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0120] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0121] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0122] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0123] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0124] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention.

[0125] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0126] Exemplary embodiments of the present invention have been specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, arrangements, or implementations described herein; rather, the present invention is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended provisions.

Claims

1. A method for optimizing memory operations of a data stream chip, characterized in that: include: Get the graph after model compilation; Traversing the nodes of the graph, collecting and completing operation nodes, and adding them to the first node set; Traversing the first node set, if the predecessor node of the padding operation node is a cutting operation node and the padding operation node is the only successor node of the cutting operation node, merging the cutting operation node with the padding operation node; Determine a new node obtained after merging according to the result of the merging operation, wherein the new node is an empty node, a node for cutting operation, a node for filling operation, or a node for both cutting and filling operation; The merging operation of the cutting operation node and the filling operation node includes: Merge the padding information and the cutting information on the corresponding axes of the padding operation node and the cutting operation node to obtain a mapping dictionary with the axis information and the merged information as key-value pairs; The key-value pairs in the mapping dictionary are processed to obtain the padding axis information and the corresponding padding information and / or the resection axis information and the corresponding resection information according to the merge information.

2. The method according to claim 1, characterized in that The padding information and the resection information on the corresponding axes of the padding operation node and the resection operation node are merged to obtain a mapping dictionary with the axis information and the merged information as key-value pairs, including: Taking out all the filling axis information and filling information in the filling operation node, taking out all the cutting axis information and cutting information in the cutting operation node; Using each value in the padding axis information as a key and the corresponding padding information as data to create a mapping, and store it in the mapping dictionary; Each value in the resection axis information is used as a key. If the key in the resection axis information exists in the mapping dictionary, the value of the corresponding key in the mapping dictionary is updated using the corresponding resection information.

3. The method according to claim 1, characterized in that Process the key-value pairs in the mapping dictionary, including: When none of the merge information is less than 0, the corresponding axis is a padding operation, the corresponding axis is added to the padding axis, and the merge information is added to the padding information.

4. The method according to claim 1, characterized in that: Process the key-value pairs in the mapping dictionary, including: When the merge information is all less than 0, the corresponding axis is a resection operation, the corresponding axis is added to the resection axis, and the merge information after positive addition is added to the resection information.

5. The method according to claim 1, characterized in that Processing key-value pairs in a mapping dictionary also includes: If the first part of the merged information is not greater than 0 and the second part of the merged information is not less than 0, the corresponding axis is a resection and padding operation, the corresponding axis is added to the resection axis, the first part of the merged information is positive, the second part of the merged information is 0, and added to the resection information, and the corresponding axis is added to the padding axis, the first part of the merged information is 0, the second part of the merged information is unchanged, and added to the padding information; The second part of the merged information is not greater than 0 and the first part of the merged information is not less than 0, the corresponding axis is a resection and padding operation, the corresponding axis is added to the resection axis, the first part of the merged information is 0, the second part of the merged information is positive, and added to the resection information, the corresponding axis is added to the padding axis, the first part of the merged information remains unchanged, the second part of the merged information is 0, and added to the padding information.

6. The method according to claim 1, characterized in that The new node obtained after the merge is determined according to the result of the merge operation, including: If both the padding information and the removal information are empty, or their values ​​are both 0, then the fusion result is an empty operation, the new node is an empty node, and the input of the removal operation node is directly output to the successor node of the padding operation node.

7. The method according to claim 1, characterized in that The new node obtained after the merge is determined according to the result of the merge operation, including: If the excision information is empty, or the values ​​of the excision data are all 0, then the fusion result is a padding operator, and the new node is a padding operation node; and / or If the padding information is empty, or the values ​​of the padding data are all 0, then the fusion result is a resection operator, and the new node is a resection operation node.

8. The method according to claim 1, characterized in that The new node obtained after the merge is determined according to the result of the merge operation, including: If both the padding information and the cutting information are not empty and the value of the operation data is not 0, the node is a composite operation, including both cutting and padding operations, and the new node is a node with both cutting and padding operations.

9. A computing device, characterized in that include: processor; as well as A memory storing a computer program, which, when executed by the processor, enables the processor to perform the method according to any one of claims 1 to 8.

Images