Method and system for optimizing memory operation of data stream chip and computing equipment
By combining the complement and cut-off operation nodes of the data stream chip model on the data stream chip, the inference efficiency of the model is optimized, the problem of frequent filling and cut-off operations affecting efficiency is solved, and memory consumption and performance improvement is achieved.
Patent Information
- Application Number
- CN202411495834.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-25
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-10-25
AI Technical Summary
In artificial intelligence model calculation, the existing technology fails to effectively optimize the data filling and cutting operations of nodes on the data stream chip, resulting in frequent filling and cutting operations affecting the overall inference efficiency of the model.
By obtaining the graph compiled by the model, traversing the nodes to collect the complementary operation nodes, and merging the complementary operation nodes and the removal operation nodes whose successor nodes are cut off and are the only successor nodes to generate new nodes to reduce redundancy Information and intermediate data replication, optimize memory access.
It improves the overall inference efficiency of the model, reduces memory consumption and redundant operations, simplifies the computing graph structure, facilitates management and debugging, and improves the performance of computing devices.
Smart Images

Figure CN119026692A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method and computing device for optimizing memory operations of a data stream chip. Background Art
[0002] In artificial intelligence model calculations, in order to match the data alignment requirements of the data flow chip, some nodes running on the chip are usually padded with data, and the excess data is removed after calculation, so as to meet the hardware's requirements for data alignment and ensure the correctness of the model calculation results.
[0003] In the data padding and cutting operations of model nodes, usually only the memory-aligned cutting operations of a single node are considered, but there is a lack of optimization of the connection between the previous and next nodes. Therefore, frequent padding and cutting operations may occur, affecting the overall reasoning efficiency of the model.
[0004] Therefore, a method for optimizing data flow chip memory operations is needed to improve the overall reasoning efficiency of the model. Summary of the invention
[0005] The present invention aims to provide a method and computing device for optimizing memory operations of data flow chips, which helps to improve the reasoning efficiency of the model, reduce memory consumption, and improve overall performance.
[0006] According to one aspect of the present invention, a method for optimizing memory operations of a data stream chip is provided, comprising: Get the graph after model compilation; Traversing the nodes of the graph, collecting and completing operation nodes, and adding them to the first node set; Traversing the first node set, if the predecessor node of the completion operation node is a removal operation node and the completion operation node is the only successor node of the removal operation node, merging the removal operation node with the completion operation node; A new node obtained after merging is determined according to the result of the merging operation, wherein the new node is an empty node, a cutting operation node, a filling operation node, or a cutting and filling operation node.
[0007] According to some embodiments, performing a merging operation on the cut operation node and the fill operation node includes: Merge the padding information and the cutting information on the corresponding axes of the padding operation node and the cutting operation node to obtain a mapping dictionary with the axis information and the merged information as key-value pairs; The key-value pairs in the mapping dictionary are processed to obtain the padding axis information and the corresponding padding information and / or the resection axis information and the corresponding resection information according to the merge information.
[0008] According to some embodiments, merging the padding information and the resection information on the corresponding axes of the padding operation node and the resection operation node to obtain a mapping dictionary with the axis information and the merged information as key-value pairs includes: Taking out all the filling axis information and filling information in the filling operation node, taking out all the cutting axis information and cutting information in the cutting operation node; Using each value in the padding axis information as a key and the corresponding padding information as data to create a mapping, and store it in the mapping dictionary; Each value in the resection axis information is used as a key. If the key in the resection axis information exists in the mapping dictionary, the value of the corresponding key in the mapping dictionary is updated using the corresponding resection information.
[0009] According to some embodiments, processing a key-value pair in a mapping dictionary includes: When none of the merge information is less than 0, the corresponding axis is a padding operation, the corresponding axis is added to the padding axis, and the merge information is added to the padding information.
[0010] According to some embodiments, processing a key-value pair in a mapping dictionary includes: When the merge information is all less than 0, the corresponding axis is a resection operation, the corresponding axis is added to the resection axis, and the merge information after positive addition is added to the resection information.
[0011] According to some embodiments, processing a key-value pair in a mapping dictionary further includes: If the first part of the merged information is not greater than 0 and the second part of the merged information is not less than 0, the corresponding axis is a resection and padding operation, the corresponding axis is added to the resection axis, the first part of the merged information is positive, the second part of the merged information is 0, and added to the resection information, and the corresponding axis is added to the padding axis, the first part of the merged information is 0, the second part of the merged information is unchanged, and added to the padding information; The second part of the merged information is not greater than 0 and the first part of the merged information is not less than 0, the corresponding axis is a resection and padding operation, the corresponding axis is added to the resection axis, the first part of the merged information is 0, the second part of the merged information is positive, and added to the resection information, the corresponding axis is added to the padding axis, the first part of the merged information remains unchanged, the second part of the merged information is 0, and added to the padding information.
[0012] According to some embodiments, determining a new node obtained after merging according to a result of the merging operation includes: If both the padding information and the removal information are empty, or their values are both 0, then the fusion result is an empty operation, the new node is an empty node, and the input of the removal operation node is directly output to the successor node of the padding operation node.
[0013] According to some embodiments, determining a new node obtained after merging according to a result of the merging operation includes: If the excision information is empty, or the values of the excision data are all 0, then the fusion result is a padding operator, and the new node is a padding operation node; and / or If the padding information is empty, or the values of the padding data are all 0, then the fusion result is a resection operator, and the new node is a resection operation node.
[0014] According to some embodiments, determining a new node obtained after merging according to a result of the merging operation includes: If both the padding information and the cutting information are not empty and the value of the operation data is not 0, the node is a composite operation, including both cutting and padding operations, and the new node is a node with both cutting and padding operations.
[0015] According to another aspect of the present invention, there is provided a computing device, comprising: Processor; and A memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform any of the aforementioned methods.
[0016] According to an exemplary embodiment of the present invention, the graph after model compilation is obtained, and by traversing the entire model graph, it can be ensured that all possible completion operation nodes are identified, and no potential optimization target is missed; the collected completion operation nodes are accurately collected into a set, providing a reliable basis for subsequent optimization steps. By traversing the first node set, by merging the completion operation nodes and the removal operation nodes, the copy operation of the intermediate data can be reduced, thereby reducing unnecessary memory access and data copying; by generating new nodes, the number of nodes in the graph is reduced, making the calculation graph more concise and easy to manage and debug.
[0017] According to the example embodiment, the padding information and the resection information on the corresponding axes of the padding operation node and the resection operation node are merged. By merging the information, the number of times each axis is processed separately in subsequent steps is reduced, thereby improving the computational efficiency. A mapping dictionary of the axis information and the merged information as key-value pairs is obtained, thereby reducing the storage and processing of redundant information. The mapping dictionary makes the data structure more compact and convenient for management and query.
[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below.
[0020] Figure 1 A flow chart of a method for optimizing memory operations of a data stream chip according to an example embodiment is shown.
[0021] Figure 2 A flow chart of a method for obtaining a new node according to an exemplary embodiment is shown.
[0022] Figure 3 A flow chart of a method for node merging according to an example embodiment is shown.
[0023] Figure 4 A block diagram of a computing device according to an example embodiment of the invention is shown. DETAILED DESCRIPTION
[0024] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.
[0025] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.
[0026] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0027] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.
[0028] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the present inventive concept. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.
[0029] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0030] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present invention, and therefore cannot be used to limit the protection scope of the present invention.
[0031] After optimizing the padding and cutting operations on the original graph structure of the artificial intelligence model, the present invention optimizes the overall execution efficiency of the model by optimizing the padding and cutting operations of adjacent nodes and eliminating redundant padding and cutting operations, thereby achieving the purpose of computing acceleration.
[0032] Before describing the embodiments of the present application, some terms or concepts involved in the embodiments of the present application are explained.
[0033] MLIR (Multi-Level Intermediate Representation) is a powerful framework for building interoperable compiler infrastructure. It provides a multi-level intermediate representation that allows developers to efficiently perform various graph-level optimizations.
[0034] Exemplary embodiments of the present invention are described below with reference to the accompanying drawings.
[0035] Figure 1 A flow chart of a method for optimizing memory operations of a data stream chip according to an example embodiment is shown.
[0036] In S101, a graph after model compilation is obtained.
[0037] In order to optimize the graph after model compilation, first, it is necessary to obtain the graph structure after model compilation. According to some embodiments, this can usually be achieved through an API provided by a model compilation tool or framework, for example, through an MLIR compilation tool in LLVM.
[0038] In S103, the nodes of the graph are traversed, and the completion operation nodes are collected and added to the first node set.
[0039] According to an example embodiment, all nodes of the model graph are traversed, all completion operation nodes are collected, and added into a set.
[0040] In S105, the first node set is traversed, and if the predecessor node of the completion operation node is the removal operation node and the completion operation node is the only successor node of the removal operation node, a merge operation is performed on the removal operation node and the completion operation node.
[0041] According to an example embodiment, each padding operation node in the first node set is traversed, and for each padding operation node, its predecessor node is obtained. The predecessor node is checked to see whether the type of the predecessor node is a crop operation node (Crop). If the predecessor node of the padding operation node is a crop operation node and the padding operation node is the only successor node of the crop operation node; if the above conditions are met, the crop operation node and the padding operation node are merged.
[0042] According to some embodiments, when the resection operation node and the padding operation node are merged, the padding information and the resection information on the corresponding axes of the padding operation node and the resection operation node are merged to obtain a mapping dictionary with the axis information and the merged information as key-value pairs; the key-value pairs in the mapping dictionary are processed to obtain the padding axis information and the corresponding padding information and / or the resection axis information and the corresponding resection information according to the merged information.
[0043] In S107, a new node obtained after merging is determined according to the result of the merging operation, wherein the new node is an empty node, a cut-operation node, a fill-operation node, or a cut-and-fill-operation node.
[0044] According to an example embodiment, a new node obtained after the merge is determined based on the result of the merge operation. The types of the new node include an empty node, a cut operation node, a padding operation node, or a cut and padding operation node. The new node type can be determined based on the result of the merge operation, thereby optimizing the model graph, reducing redundant operations, and improving the overall execution efficiency of the model.
[0045] Figure 2 A flow chart of a method for obtaining a new node according to an exemplary embodiment is shown.
[0046] See also Figure 2 After compiling the model and performing necessary padding and cropping operations on each node, the Pad and Crop in the model are merged and eliminated.
[0047] In S201, the entire model graph node is traversed, all the completion operation nodes are collected, and added into the node set.
[0048] According to some embodiments, the graph in the model compilation is based on the graph representation of MLIR, and the nodes in each graph are node names and node type names. The nodes collected here are nodes with the node type name of Pad. Specifically, the collection is initialized first, and an empty collection AllPads is created to store all the found padding operation nodes; then the entire model graph node is traversed, all the padding operation nodes are collected, and added to the node collection AllPads.
[0049] In S203, a completion operation node is taken out from the node set, and it is determined whether the predecessor node of the completion operation node is a removal operation and the completion operation node is the only successor node of the removal operation node.
[0050] According to an example embodiment, a padding operation node A is taken out from AllPads, and it is determined whether its predecessor node C is a Crop operation and the padding operation node A is the only successor node of the Crop operation node C. If the type name of the node C is not Crop or the padding operation node A is not the only successor node of the Crop operation node C, this node is skipped and the next padding operation node is processed.
[0051] In S205, the completion operation node and the removal operation node are merged, and the obtained node can be recorded as node M.
[0052] According to the exemplary embodiment, a padding operation node A and a cutting operation node C are merged, and the padding information and the cutting information on the corresponding axes of the padding operation node and the cutting operation node are merged to obtain a mapping dictionary padcrop_info with the axis information and the merged information as key-value pairs (see Figure 2 Axis is the dimension index. For example, the axes of four-dimensional data can be 0, 1, 2, and 3.
[0053] Repeat steps S203 and S205 until the AllPads collection is empty.
[0054] At S207 , the key-value pairs in the mapping dictionary are processed.
[0055] According to an example embodiment, the key-value pairs in the mapping dictionary padcrop_info are processed, and padding axis information and corresponding padding information, and / or resection axis information and corresponding resection information are obtained according to the merge information.
[0056] According to an example embodiment, the merge information (left, right) corresponding to each axis axis in the mapping dictionary padcrop_info can be divided into the following cases.
[0057] Case 1: When left and right are not less than 0, the axis is pad-operated, axis is added to the padding axis pad_axis, and the merge information (left, right) is added to the padding information pad_sizes.
[0058] Case 2: If the merge information (left, right) is less than 0, the axis is a crop operation, axis is added to the resection axis crop_axis, and the merge information (-left, -right) after taking the positive value is added to the resection information crop_sizes.
[0059] Case three, if it is judged that the first part of the merged information left is not greater than 0 and the second part of the merged information right is not less than 0, then the axis is crop and pad operation, add axis to the resection axis crop_axis, take the first part of the merged information positive, and take the second part of the merged information 0, that is, (-left, 0) is added to the resection information crop_sizes, and add axis to the padding axis pad_axis, take the first part of the merged information 0, and the second part of the merged information remains unchanged, that is, (0, right) is added to the padding information pad_sizes.
[0060] Case 4: If the second part of the merged information right is not greater than 0 and the first part of the merged information left is not less than 0, the axis is a crop and pad operation. Add axis to the resection axis crop_axis, take the first part of the merged information as 0, and take the second part of the merged information as positive, that is, (0, -right) is added to the resection information crop_sizes, add axis to pad_axis, keep the first part of the merged information unchanged, take the second part of the merged information as 0, that is, (left, 0) is added to the padding information pad_sizes.
[0061] In S209, a new node obtained after merging is determined according to the result of the merging operation.
[0062] According to the example embodiment, ensure that all eligible Pad nodes and Crop nodes have been merged, and determine the type name of the new node based on the merging result, for example, it can be an empty node, a cut operation node, a fill operation node, or a cut and fill operation node.
[0063] If both the padding information and the removal information are empty, or the values are both 0, then the fusion result is an empty operation, the new node is an empty node, and the input of the removal operation node C can be directly output to the successor node of the padding operation node A.
[0064] If the cut information is empty, or the values of the cut data are all 0, the fusion result is the padding operator, and the new node is the padding operation node. At this time, you can create the corresponding Pad operator: create Pad(pad_axis, pad_sizes).
[0065] If the padding information is empty, or the values of the padding data are all 0, the fusion result is the Crop operator, and the new node is the Crop operation node. You can create the corresponding Crop operator: create Crop(crop_axis, crop_sizes).
[0066] If both the padding information and the cropping information are not empty and the value of the operation data is not 0, the node is a compound operation, with both cropping and padding. The new node is a node with both cropping and padding operations. You can create the corresponding cropping and padding operator CropAndPad: create CropAndPad(crop_axis, crop_sizes, pad_axis, pad_sizes).
[0067] After the fusion operation, four possible results can be obtained: a Crop node, a padding operation node, a CropAnd padding operation node or an empty node. The input of the Crop node C can be directly given to the output of the padding operation node A. Through the fusion operation, the best case is to directly eliminate the Crop and padding operation nodes, saving two steps of calculation; the worst case is that the Crop and Pad operations exist at the same time, which are fused into a CropAndPad operation. Although two operations still need to be performed, the structure of the graph is simplified.
[0068] The present invention is based on a data stream chip, and one of the characteristics of a data stream chip is that it has a high degree of computational parallelism and a higher computational efficiency than an instruction stream. As long as the data is moved to the chip storage location, it can be quickly computed. However, if there are memory operations in each computing node and data needs to be copied frequently, the pipeline operation advantage of the data stream chip will be interrupted and the computational efficiency will be rapidly reduced. The present invention mainly integrates the padding operation and the excision operation, thereby making it possible to completely eliminate the intermediate memory operation and data copying, maintain the computational continuity of the data stream, and give full play to its computational performance.
[0069] Figure 3 A flow chart of a method for performing a merging operation on nodes according to an example embodiment is shown.
[0070] In S301, all the completion axis information and the completion information in the completion operation node A are retrieved; and all the resection axis information and the resection information in the resection operation node C are retrieved.
[0071] According to an exemplary embodiment, all the padding axis information A_axes and padding information A_pad_sizes in the padding operation node A are taken out, and all the resection axis information C_axes and resection information C_crop_sizes in the resection operation node C are taken out. The padding information A_pad_sizes corresponds to the padding information of each axis in the padding axis information A_axes. For example, for two-dimensional data A(6, 8), which represents 6 rows and 8 columns of data, the padding axis information A_axes=[0, 1], the padding information A_pad_sizes=[2,3, 4, 6] means that 2 rows of data are padded on the top of the input data, 3 rows of data are padded on the bottom, 4 columns of data are padded on the left, and 6 columns of data are padded on the right. After the padding operation, the node pA(11, 18) becomes 11 rows and 18 columns of data. In the resection operation, the two-dimensional data C(8, 10) represents 8 rows and 10 columns of data. C_axes=[0, 1], C_crop_sizes=[1, 2, 3, 4] means cutting 1 row on the top, 2 rows on the bottom, 3 columns on the left, and 4 columns on the right. After the resection operation, the node C(5, 3) becomes 5 rows and 3 columns of data.
[0072] In S303, a mapping is created by taking each value in the padding axis information as a key and the corresponding padding information as data, and the mapping is stored in the mapping dictionary.
[0073] According to an example embodiment, each value in the padding axis information A is used as a key, and the corresponding padding operation information is used as data (value) to create a mapping, which is stored in the mapping dictionary. Each value in the padding axis information A_axes is used as a key, and the corresponding padding operation data (left, right) is used as data (value) to create a mapping, which is stored in the mapping dictionary padcrop_info. Each number in A_axes will correspond to two numbers in A_pad_sizes, indicating the number of data to be padded in front and behind this axis. For example, in the example A_axes=[0, 1], A_pad_sizes=[2, 3, 4, 6], then for axis 0, its left=2, right=3; for axis 1, its left=4, right=6.
[0074] According to some embodiments, the two-dimensional data A(6,8) is transformed into pA(11, 18) after a padding operation of A_axes=[0, 1], A_pad_sizes=[2, 3, 4, 6] and stored in the padcrop_info dictionary.
[0075] In S305, each value in the resection axis information is used as a key. If the key in the resection axis information exists in the mapping dictionary, the value of the corresponding key in the mapping dictionary is updated using the corresponding resection information.
[0076] According to an example embodiment, each axis information value axis in the resection axis information C_axes is used as a key. If axis is in the mapping dictionary padcrop_info, the corresponding value padcrop_info[axis] is taken out. The mapping dictionary padcrop_info is saved with axis as the key and operation data (left, right) as the value. Pad and Crop are both operations of the corresponding axis. Assuming the value is (left, right), the corresponding resection data (cleft, cright) in C_crop_sizes is taken out, and new_left=left-cleft, new_right=right-cright is set, and the new value is stored in the mapping dictionary padcrop_info, that is, padcrop_info[axis]=(new_left, new_right).
[0077] According to the method of the present invention, the padding operation node is merged with the cutting operation node, which can effectively reduce the copy operation of memory data and improve the overall reasoning efficiency of the model. The method for optimizing the memory operation of data flow chips can not only reduce unnecessary memory access, but also simplify the calculation graph and improve the calculation speed and energy efficiency.
[0078] According to some embodiments, during the training process, for example, the image has 3 channels, the data flow chip parallelism is generally 4 or 64, and an input data of 1*224*224*3 will be padded to 1*224*224*64 when entering the convolution calculation. If the original weight of the convolution is 3*1*1*3, the weight will be padded to 64*1*1*64, then the new output is 1*224*224*64. The normal output before padding should be 1*224*224*3, and the extra data needs to be removed.
[0079] Similar to a pipeline, 1*224*224*3 fills the data in the channel dimension at the back to 64 channels. In AI processing, an image data is represented as 1*224*224*3, which represents the size of the image batch (Batch Size). In this example, the batch size is 1, which means there is only one image. In practical applications, the batch size is usually greater than 1 to improve computing efficiency and model stability; "224*224" represents the height and width of the image, which is 224 pixels, indicating that the image is a square image; 3 is the number of color channels of the image, indicating that the image is an RGB image with three color channels: red, green, and blue.
[0080] If the size of the original image is not 224×224, it is usually necessary to resize it to match the model requirements by cropping or resizing. The batch size can be adjusted according to actual needs to improve computational efficiency.
[0081] Figure 4 A block diagram of a computing device according to an example embodiment of the invention is shown.
[0082] like Figure 4 As shown, computing device 30 includes processor 12 and memory 14. Computing device 30 may also include bus 22, network interface 16, and I / O interface 18. Processor 12, memory 14, network interface 16, and I / O interface 18 may communicate with each other via bus 22.
[0083] The processor 12 may include one or more general-purpose CPUs (Central Processing Units, processors), microprocessors, or application-specific integrated circuits, etc., for executing relevant program instructions.
[0084] The memory 14 may include a machine system readable medium in the form of a volatile memory, such as a random access memory (RAM), a read-only memory (ROM) and / or a cache memory. The memory 14 is used to store one or more programs including instructions and data. The processor 12 may read the instructions stored in the memory 14 to execute the above-mentioned method according to the embodiment of the present invention.
[0085] The computing device 30 may also communicate with one or more networks via the network interface 16. The network interface 16 may be a wireless network interface.
[0086] The bus 22 may include an address bus, a data bus, a control bus, etc. The bus 22 provides a path for exchanging information between components.
[0087] It should be noted that, in the specific implementation process, the computing device 30 may also include other components necessary for normal operation. In addition, those skilled in the art may understand that the above device may only include components necessary for implementing the embodiments of this specification, and need not include all components shown in the figure.
[0088] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the program is executed by a processor. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), a network storage device, a cloud storage device, or any type of medium or device suitable for storing instructions and / or data.
[0089] An embodiment of the present invention also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any method recorded in the above method embodiments.
[0090] Those skilled in the art can clearly understand that the technical solution of the present invention can be implemented with the help of software and / or hardware. "Unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array, an integrated circuit, etc.
[0091] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.
[0092] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0093] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.
[0094] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0095] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0096] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.
[0097] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0098] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended clauses.
[0099] Those skilled in the art can clearly understand that the technical solution of the present invention can be implemented with the help of software and / or hardware. "Unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array, an integrated circuit, etc.
[0100] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.
[0101] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0102] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.
[0103] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0104] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0105] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.
[0106] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0107] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended clauses.
Claims
1. A method for optimizing memory operations in data flow chips, It is characterized in that include: Get the graph after model compilation; Traversing the nodes of the graph, collecting and completing operation nodes, and adding them to the first node set; Traversing the first node set, if the predecessor node of the padding operation node is a cutting operation node and the padding operation node is the only successor node of the cutting operation node, merging the cutting operation node with the padding operation node; A new node obtained after merging is determined according to the result of the merging operation, wherein the new node is an empty node, a cutting operation node, a filling operation node, or a cutting and filling operation node.
2. The method according to claim 1, It is characterized in that The step of merging the cutting operation node and the filling operation node comprises: Merge the padding information and the cutting information on the corresponding axes of the padding operation node and the cutting operation node to obtain a mapping dictionary with the axis information and the merged information as key-value pairs; The key-value pairs in the mapping dictionary are processed to obtain the padding axis information and the corresponding padding information and / or the resection axis information and the corresponding resection information according to the merge information.
3. The method according to claim 2, It is characterized in that The padding information and the resection information on the corresponding axes of the padding operation node and the resection operation node are merged to obtain a mapping dictionary with the axis information and the merged information as key-value pairs, including: Taking out all the filling axis information and filling information in the filling operation node, taking out all the cutting axis information and cutting information in the cutting operation node; Using each value in the padding axis information as a key and the corresponding padding information as data to create a mapping, and store it in the mapping dictionary; Each value in the resection axis information is used as a key. If the key in the resection axis information exists in the mapping dictionary, the value of the corresponding key in the mapping dictionary is updated using the corresponding resection information.
4. The method according to claim 2, It is characterized in that Process the key-value pairs in the mapping dictionary, including: When none of the merge information is less than 0, the corresponding axis is a padding operation, the corresponding axis is added to the padding axis, and the merge information is added to the padding information.
5. The method according to claim 2, It is characterized in that Process the key-value pairs in the mapping dictionary, including: When the merge information is all less than 0, the corresponding axis is a resection operation, the corresponding axis is added to the resection axis, and the merge information after positive addition is added to the resection information.
6. The method according to claim 2, It is characterized in that Processing key-value pairs in a mapping dictionary also includes: If the first part of the merged information is not greater than 0 and the second part of the merged information is not less than 0, the corresponding axis is a resection and padding operation, the corresponding axis is added to the resection axis, the first part of the merged information is positive, the second part of the merged information is 0, and added to the resection information, and the corresponding axis is added to the padding axis, the first part of the merged information is 0, the second part of the merged information is unchanged, and added to the padding information; The second part of the merged information is not greater than 0 and the first part of the merged information is not less than 0, the corresponding axis is a resection and padding operation, the corresponding axis is added to the resection axis, the first part of the merged information is 0, the second part of the merged information is positive, and added to the resection information, the corresponding axis is added to the padding axis, the first part of the merged information remains unchanged, the second part of the merged information is 0, and added to the padding information.
7. The method according to claim 2, It is characterized in that The new node obtained after the merge is determined according to the result of the merge operation, including: If both the padding information and the removal information are empty, or their values are both 0, then the fusion result is an empty operation, the new node is an empty node, and the input of the removal operation node is directly output to the successor node of the padding operation node.
8. The method according to claim 2, It is characterized in that The new node obtained after the merge is determined according to the result of the merge operation, including: If the excision information is empty, or the values of the excision data are all 0, then the fusion result is a padding operator, and the new node is a padding operation node; and / or If the padding information is empty, or the values of the padding data are all 0, then the fusion result is a resection operator, and the new node is a resection operation node.
9. The method according to claim 2, It is characterized in that The new node obtained after the merge is determined according to the result of the merge operation, including: If both the padding information and the cutting information are not empty and the value of the operation data is not 0, the node is a composite operation, including both cutting and padding operations, and the new node is a node with both cutting and padding operations.
10. A computing device, It is characterized in that include: processor; as well as A memory storing a computer program, which, when executed by the processor, enables the processor to perform the method according to any one of claims 1 to 9.
Citation Information
Patent Citations
Data analysis method and device based on open source arangodb and storage medium
CN113254763A
A calculation graph node division method and device and a computer readable storage medium
CN113553287A
Neural network computational graph processing method and processing device
CN114881214A
Method and device for adaptively accelerating operator fusion
CN115659281A
Data flow diagram processing method and device and computer readable storage medium
CN115730107A
