Method for constructing a subgraph and allocating strip sizes in the subgraph

Abstract

A method for constructing a subgraph, comprising: receiving a directed acyclic graph (DAG); dividing the directed acyclic graph into at least one section; determining at least one hardware property; determining at least one DAG hardware limit for the at least one section; and determining a list of the maximum consecutive nodes of the at least one section, wherein the at least one hardware property satisfies at least one DAG hardware limit in the list of the maximum consecutive nodes.

Description

Technical Field

[0001] The present disclosure relates to neural network acceleration, and more particularly, to neural network acceleration by means of graph partitioning. Background Art

[0002] Currently, the scale of neural networks has been increasing. Due to limitations of resources such as memory, it is impossible to run larger neural networks on one accelerator or device. Summary of the Invention

[0003] A method for constructing a subgraph, comprising: receiving a directed acyclic graph (DAG); partitioning the directed acyclic graph into at least one segment; determining at least one hardware property; determining at least one DAG hardware limit of at least one segment; and determining a list of maximum consecutive nodes of at least one segment, wherein the at least one hardware property satisfies the at least one DAG hardware limit in the list of maximum consecutive nodes.

[0004] A method for allocating strip sizes in a subgraph, comprising: receiving a directed acyclic graph; partitioning the directed acyclic graph into at least one segment; determining an input tensor strip size; and updating at least one hardware property based on the input tensor strip size. Brief Description of the Drawings

[0005] In the drawings:

[0006] Figure 1 is a first exemplary system schematic diagram according to one embodiment disclosed;

[0007] Figure 2 is a second exemplary system schematic diagram according to one embodiment disclosed;

[0008] Figure 3 is an exemplary flowchart of graph partitioning according to one embodiment disclosed;

[0009] Figure 4 is an exemplary boundary candidate search according to one embodiment disclosed;

[0010] Figure 5 is an exemplary strip size allocation process according to one embodiment disclosed;

[0011] Figure 6 is an exemplary sawtooth search for strip sizes according to one embodiment disclosed;

[0012] Figure 7 is a comparison of an example of updating a strip with a previously saved strip according to one embodiment disclosed;

[0013] Figure 8 is a first exemplary method of graph partitioning according to one embodiment disclosed;

[0014] Figure 9 is a second exemplary method partitioned according to the figures of one disclosed embodiment;

[0015] Figure 10 is a third exemplary method partitioned according to the figures of one disclosed embodiment;

[0016] Figure 11 is a first exemplary method for allocating strip sizes in a sub - graph according to one disclosed embodiment; and

[0017] Figure 12 is a second exemplary method for allocating strip sizes in a sub - graph according to one disclosed embodiment. Detailed Description

[0018] The embodiments listed below are only used to illustrate the application of the device and method, rather than to limit the scope. Equivalent modified forms of the device and method should fall within the scope of the claims.

[0019] Throughout the following specification and claims, certain terms are used to refer to specific system components. Those skilled in the art can understand that different companies may use different names to refer to a component and / or method. This document does not intend to distinguish components and / or methods with different names but the same functions.

[0020] In the following specification and claims, the terms "comprising" and "including" are used in an open - ended manner. Thus, they can be interpreted as "including but not limited to". In addition, the term "coupled" is intended to mean an indirect or direct connection. Thus, if a first device is coupled to a second device, the connection can be a direct connection or an indirect connection via other devices and connections.

[0021] Figure 1 An exemplary hybrid computing system 100 is shown. The hybrid computing system 100 can be used to implement a neural network associated with the operation of one or more parts or steps of the following processes. In this example, the processors associated with the hybrid system include a Field - Programmable Gate Array (FPGA) 122, a Graphics Processing Unit (GPU) 120, and a Central Processing Unit (CPU) 118.

[0022] The CPU 118, GPU 120, and FPGA 122 have the ability to provide neural networks. The CPU is a general-purpose processor that can execute multiple different functions. Its versatility enables it to perform a variety of different tasks. However, the CPU's processing of multiple data streams and its capabilities in neural networks are limited. The GPU is a graphics processing unit that has multiple small processing cores capable of processing parallel tasks in sequence. The FPGA is a field-programmable device that can be reconfigured and perform any function that can be programmed into the CPU or GPU in a hardwired circuit manner. Since the programming of the FPGA is in the form of a circuit, its speed is several times faster than that of the CPU and much faster than that of the GPU.

[0023] The system can include other types of processors, such as an accelerated processing unit (APU) and a digital signal processor (DSP). The accelerated processing unit includes a CPU with an on-chip GPU element, and the digital signal processor is designed to perform high-speed numerical data processing. An application-specific integrated circuit (ASIC) can also perform the hardwired functions of the FPGA. However, the lead time for designing and manufacturing an ASIC is approximately several quarters, and it is not a fast-turnaround implementation available in FPGA programming.

[0024] The graphics processing unit 120, central processing unit 118, and field-programmable array 122 are interconnected and are all connected to the memory interface and controller 112. The field-programmable array is connected to the memory interface through a programmable logic circuit connected to the memory interconnect 130. This additional device is used because the FPGA operates with a large bandwidth and to minimize the circuits used by the FPGA for performing memory tasks. The memory interface and controller 112 are also connected to the permanent storage disk 110, system memory 114, and read-only memory (ROM) 116.

[0025] Figure 1 The system can be used to program and train the FPGA. The GPU can handle unstructured data well and can be used for training. Once the data is trained, a deterministic inference model can be established, and the CPU can program the FPGA with the model data determined by the GPU.

[0026] The memory interface and controller are connected to the central interconnect 124. The central interconnect is also connected to the GPU 120, CPU 118, and FPGA 122. The central interconnect 124 is also connected to the input / output interface 128 and the network interface 126.

[0027] Figure 2Figure 200 shows a second exemplary hybrid computing system 200 that can be used to implement a neural network associated with the operation of one or more parts or steps of processes 800, 900, 1000, 1100, and 1200. In this example, the processors associated with the hybrid system include a field programmable array (FPGA) 210 and a central processing unit (CPU) 220.

[0028] The FPGA is electrically connected to an FPGA controller 212, which interfaces with a direct memory access (DMA) 218. The DMA is connected to an input buffer 214 and an output buffer 216, and the input buffer 214 and the output buffer 216 are coupled to the FPGA to buffer data into and out of the FPGA. The DMA 218 includes two first-in, first-out (FIFO) buffers, one for the main CPU and the other for the FPGA, and the DMA allows data to be written to and read from the appropriate buffer.

[0029] On the CPU side of the DMA is a main switch 228 that transfers data and commands to the DMA. The DMA is also connected to an SDRAM controller 224 that allows data to be transferred between the FPGA and the CPU 220. The SDRAM controller is also connected to an external SDRAM 226 and the CPU 220. The main switch 228 is connected to a peripheral interface 230. A flash controller 222 controls the permanent memory and is connected to the CPU 220.

[0030] As neural networks are increasingly used in ubiquitous microcomputing environments, it may lead to resource limitations in terms of platform processing power and memory. A possible solution for running neural networks on resource-constrained hardware systems can be to split the original neural network into multiple subgraphs and allow multiple subgraphs to be executed in the available subsystems. Graph partitioning can provide a possible solution for resource-limited artificial intelligence applications.

[0031] Given a neural network, an easy way to execute is to split the neural network into multiple subnets. A measure of execution efficiency can be to determine the hardware utilization as a cost function for different partitioning schemes. Additionally, methods can be used to determine which partitioning scheme is the most efficient to run on a specific platform.

[0032] Partitioning the graph into smaller sub - graph segments allows it to be loaded onto the available hardware. Typically, an AI chip may not be able to load the entire graph onto the chip, for example, in the case of large nodes (deep convolutional networks). Additionally, some rules may be applied during graph partitioning based on hardware limitations, such as restricting connected nodes at the boundaries, which further limits the options for graph segmentation.

[0033] An example of a possible solution could be to allow different operators to run on different hardware (e.g., arrays, DSPs, etc.) and to allow a large graph to be partitioned into multiple small sub - graphs such that the sub - graphs meet the hardware limitations. An example of a possible solution could be to allow the determination of an effective execution order of these sub - graphs on the available hardware.

[0034] Definitions:

[0035] Boundary: The last node of a section, including the last node name and section input parameters.

[0036] BTMW (byte tensor move mask and weight): Byte tensor move mask and weight.

[0037] BTMD (byte tensor direct memory access input and output): Byte tensor direct memory access input and output.

[0038] DAG (directed acyclic graph): A directed acyclic graph, the low - level intermediate representation of an AI chip, which passes graph information to the chip.

[0039] DBUF: Data buffer for node input values and node output values.

[0040] DMA_IN: Direct memory access input tensor.

[0041] DMA_OUT: Direct memory access output tensor.

[0042] Graph: An abstract - level description of a neural network model.

[0043] NNX (neural network exchange): Neural network interaction.

[0044] OVBUF: Overlap buffer.

[0045] Partition: The operation of dividing a larger code segment into multiple smaller code segments.

[0046] Partition section: sub-graph.

[0047] Figure 3 A graph partitioning workflow is shown, including manual partitioning 310, dynamic partitioning 312 using dynamic programming, and greedy partitioning 314. Greedy partitioning 314 uses a section checker function and partitions the graph into the minimum number of sections. The graph partitioning phase 316 allows selection of a partitioning method 318. The graph optimization phase 320 allows modification of the partitioning, execution order, hardware sections, etc. for optimization. Then, the section binding phase 322 determines the hardware attributes for running the partitioning.

[0048] Graph partitioning divides a graph into multiple sections, and the output of each phase can be a partitioned graph with attributes. Some optimization channels can utilize previously revised partitions for further changes. In one example, replacing fused add nodes with estimated add nodes can produce an extended additional optimizer (which depends on the input node stripe size and output node stripe size determined during graph partitioning) to further incorporate penalty measures, such as inserting 1×1 convolutions, permutations, and pooling nodes.

[0049] The partition section binding phase reviews the hardware attributes. The graph structure may change after graph optimization, resulting in changes to the hardware attributes. The section binding phase ensures that the sections are valid for generating a directed acyclic graph and that the C code passes the analysis mode. After enabling profiling comparison, comma-separated numerical documents can also be generated in the phase results for the directory analysis folder in the benchmark node performance section and the benchmark section performance section.

[0050] Partitioning can be an iterative process of multiple calls to different partitioning methods. The library provides rule-based partitioning methods (e.g., based on hardware limitations), and the partitioning method based on a cost function can be based on hardware utilization, etc. The user can decide the execution order of specific partitioning methods called by the data serialization language file configuration.

[0051] Multiple partitioning methods can be used. Manual partitioning can be based on a configuration containing boundaries. The boundary can be the last node of a section. The boundary can include the last node name and section input parameters. Virtual partitioning can create a single section with the nodes in a section, and the number of sections is 1. Extreme partitioning can divide the graph into multiple sections, with each section containing one node, and the number of nodes is equal to the number of sections. Greedy partitioning can check the rules for validating sections based on the section class checker function. Greedily search for nodes to assign the start index and end index of a section. This may take approximately O(n) time, where n is the number of nodes. The partitioning of dynamic programming partitioning can be the same as greedy partitioning. In contrast, dynamic programming partitioning uses dynamic programming and takes O(n^2) time.

[0052] The greedy partitioning attempts to minimize the number of segments. The greedy partitioning method has a segment list, and there are loop segments within the segment list, starting at 0 and ending at 1. The greedy partitioning searches from start to end for the largest list of end-while nodes that satisfy the hardware constraint checker. A new segment with a start and an end is added to the segment list, which starts at the end of the first segment list, ends at the end plus 1, and returns the new segment list. The complexity of the greedy segment may require time O(n*x*y), where n is the number of nodes, x is the number of strip size x selections, and y is the number of strip size y selections in the worst case.

[0053] Figure 4 The boundary candidate search for boundary candidates is shown. The boundary candidate search sets the start node identifier 410 and sets the initial end 412. Also, it searches for the end node identifier 416 starting from the start node identifier 414.

[0054] The partition segment checker can be based on rules that are registered and grouped by category. The segment checker can check the byte tensor movement mask and weight overflow, and check the byte tensor direct memory access input and output overflow. The segment checker can check the overlapping buffer size overflow, and check the data buffer of the node input value and the node output value size overflow. The segment checker can check to ensure that the number of input tensors and output tensors is less than a predetermined limit.

[0055] The partition segment checker can also check the output tensor strip size of the segment and return an error if it fails. The output tensor strip size ensures that the normal node strip size is greater than the kernel size. The last strip size of the last node in the segment is less than or equal to the normal strip size. If the number of strips is greater than 1, the first strip size of the last node in the segment is greater than or equal to the minimum first strip size. The normal node strip size is a multiple of 4. The segment checker can check the memory access rules, and the segment output is not used by the segment nodes. The segment checker can check the input tensor shape. In one example, the first segment DMA_IN has the same height and width. In the second example, (the first DMA_IN shape 1 / the second DMA_IN shape 2) can be 1, 2, 4, or 8.

[0056] The Byte Tensor Move Mask and Weight (BTMW) uses a checker to move the directed acyclic graph, move mask, and weights from memory to the BTMW. When the system receives a start signal, the entire directed acyclic graph can be loaded into the BTMW, while the move mask and weights are loaded section by section. The compiler ensures that the size of the move mask for the current section and the weights plus the directed acyclic graph do not exceed the BTMW size. The individual size can be a multiple of 64 bytes. The move mask is not considered part of the neural network exchange (NNX).

[0057] The size of the directed acyclic graph can be limited to a predetermined size, such as 128 KB. Nodes have a weight size function that returns the packed weight size of the node. The section weight size is the sum of the section node weight sizes and is used as the BTMW_USEAGE.

[0058] The Byte Tensor Direct Memory Access Input and Output (BTMD) uses a checker to utilize the BTMD that stores the input tensor (DMA_IN) and output tensor (DMA_OUT). The BTMD stores tensor stripes and is divided into multiple lanes, and the compiler ensures that the tensor stripes fit into the lanes. The BTMD use checker determines the number of BTMD lanes based on the number of input tensors and the number of output tensors. When determining the input tensor stripes and output tensor stripes in the BTMD, the BTMD stripe pitch can be found in the z direction, the combined z and x directions, and the y direction, where the final stripe size is the BTMD stripe pitch in the combined z and x directions multiplied by the pitch in the y direction.

[0059] The overlapping buffer size checker uses an overlapping buffer to determine the overlapping data between stripes. Different node types have different determination methods to determine the stripe-dependent overlapping buffer. The section maximum values of the node input values and node output values are determined by the nodes in that section.

[0060] Figure 5 A stripe size allocation workflow is depicted, which includes starting from section 510, allocating 512 the stripe size height and width to the section input and outputting the input values 514. Performing the collection 516 of the input stripe size and allocating 518 the stripe size input output to the node 520 hardware properties. Determining 522 the output stripe size and updating 524 the output to the input values.

[0061] The strip size allocation workflow may include: setting the strip size of a partition segment, represented as the input tensor strip size of DMA_IN. In a directed acyclic graph, the partition segment has a direct memory access input tensor (DMA_IN) and a direct memory access output tensor (DMA_OUT) defined at the beginning. This process workflow may determine hardware attributes based on the input tensor strip size, output tensor strip size, input node strip size, output node strip size, BTMD usage, OVBUF size, and DBUF. This process workflow may allocate the partition segment input tensor based on the first input tensor of the partition segment. Normal strip sizes may be set for the input node strip size and the output node strip size.

[0062] The strip window may move in a zigzag pattern in the vertical direction, and at each strip position, the segment input tensor strip size, output tensor strip size, input node strip size, and output node strip size may be determined. For the next strip position, the previous x-position count and y-position count may be inherited. It may be determined whether the strip is within range in the x and y directions. The top, bottom, and the last position in the x direction and the last position in the y direction of the strip specifying the segment are represented to indicate the position of the strip. For the input tensor of the segment, the input tensor strip size is set, and the hardware attributes are updated to reflect the input tensor strip size. The hardware attributes may also reflect the input and output nodes of the segment, the input node strip size, and the output node strip size. The input node strip size and the output node strip size may be assigned as the node input strip size and the node output strip size in the hardware attributes. After determining the hardware attributes, the segment checker may determine whether the segment is valid.

[0063] Figure 6 An exemplary zigzag search of strip sizes is shown, where the strip sizes are determined in the zigzag pattern 610.

[0064] Figure 7 Previously saved strips 0 (710), 1 (712), and 2 (714) are shown. The spans of the previously saved strips are 2 (722, 724, 726, 728, 730, and 732). Blocks 746 and 748 show an overlap of size 2. The sizes of the previously saved strips 710, 712, and 714 are 4. Masking blocks may be used in the next strip based on the previous strips. The next strip includes strips 0 (716), 1 (718), and 2 (720). The spans of the new strips are 2 (734, 736, 738, 740, 742, and 744). Blocks 750 and 752 have an overlap of size 1. The sizes of the new strips 716, 718, and 720 are 3, 4, and 5 respectively.

[0065] Figure 8An exemplary method of constructing a sub - graph is depicted, including: receiving 810 a directed acyclic graph (DAG); partitioning 812 the directed acyclic graph into at least one section; determining 814 at least one hardware property. The method further includes: determining 816 at least one DAG hardware limit for at least one section; and determining 818 a list of the maximum consecutive nodes for at least one section, wherein in the list of the maximum consecutive nodes, the at least one hardware property satisfies the at least one DAG hardware limit.

[0066] Figure 9 An exemplary method of constructing a sub - graph is shown, including: loading 910 a directed acyclic graph from volatile memory into at least one BTMW; determining 912 the node weight size of at least one node in at least one section of the at least one BTMW; adding 914 the section weight sizes of at least one node in at least one section of the at least one BTMW; and determining 916 whether the sum of the directed acyclic graph and the added section weight sizes exceeds a pre - determined maximum BTMW size.

[0067] Figure 10 An exemplary method of constructing a sub - graph is depicted, including: determining 1010 whether the byte - tensor movement mask and weight (BTMW) of at least one section exceeds a pre - determined BTMW usage overflow; and determining 1012 whether the byte - tensor direct memory access input and output (BTMD) of at least one section exceeds a pre - determined BTMD usage overflow. The method may further include: determining 1014 whether the overlapping buffer (OVBUF) of at least one section exceeds a pre - determined OVBUF size overflow; determining 1016 whether the data buffer (DBUF) of the input nodes and output nodes of at least one section exceeds a pre - determined DBUF size overflow; and updating 1018 at least one hardware property with BTMW, BTMD, OVBUF, and DBUF.

[0068] The method of constructing the sub - graph may further include: loading at least one strip of input tensors and output tensors, determining whether at least one strip exceeds a pre - determined strip size, and determining whether the number of input tensors exceeds a pre - determined input tensor number limit. The method may further include: determining whether the number of output tensors exceeds a pre - determined output tensor number limit, and determining whether the section output tensor strip size exceeds a pre - determined output tensor strip size limit. The overlapping buffer may store overlapping data between strips.

[0069] Figure 11 An exemplary method of allocating strip sizes in a sub - graph is depicted, including: receiving 1110 a directed acyclic graph; partitioning 1112 the directed acyclic graph into at least one section; determining 1114 the input tensor strip size; and updating 1116 at least one hardware property based on the input tensor strip size.

[0070] The method of allocating strip sizes in the allocation sub-graph may further include: determining whether the byte tensor movement mask and weight (BTMW) of at least one section exceeds a pre-determined BTMW usage overflow; and determining whether the byte tensor direct memory access input and output (BTMD) of at least one section exceeds a pre-determined BTMD usage overflow. The method may further include: determining whether the overlapping buffer (OVBUF) of at least one section exceeds a pre-determined OVBUF size overflow, determining whether the data buffer (DBUF) of the input and output nodes of at least one section exceeds a pre-determined DBUF size overflow, and updating at least one hardware attribute with BTMW, BTMD, OVBUF, and DBUF.

[0071] Figure 12 An exemplary method of allocating strip sizes in the allocation sub-graph is depicted, including: allocating 1210 input tensor strips based on a first input tensor of at least one section; and setting 1212 the input tensor strip size and the output tensor strip size of at least one section. The method may further include: determining 1214 the input tensor strip size of the input tensor strip, the input node strip size and the output node strip size of at least one section in a zigzag pattern; and allocating 1216 the input tensor strip size, the output tensor strip size, the input node strip size and the output node strip size of at least one hardware attribute.

[0072] Those skilled in the art can understand that the various illustrative blocks, modules, elements, components, methods, and algorithms described herein can be implemented as electronic hardware, computer software, or a combination of both. To illustrate the interchangeability between hardware and software, the various illustrative blocks, modules, elements, components, methods, and algorithms have been generally described above according to their functions. Whether this function is implemented as hardware or software depends on the specific application and the design constraints of the system. Those skilled in the art can implement the described functions in different ways for each specific application. Without departing from the technical scope of the subject matter, the various components and blocks can be deployed differently, for example, in a different order, or divided in a different way.

[0073] It should be understood that the specific order or hierarchy of the steps in the disclosed process is an illustration of an exemplary method. It can be understood that the specific order or hierarchy of the steps in the disclosed process can be rearranged according to design preferences. Some steps can be performed simultaneously. The appended method claims present the elements of the various steps in an exemplary order and do not imply a limitation to the specific order or hierarchy presented.

[0074] The foregoing description is provided to enable a person skilled in the art to implement the various aspects described herein. The foregoing description provides various examples of the subject technology, but the subject technology is not limited to these examples. Modifications to these aspects will be apparent to a person skilled in the art, and the general principles defined herein may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown herein but are to be accorded the full scope consistent with the language claimed. References to elements in the singular are not intended to mean "one and only one" unless specifically stated otherwise, but rather "one or more." Unless specifically stated otherwise, the term "some" means one or more. Masculine pronouns (e.g., "his") include feminine (e.g., "her") and neuter (e.g., "its"), and vice versa. Headings and subheadings, if any, are used for convenience only and do not limit the disclosure. The predicates "configured to," "operable as," and "programmed to" do not imply any particular tangible or intangible modification of the subject, but rather may be used interchangeably. For example, a processor configured to monitor and control operations or components may also refer to a processor programmed to monitor and control operations or a processor operable to monitor and control operations. Similarly, a processor configured to execute code may be interpreted as a processor programmed to execute code or a processor operable to execute code.

[0075] Phrases such as "aspect" do not imply that the aspect is essential to the subject technology or applicable to a configuration of the subject technology. Disclosure related to an aspect may apply to a configuration, or one or more configurations. An aspect may provide one or more examples. Phrases such as "aspect" may refer to one or more aspects, and vice versa. Phrases such as "embodiment" do not imply that the embodiment is essential to the subject technology or applicable to a configuration of the subject technology. Disclosure related to an embodiment may apply to an embodiment, or one or more embodiments. An embodiment may provide one or more examples. Phrases such as "embodiment" may refer to one or more embodiments, and vice versa. Phrases such as "configuration" do not imply that the configuration is essential to the subject technology or applicable to a configuration of the subject technology. Disclosure related to a configuration may apply to a configuration, or one or more configurations. A configuration may provide one or more examples. Phrases such as "configuration" may refer to one or more configurations, and vice versa.

[0076] As used herein, "example" means "serving as an example or illustration." Any aspect or design described herein as an "example" is not necessarily to be construed as superior to other aspects or designs.

[0077] Equivalents of structures and functions of elements of the various aspects described in this disclosure that are known or later become known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. In addition, nothing disclosed herein is intended to be dedicated to the public domain, whether or not such disclosure is expressly recited in the claims. Further, since "comprising" is interpreted as a transitional term in the claims, for the terms "comprising", "having", etc. used in the specification or claims, such terms are intended to be included in a manner similar to the term "comprising".

[0078] References to "one embodiment", "an embodiment", "some embodiments", "various embodiments", etc. indicate that a particular element or feature is included in at least one embodiment of the present disclosure. Although these phrases may appear in different places, these units do not necessarily refer to the same embodiment. In connection with the present disclosure, those of ordinary skill in the art can design and combine any one of the various mechanisms suitable for achieving the above functions.

[0079] It should be understood that the present disclosure only teaches examples of illustrative embodiments, and those of ordinary skill in the art can easily design various variations of the present disclosure after reading the present disclosure, and the scope of the present disclosure is determined by the following claims.

Claims

1. A method for constructing a subgraph, comprising: Receiving a directed acyclic graph; Dividing the directed acyclic graph into at least one section; Determining at least one hardware property; Determining at least one directed acyclic graph hardware limit of the at least one section; Determining a list of the maximum continuous nodes of the at least one section, wherein in the list of the maximum continuous nodes the at least one hardware property satisfies the at least one directed acyclic graph hardware limit; And Determining whether the byte tensor movement mask and weights of the at least one section exceed a predetermined byte tensor movement mask and weights usage overflow, including: Loading the directed acyclic graph from volatile memory into at least one byte tensor movement mask and weights; Determining the node weight size of at least one node in at least one section of the at least one byte tensor movement mask and weights; Adding up the section weight sizes of at least one node in at least one section of the at least one byte tensor movement mask and weights; and Determining whether the sum of the directed acyclic graph and the added-up section weight sizes exceeds a predetermined maximum byte tensor movement mask and weights size.

2. The method for constructing a sub-graph according to claim 1, wherein Further comprising: Determining whether the at least one byte tensor direct memory access input and output of the at least one section exceed a predetermined byte tensor direct memory access input and output usage overflow.

3. The method for constructing a sub-graph according to claim 2, wherein Further comprising: Loading at least one strip of input tensors and output tensors; And Determining whether the at least one strip exceeds a predetermined strip size.

4. The method for constructing a sub-graph according to claim 2, wherein Further comprising: Determining whether the overlapping buffer of the at least one section exceeds a predetermined overlapping buffer size overflow.

5. The method for constructing a sub-graph according to claim 4, wherein The overlapping buffer stores overlapping data between at least one strip.

6. The method for constructing a subgraph according to claim 2, wherein Further comprising: Determining whether the data buffer of the input nodes and output nodes of the at least one section exceeds a predetermined data buffer size overflow of the input nodes and output nodes.

7. The method for constructing a sub-graph according to claim 6, wherein Further comprising: Determining whether the number of input tensors exceeds a predetermined input tensor number limit.

8. The method for constructing a subgraph according to claim 7, wherein Further comprising: Determining whether the number of output tensors exceeds a predetermined output tensor number limit.

9. The method for constructing a subgraph according to claim 8, wherein Further comprising: Determining whether the section output tensor strip size exceeds a predetermined output tensor strip size limit.

10. A method for allocating strip sizes in a subgraph, comprising: Receiving a directed acyclic graph; Dividing the directed acyclic graph into at least one section; Determining the input tensor strip size; And Updating at least one hardware property based on the input tensor strip size; Wherein, the method further comprises: Determining whether the byte tensor movement mask and weights of the at least one section exceed a predetermined byte tensor movement mask and weights usage overflow; Determining whether the at least one byte tensor direct memory access input and output of the at least one section exceed a predetermined byte tensor direct memory access input and output usage overflow; Determining whether the overlapping buffer of the at least one section exceeds a predetermined overlapping buffer size overflow; Determining whether the data buffer of the input nodes and output nodes of the at least one section exceeds a predetermined data buffer size overflow of the input nodes and output nodes; and Update the at least one hardware property with the byte tensor movement mask and weights, the byte tensor direct memory access input and output, the overlapping buffer, and the data buffering of the input node and output node.

11. The method for allocating stripe sizes in a distribution subgraph according to claim 10, wherein, Further comprising: Allocate input tensor strips based on a first input tensor of the at least one section; Set the input tensor strip size and the output tensor strip size of the at least one section; Determine the input tensor strip size of the input tensor strips, the input node strip size and output node strip size of the at least one section in a zigzag pattern; And Allocate the input tensor strip size, the output tensor strip size, the input node strip size, and the output node strip size of the at least one hardware property.

Images