Out-of-order parallel maximum flow and minimum cut method implemented on energy-efficient FPGA

Abstract

The present invention discloses an out-of-order parallel maximum flow / minimum cut method implemented by a high-energy-efficiency FPGA. The present invention folds a single-layer large two-dimensional trellis graph into a multi-layer small trellis graph. This method has two advantages: the folded grid architecture can store and process a trellis graph whose size is much larger than the size of the processor array; this architecture also gives the two-dimensional processor array vertical freedom, allowing it to use the vertical freedom to improve the parallel potential of the architecture. The folded grid architecture proposed by the present invention enables a small-sized processor array to have the ability to process a trellis graph whose size is much larger than its own. At the same time, this folded grid architecture can utilize the axial symmetry of the folding to greatly reduce the cross-border transmission of data in the processor array, thereby reducing the additional overhead caused by data movement. The present invention also proposes an out-of-order parallel execution technology, which can fully tap the parallel potential in the folded grid architecture.

Description

Technical Field

[0001] The present invention relates to an implementation method of an FPGA accelerator for a high-speed and high-energy-efficiency maximum flow / minimum cut algorithm capable of solving graph cut problems in large lattice graphs. Background Art

[0002] Max flow / min cut algorithms are widely used in optimization tasks, such as neural network optimization [1], physically unreplicable functions [2], compiler optimization [3], and computer vision tasks [4]. Previous work [5,6] has explored accelerating the max flow / min cut algorithm on general computing platforms. During the graph cut algorithm operation, a large number of DRAM random accesses are generated, which will cause a large memory access delay and affect the calculation time. JF-cut [5] proposed a jump operation method and divided the grid graph nodes into different parts according to the odd-even order, thereby alleviating the memory read and write conflicts generated during GPU parallel acceleration. [6] proposed a cache-friendly compact data storage structure for multi-level memory models on a general computing platform. Using this data structure to calculate the grid graph max flow / min cut algorithm can effectively reduce the performance loss caused by memory access. However, since the grid graph max flow / min cut algorithm has strong data dependencies between adjacent nodes in the graph, it is impossible to completely alleviate the data dependencies caused by the architecture on a general computing platform. Even with the most advanced GPU platforms, general-purpose computing platforms still cannot quickly (60 frames per second) process the maximum flow / minimum cut problem on large trellis graphs (1080x1920 nodes). Furthermore, the enormous energy consumption required by general-purpose computing platforms is unacceptable for current work.

[0003] To solve these problems, current work will use a more flexible and efficient FPGA platform to accelerate such algorithms. [7] is the most advanced implementation of the maximum flow / minimum cut algorithm on the FPGA platform in recent years. It fully explores the potential for parallelism in the grid graph. It corresponds each computing node one-to-one to each node in the grid graph, and through checkerboard scheduling, all computing nodes can run the algorithm simultaneously without the need for additional time processing or waiting for data conflicts. In addition, it also proposes the "RipplePush" push method to further improve the parallelism during the operation of the maximum flow / minimum cut algorithm. In addition, it uses the "EarlyTermination" technology to reduce redundant calculations in the calculation process from the algorithm dimension, accelerating the convergence time of the algorithm.

[0004] However, since [7] uses an architecture in which computing nodes correspond one-to-one with nodes in the graph, it consumes a large amount of FPGA resources and cannot solve larger grid graphs (grid graphs with 1080x1920 nodes).

[0005] References

[0006] [1]J.Li,M.Peng,Q.Li,M.Peng,and M.Yuan,“Glite:A fast andefficientautomatic graph-level optimizer for large-scale dnns,”in Proceedingsof the 59th ACM / IEEE Design Automation Conference(DAC’22).NewYork,NY,USA:Association for Computing Machinery,2022,p.199–204.

[0007] [2]M.Li,J.Miao,K.Zhong,and D.Z.Pan,“Practical public puf enabledbysolving max-flow problem on chip,”in Proceedings of the 53rdAnnual DesignAutomation Conference(DAC’16).New York,NY,USA:Association for ComputingMachinery,2016.

[0008] [3]S.Reder and J.Becker,“Wcet-aware code generation andcommunicationoptimization for parallelizing compilers,”in Proceedings of the23rd Conference on Design,Automation and Test in Europe,ser.DATE’20.

[0009] [4]P.M.Jensen,N.Jeppesen,A.B.Dahl,and V.A.Dahl,“Review ofserial andparallel min-cut / max-flow algorithms for computer vision,”IEEETransactions onPattern Analysis and Machine Intelligence,pp.1–1,2022.

[0010] [5] Y.Peng, L.Chen, F.-X.Ou-Yang, W.Chen, and J.-H.Yong, "JF-cut: A parallelgraph cut approach for large-scale image and video," IEEETransactions on ImageProcessing, vol.24, no.2, pp.655–666, 2015.

[0011] [6] O. Jamriska, D. Sˇykora, and A. Hornung, “Cache-efficient graph cuts on structured grids,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 3673–3680.

[0012] [7] G.Yan, Summary of the Invention

[0013] The purpose of the present invention is to provide a method for implementing an FPGA accelerator of a high-speed and high-energy-efficiency maximum flow / minimum cut algorithm that can solve the graph cut problem of large lattice graphs.

[0014] In order to achieve the above object, the technical solution of the present invention is to provide a high-energy-efficiency FPGA-implemented out-of-order parallel maximum flow / minimum cut method, which is characterized by comprising the following steps:

[0015] Step 1: Preprocess the trellis graph into a folded grid data structure according to the size of the processor array. Let (H, W) be the size of the trellis graph, (h, w) be the size of the processor array, and (X, Y) be the coordinates of the nodes in the trellis graph, where X∈[1, H] and Y∈[1, W]. The preprocessing process includes the following steps:

[0016] Step 101: Convert the node coordinates (X, Y) in the grid graph to the processor array coordinate system. The converted coordinates are (x, y), where x∈[1, h] and y∈[1, w]. After the coordinate conversion, fold all nodes of a large grid graph into a multi-layer small grid graph, and the size of the small grid graph is the size of the processor array.

[0017] Step 102: Input the multi-layer small trellis data obtained in step 101 into the accelerator according to the node coordinates in the processor array coordinate system after conversion. The trellis nodes corresponding to the same coordinates are stored in the same processor, and (h, w) are input at the same time. and As a parameter, ceil(a) returns the smallest integer greater than or equal to a, where (h,w) is used to initialize the specific clock delay in the input and output process of the accelerator. and Used to determine whether the currently processed node data is mirrored, and Coordinates are used to represent a specific layer of a multi-layer small grid map;

[0018] Step 2: Each node in the multi-layer small trellis graph contains data (ef) indicating the maximum amount of traffic it can accommodate, data (edge) indicating the capacity of edges pointing to surrounding nodes, data (h) indicating the node's height, and data (sink) indicating the capacity of edges pointing to a virtual sink. During the input phase, this data is stored in the corresponding processor in the order of the processor array coordinates. After the accelerator loads all the data, a global relabeling operation is performed. The processor unit uses FIFO-based out-of-order execution technology to process each layer of the small trellis graph:

[0019] In the FIFO-based out-of-order execution technology, when performing the global relabeling operation: traverse all nodes in the small grid graph. If the capacity sink data of the nodes in the small grid graph is not 0, initialize these nodes as seed points and put them all into the FIFO queue. Each time a node is taken out from the FIFO queue for calculation, update the node height h data of the current node to the height read from the FIFO queue plus 1; if the capacity edge data of the edge pointing to the cache node in the FIFO queue of the current node is greater than 0, increase this node; when the FIFO queues in all processor nodes are empty, the global relabeling operation is completed. At the same time, during the execution process, if the ef data in the node is greater than 0, it is stored in the pending FIFO of the next step Push operation;

[0020] Step 3: If the pending FIFO for the Push operation is not empty, execute the Push operation to complete the streaming operation.

[0021] Preferably, in step 101, the coordinate system is transformed using the following formula:

[0022] like is an odd number and is an odd number, then

[0023] like is an odd number and is an even number, then

[0024] like is an even number and is an odd number, then

[0025] like is an even number and is an even number, then

[0026] Preferably, in step 3, the Push operation includes the following steps:

[0027] Step 301: Read the nodes whose ef data is greater than 0 from the FIFO queue dedicated to the Push operation, and notify all the pointing nodes whose edge capacity edge data is greater than 0 to send their node height h data back to the current node;

[0028] Step 302: After the surrounding nodes return the node height h data, the height of the surrounding nodes is determined based on the current node's height hcurrent to see if it is hcurrent-1. If so, the ef data stored in the current node is pushed to all surrounding nodes that meet the conditions. Let the pushed flow be "flow", and the data in the current node is updated to: ef = ef-flow, edge = edge-flow; after receiving the flow, the surrounding nodes simultaneously update their own node data to: ef = ef+flow, edge = edge+flow;

[0029] Step 303: When the data in the FIFO queue dedicated to the Push operation is processed, the Push operation is completed, all processor units enter the reset phase, and traverse all nodes in the grid graph and reset the node height h data to 0, and then re-perform the global remarking operation in step 2. If the FIFO queue dedicated to the Push operation is empty after the global remarking operation is completed, the algorithm is terminated.

[0030] Compared with the existing technical solutions, the innovation of the present invention lies in:

[0031] 1) The present invention folds a single-layer large two-dimensional trellis graph into a multi-layer small trellis graph. This method has two advantages: the folded grid architecture can store and process a trellis graph that is much larger than the size of the processor array; this architecture also gives the two-dimensional processor array vertical freedom, allowing it to use this vertical freedom to increase the parallel potential of the architecture. The folded grid architecture proposed by the present invention enables a small-sized processor array to process a trellis graph that is much larger than itself. At the same time, this folded grid architecture can utilize the axial symmetry of the folding to significantly reduce cross-border data transmission within the processor array, thereby reducing the additional overhead caused by data movement.

[0032] 2) This invention also proposes an out-of-order parallel execution technology that fully exploits the parallel potential of the folded grid architecture. This technology leverages the ability of each processor unit to simultaneously access multiple layers of small trellises. It detects nodes in the multi-layer trellis that require computation and temporarily stores the data in a FIFO within the FPGA, enabling out-of-order parallel execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Illustration of the data storage format in the folded mesh architecture, including: (a) an 8x8 grid graph; (b) a 4x4 processor array storing the grid graph in (a), with the number on each processor unit representing the node of the graph it stores; (c) a schematic diagram of data storage in the folded mesh architecture.

[0034] Figure 2 Schematic diagram of a folded grid architecture for a 4x4 processor array.

[0035] Figure 3 This diagram illustrates the sequential processing flow of each processor unit. Arrows represent data exchange, and the finger points to the small grid layer currently being processed by the processor. Dark gray nodes correspond to nodes being processed, while IDLE indicates that the processor is idle, and light gray indicates nodes awaiting processing.

[0036] Figure 4 This is a schematic diagram of the execution of an 8x8 grid graph under out-of-order parallel execution technology. The numbers in (a) represent the depth of the FIFO in each processor unit in the 4x4 processor array. (b) shows the processing flow chart of each processor unit, and the arrows represent data communication. Different grayscales represent the processing of different layers of the 4x4 small grid graph (see Figure 1 ). The triangle symbol at the top of each iteration represents the head data of the FIFO, the square represents the tail data, and white represents no processing tasks.

[0037] Figure 5 Execute state diagram for accelerator DETAILED DESCRIPTION

[0038] Below in conjunction with specific embodiment, further set forth the present invention.Should be understood that these embodiments are only used to illustrate the present invention and are not used in limiting the scope of the present invention.In addition, should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall equally within the scope limited by the appended claims of the application.

[0039] The architecture of the algorithm disclosed in this embodiment implemented in FPGA is as follows Figure 2 As shown in the figure, the workflow of the accelerator is as follows Figure 4 As shown, the specific process is as follows:

[0040] Step 1: First, you need to Figure 1 The large-scale trellis graph shown in (a) is preprocessed into a collapsed grid data structure based on the size of the processor array. Let (H, W) be the size of the trellis graph, (h, w) be the size of the processor array, and (X, Y) be the coordinates of the nodes in the trellis graph, where X∈[1,H] and Y∈[1,W]. The preprocessing process includes the following steps:

[0041] The trellis diagram is folded according to the size of the processor array. The corresponding nodes (X, Y) in the trellis diagram will be converted to the processor array coordinate system. The converted coordinates are assumed to be (x, y), where x∈[1,h] and y∈[1,w]. The conversion method is as follows:

[0042] like is an odd number and is an odd number, where ceil(a) returns the smallest integer greater than or equal to a (e.g., ceil(0.1) = 1), then it is of type a), and:

[0043]

[0044] like is an odd number and If is an even number, then it belongs to type b), and there are:

[0045]

[0046] like is an even number and If is an odd number, then it is type c), and there are:

[0047]

[0048] like is an even number and If is an even number, then it is type d), and there are:

[0049]

[0050] After the conversion described above, all nodes of a large trellis graph are folded and mapped into a multi-layered small trellis graph, where the size of the small trellis graph is the size of the processor array. In the above formula, category a) indicates that the data does not need to be mirrored, category b) indicates that the data needs to be mirrored along the y-axis, category c) indicates that the data needs to be mirrored along the x-axis, and category d) indicates that the data needs to be mirrored along the x-axis and then along the y-axis.

[0051] After that, the grid data is input into the accelerator according to the converted processor array coordinates, and (h,w), and As parameters, (h,w) is used to initialize the specific clock delay in the input and output process of the accelerator. and Used to determine whether the currently processed node data is mirrored. The coordinates can also represent a specific layer of a multi-layer small trellis. Since the coordinates of the nodes in each layer of the small trellis are the size of the processor array, the trellis nodes corresponding to the same coordinates are stored in the same processor. For example, Figure 1 In (a), nodes 27, 28, 35, and 36 are all stored in the PE15 processor node. When a node in the PE15 processor interacts with nodes in other layers, this architecture eliminates the need for collaborative processing with other processors. Consequently, this design eliminates the need for multi-processor collaboration required for cross-layer data transfers in a non-folded architecture. Instead, a single processor performs this operation internally, reducing both time and hardware resource overhead.

[0052] Step 2: First, each node in the grid graph contains data (ef) indicating the maximum amount of traffic it can handle, edge data indicating the capacity of its edges pointing to surrounding nodes, and node height data (h). It also contains data indicating the capacity of its edges pointing to a virtual sink. During the input phase, this data is stored in the corresponding processors in the order of the processor array coordinates. After the accelerator has loaded all the data, it performs a global relabeling operation, which is as follows:

[0053] Traverse all nodes in the grid graph. If the sink data of the node is greater than 0, initialize these nodes as seed points and set their height data h to 1.

[0054] Then, take these seed points as root nodes, traverse all nodes pointed to by the edges in the root node, and inform the surrounding nodes that this node is the root node.

[0055] After receiving the data, the surrounding nodes read the capacity of the edge pointing to the root node. If it is greater than 0, the node is included in the tree of the root node and the height data h of the node is set to the height of the root node hroot+1.

[0056] The global rescaling operation will repeat the above root node operation until there are no more nodes in the trellis graph that can be added to the tree, and then stop.

[0057] In the past, the sequential execution operation is to synchronize the processor array so that it executes each layer of the small grid graph in a loop. During the execution process, the data transmitted across the layers will be delayed until the corresponding layer is executed. The execution flow chart of the processor unit is as follows Figure 3 As shown, when processing each layer of small trellises in sequence, a large number of processors will be idle. Therefore, the present invention proposes a FIFO-based out-of-order execution technology to open up the cross-layer dimension of the multi-layer stacked small trellises in the folded grid architecture in step 1.

[0058] For example, in the out-of-order execution technology, when performing a global rescaling operation: traverse all nodes in the trellis graph, and if the node sink data in the trellis graph is not 0, initialize these nodes as seed points. Put them all into the FIFO queue, take out a node each time for calculation, and update the height h1 information of the current node to the height h0+1 read from the FIFO. If the edge data of the edge of the current node pointing to the cache node in the FIFO is greater than 0, then grow this node. In this way, each processor unit can independently handle cross-layer data transmission and is no longer limited to synchronous sequential execution. From a macro perspective, each processor is executing small trellis nodes of different layers out of order, such as Figure 4 As shown in (b), this significantly reduces the cycles required to complete a task and improves processor utilization. The global relabeling operation is complete when the FIFOs in all processor nodes are empty. During execution, if the ef data in a node is greater than 0, it is stored in the FIFO for the next push operation.

[0059] Step 3: If the pending FIFO for the Push operation is not empty, the Push operation is then executed to complete the push operation. The Push operation process is as follows: nodes with ef data greater than 0 are read from the FIFO dedicated to the Push operation, and all nodes with edges greater than 0 are notified to send their node height data h back to the current node. After the surrounding nodes return the height data h, the height of the surrounding nodes is determined based on the current node's height hcurrent to see if it is hcurrent-1. If the conditions are met, the flow ef stored in the current node is pushed to all surrounding nodes that meet the conditions. Assuming the pushed flow is flow, the data in the current node is updated to: ef = ef-flow, edge = edge-flow. After receiving flow, the surrounding nodes simultaneously update their own node data to: ef = ef+flow, edge = edge+flow. When the data in the FIFO dedicated to the Push operation is processed, the Push operation is completed. All processor units enter the reset phase, traverse all nodes in the grid graph, and reset the node height data h to 0. Then, the global relabeling operation in step 2 is repeated. If the FIFO dedicated to the Push operation is empty after the global relabeling operation is completed, the algorithm is terminated.

[0060] Ultimately, compared to the most advanced max-flow / min-cut accelerator, our design leverages out-of-order parallel execution during the global relabeling phase, increasing processor unit utilization by up to 8.35 times. Furthermore, when performing image segmentation tasks on the Middlebury and DAVIS2016 datasets, our design is 5.4 times faster than the most advanced accelerator on an FPGA platform.

[0061] The above technical solution can be used for maximum flow / minimum cut tasks that require low power consumption and real-time processing. This architecture is not only suitable for FPGA design, but also for ASIC design.

Claims

1. A high-efficiency FPGA-based out-of-order parallel maximum flow / minimum cut method, characterized by: The following steps are involved: Step 1: Preprocess the trellis graph into a folded grid data structure according to the size of the processor array. Let (H, W) be the size of the trellis graph, (h, w) be the size of the processor array, and (X, Y) be the coordinates of the nodes in the trellis graph, where X∈[1, H] and Y∈[1, W]. The preprocessing process includes the following steps: Step 101: Convert the node coordinates (X, Y) in the grid graph to the processor array coordinate system. The converted coordinates are (x, y), where x∈[1, h] and y∈[1, w]. After the coordinate conversion, fold all nodes of a large grid graph into a multi-layer small grid graph, and the size of the small grid graph is the size of the processor array. Step 102: Input the multi-layer small trellis data obtained in step 101 into the accelerator according to the node coordinates in the processor array coordinate system after conversion. The trellis nodes corresponding to the same coordinates are stored in the same processor, and (h, w) are input at the same time. and As a parameter, ceil(a) returns the smallest integer greater than or equal to a, where (h,w) is used to initialize the specific clock delay in the input and output process of the accelerator. and Used to determine whether the currently processed node data is mirrored, and Coordinates are used to represent a specific layer of a multi-layer small grid map; Step 2: Each node in the multi-layer small trellis graph contains data (ef) indicating the maximum amount of traffic it can accommodate, data (edge) indicating the capacity of edges pointing to surrounding nodes, data (h) indicating the node's height, and data (sink) indicating the capacity of edges pointing to a virtual sink. During the input phase, this data is stored in the corresponding processor in the order of the processor array coordinates. After the accelerator loads all the data, a global relabeling operation is performed. The processor unit uses FIFO-based out-of-order execution technology to process each layer of the small trellis graph: In the FIFO-based out-of-order execution technology, when performing the global rescaling operation: traverse all nodes in the small grid graph. If the capacity sink data of the nodes in the small grid graph is not 0, these nodes are initialized as seed points and all of them are placed in the FIFO queue. Each time a node is taken out from the FIFO queue for calculation, the node height h data of the current node is updated to the height read from the FIFO queue plus 1; If the edge data of the edge pointing to the cache node in the FIFO queue of the current node is greater than 0, the node is increased. When the FIFO queues in all processor nodes are empty, the global relabeling operation is completed. At the same time, during the execution process, if the ef data in the node is greater than 0, it is stored in the pending FIFO of the next step Push operation. Step 3: If the pending FIFO for the Push operation is not empty, execute the Push operation to complete the streaming operation.

2. The out-of-order parallel maximum flow / minimum cut method implemented by a high-efficiency FPGA as claimed in claim 1, characterized in that: In step 101, the coordinate system is transformed using the following formula: like is an odd number and is an odd number, then like is an odd number and is an even number, then like is an even number and is an odd number, then like is an even number and is an even number, then 3. The out-of-order parallel maximum flow / minimum cut method implemented by a high-efficiency FPGA as claimed in claim 1, characterized in that: In step 3, the Push operation includes the following steps: Step 301: Read the nodes whose ef data is greater than 0 from the FIFO queue dedicated to the Push operation, and notify all the pointing nodes whose edge capacity edge data is greater than 0 to send their node height h data back to the current node; Step 302: After the surrounding nodes return the node height h data, the height of the surrounding nodes is determined based on the current node's height hcurrent to see if it is hcurrent-1. If so, the ef data stored in the current node is pushed to all surrounding nodes that meet the conditions. Let the pushed flow be "flow", and the data in the current node is updated to: ef = ef-flow, edge = edge-flow; after receiving the flow, the surrounding nodes simultaneously update their own node data to: ef = ef+flow, edge = edge+flow; Step 303: When the data in the FIFO queue dedicated to the Push operation is processed, the Push operation is completed, all processor units enter the reset phase, and traverse all nodes in the grid graph and reset the node height h data to 0, and then re-perform the global remarking operation in step 2. If the FIFO queue dedicated to the Push operation is empty after the global remarking operation is completed, the algorithm is terminated.

Images