An optimization compiler system for data flow architecture and an automatic generation method of data flow accelerator

Abstract

This invention discloses an optimized compiler system and an automated generation method for dataflow accelerators oriented towards dataflow architectures. The system, built on the MLIR framework, includes sequentially connected modules for front-end transformation, dataflow violation collaborative elimination, communication optimization, automated scheduling, and code generation. It eliminates coarse-grained dataflow violations through pattern-aware transformation and addresses fine-grained read-write inconsistencies by combining reduction rewriting and substitution mapping. It optimizes on-chip communication using a FIFO priority strategy and a violation-free reused buffer. It performs resource-aware three-stage automatic scheduling and coordinates parallel strategies through a global task-to-task collaborative optimization mechanism to avoid introducing new violations. This invention achieves end-to-end automated generation from C++ / PyTorch input to high-performance, deadlock-free dataflow accelerator design, significantly improving development efficiency and hardware performance.

Description

Technical Field

[0001] This invention relates to the fields of computer architecture and compiler technology, specifically to an optimized compiler system for dataflow architecture and an automated generation method for dataflow accelerators, which is particularly applicable to the design and optimization of dataflow accelerators based on field-programmable gate arrays (FPGAs). Background Technology

[0002] Dataflow architectures, with their extremely low latency, are particularly well-suited for tasks requiring frequent data movement and intensive computation. FPGAs, with their reconfigurable logic and highly customizable data paths, are ideal platforms for implementing dataflow accelerators. However, designing efficient dataflow accelerators typically requires complex implementations using hardware description languages ​​(HDLs). To simplify the development process, high-level synthesis (HLS) tools are widely used to automatically convert C / C++ code into HDL.

[0003] While existing commercial HLS tools (such as AMD Vitis HLS) provide basic dataflow scheduling primitives (e.g., dataflow pragma) to support pipelined execution between loops or functions, their optimization performance heavily depends on whether the code structure meets strict constraints. In practical applications, many hand-written algorithms struggle to directly satisfy these constraints, leading to limited performance optimization space and even synthesis failure. Developers often need to invest significant effort in manually refactoring code to adapt to dataflow patterns, significantly increasing development costs and time.

[0004] In recent years, to improve programming efficiency, researchers have proposed methods based on Domain-Specific Languages ​​(DSLs) or directly parsing inputs from C++, PyTorch, etc., into Intermediate Representations (IRs). These methods primarily focus on optimizing single computational kernels, while paying insufficient attention to cross-task dataflow collaborative optimization. Although some compiler solutions for multi-kernel dataflow optimization have been developed, attempting to automate dataflow accelerator generation, the following key issues remain: Inadequate handling of data flow violations: Existing tools struggle to systematically eliminate both coarse-grained and fine-grained data flow violations (such as single producer-multiple consumers, inconsistent access order, etc.), potentially leading to deadlocks or poor performance in the resulting design.

[0005] Inefficient communication: The lack of intelligent decision-making and optimization for on-chip buffers (such as FIFO and ping-pong buffers) and efficient management of off-chip high-bandwidth memory (HBM) limits data transfer efficiency.

[0006] Parallel strategy conflicts: Conflicts often occur between parallel optimization (such as loop partitioning, unrolling, pipeline) and data flow constraints. Existing methods mostly handle this in a decoupled manner, lacking global coordination, which leads to mutual constraints on optimization results.

[0007] Resource and performance imbalance: When exploring parallel strategies, the balance between resource constraints and performance is often overlooked, which can easily lead to resource overload or insufficient utilization.

[0008] In particular, as deep learning models continue to expand in scale and become increasingly complex in structure, the difficulty of building efficient, deep pipeline data flow accelerators has further intensified.

[0009] Therefore, there is an urgent need for an automated scheduling method that can comprehensively solve data flow violations, optimize communication mechanisms, and achieve resource awareness, in order to generate a high-performance, correctly executable data flow accelerator design. Summary of the Invention

[0010] To address the aforementioned problems, this invention proposes an optimized compiler system and an automated dataflow accelerator generation method for dataflow architectures. It aims to solve the issues of low development efficiency, poor optimization results, and susceptibility to dataflow violations and communication bottlenecks inherent in current High-Level Synthesis (HLS) tools when generating FPGA dataflow accelerators. The technical solution of this invention achieves automated generation from high-level language algorithm descriptions to high-performance, error-free dataflow accelerator designs through a globally collaborative compilation process.

[0011] The technical solution of the present invention is as follows: An optimized compiler system for a dataflow architecture is characterized by including: The front-end conversion module receives computational tasks written in C++ or defined in PyTorch, converts them into a unified intermediate representation, and outputs it to the data flow violation elimination module. The unified intermediate representation preserves the data dependencies and computational semantics of the original algorithm. The data flow violation collaborative elimination module receives the unified intermediate representation, systematically analyzes and eliminates data flow violations therein, and outputs an intermediate representation that conforms to the single producer-single consumer constraint and has consistent fine-grained access to the communication optimization module; at the same time, it feeds back the identified data access pattern and loop structure information to the scheduling optimization module. The communication optimization module receives the intermediate representation after violation elimination, performs on-chip buffer decision-making and automatic generation, manages off-chip data communication, and outputs the intermediate representation with embedded efficient communication structure to the scheduling optimization module; at the same time, it transmits the generated multiplexed buffer structure and its associated loop dimension information to the scheduling optimization module to guide safe parallelization. The scheduling optimization module receives access mode information from the data flow violation elimination module, buffer and dimension information from the communication optimization module, and parallelism constraints from user input. It performs resource-aware automatic parallel scheduling and inter-task collaborative optimization, and outputs a final intermediate representation with a determined parallel strategy to the code generation module. The code generation module receives the globally optimized final intermediate representation and generates hardware description code for the data flow accelerator that can be implemented on the FPGA, as well as the corresponding host-side driver code.

[0012] Furthermore, the data stream violation elimination module includes a coarse-grained violation elimination unit and a fine-grained violation elimination unit cascaded together with signals; The coarse-grained violation elimination unit receives the intermediate representation, identifies and reconstructs the code to eliminate violations between three types of nodes: single producer-multiple consumers, multiple producers-single consumer, and multiple producers-multiple consumers, and outputs an intermediate representation that conforms to the single producer-single consumer constraint to the fine-grained violation elimination unit. The fine-grained violation elimination unit receives the intermediate representation that conforms to the single producer-single consumer constraint, resolves the access count mismatch by rewriting through a reduction operation, resolves the access order inconsistency by generating through a permutation mapping, and outputs the intermediate representation that supports FIFO-based fine-grained pipelined execution to the communication optimization module.

[0013] Furthermore, the coarse-grained violation elimination unit performs mode-aware code transformation, specifically including: When a single producer-multiple consumer violation pattern is detected, an intermediate replication node is inserted in the data flow graph, and the original output buffer is replicated into multiple copies, so that each replication buffer serves only one consumer. When a multi-producer-single-consumer violation pattern is detected, multiple production tasks writing to the same buffer will be merged in a loop and the write operations will be combined through an internal temporary storage mechanism. When a multi-producer-multi-consumer violation pattern is detected, the violation pattern is decoupled into multiple independent single-producer-single-consumer pairs by copying the buffer and reallocating the read-write relationship.

[0014] Furthermore, in the fine-grained violation elimination unit, the reduction operation rewriting includes: Identify the cyclic regions involving reduction operations in the intermediate representation; Separate the index dimension and reduction dimension of the loop, and move the reduction dimension to the innermost layer of the loop; By introducing a temporary accumulation buffer, the multiple write operations to the output FIFO that were originally performed within the reduction loop are transformed into accumulation operations within the temporary buffer. After the reduction is completed, the final result is written to the output FIFO all at once, thus ensuring that the number of FIFO accesses by the producer and consumer are strictly matched.

[0015] Furthermore, in the fine-grained violation elimination unit, the permutation map generation includes: Taking computationally intensive loops as a reference, we analyze their data access order; The goal is to identify the producer or consumer cycle and analyze the current data access order. Construct a mapping relationship from the reference loop iteration space to the target loop iteration space; Based on the mapping relationship, the nested structure of the target loop is transformed by loop exchange, block division, or reordering to align its data output or consumption order with the requirements of the reference loop.

[0016] Furthermore, the communication optimization module includes an on-chip optimization unit and an off-chip management unit for signal interaction; wherein, the on-chip optimization unit receives the intermediate representation with eliminated violations, selects a buffer for the inter-task channel using a FIFO priority strategy, automatically analyzes the data multiplexing mode to generate a row buffer and a window buffer, and outputs the optimized intermediate representation to the scheduling optimization module. The off-chip management unit works in conjunction with the on-chip optimization unit to analyze the access patterns of model parameters, automatically segment and map them to multiple high-bandwidth memory channels, and generate host-side memory copy instructions to achieve burst transmission.

[0017] Furthermore, in the on-chip optimization unit, the automatically generated multiplexed buffer includes: Analyze the array access patterns in nested loops of computation tasks to identify reduction dimensions where data reuse exists; Based on the size of the iterative domain of the reduction dimension, the depth and width of the reuse buffer are automatically determined; Insert the corresponding buffer declarations and data maintenance logic into the code, and refactor the surrounding loops to ensure that the timing of data filling, sliding, and reading is correct and does not violate existing data flow constraints.

[0018] Furthermore, the scheduling optimization module includes a cyclically cascaded initial allocation unit, an iterative optimization unit, and a conflict coordination unit; The initial allocation unit allocates appropriate initial parallelism parameters to each loop based on the dimensional security information and performance estimation model from the communication optimization module. The iterative optimization unit receives the initial allocation result and, with the goal of improving overall throughput, iteratively adjusts the parallelism, block factor, and pipeline depth of each loop, and sends the new loop structure information to the conflict coordination unit after each adjustment. The conflict coordination unit continuously monitors the risk of data flow violations between tasks that may be caused by the adjustment of the parallel strategy. When a conflict that cannot be resolved by adjusting the loop is detected, a degradation instruction is sent to the communication optimization module to request that the buffer implementation of a specific channel be changed from FIFO to ping-pong buffer to ensure global correctness, and the final coordinated strategy is output to the code generation module.

[0019] Second, the present invention also provides an automated data stream accelerator generation method based on the above system, characterized by comprising the following steps: S1: Receives algorithms described in high-level languages ​​or machine learning frameworks, and generates a unified intermediate representation through front-end conversion; S2: Perform data flow violation elimination on the intermediate representation, sequentially perform coarse-grained violation transformation and fine-grained read / write coordination, and generate an intermediate representation that conforms to the data flow execution constraints; S3: Perform communication optimization on the intermediate representation that meets the constraints, including prioritizing the use of FIFO to implement on-chip communication, automatically generating data multiplexing buffers, and optimizing off-chip HBM data transmission; S4: Based on resource constraints, performance model and optimization structure information from step S3, considering the balance between resources and performance, execute automated parallel policy scheduling, and perform inter-task coordination during the scheduling process to eliminate policy conflicts, and generate the final optimization scheme. S5: Compile the final optimized scheme into executable dataflow accelerator hardware code and host control code for the target FPGA platform.

[0020] Furthermore, the inter-task collaboration in step S4 specifically includes: When applying parallelization transformations such as loop partitioning and unrolling to a task, detect the loop structure of its upstream and downstream tasks connected by FIFO. If the transformation causes the FIFO access patterns of the upstream and downstream tasks to no longer match, then attempt to perform a coordinated transformation on the upstream and downstream tasks to restore the match. If the cooperative transformation cannot be implemented due to resource or structural limitations, the affected FIFO channels will be downgraded to ping-pong buffers, thereby locally removing strict sequential access constraints and ensuring the correct execution of the overall data flow graph.

[0021] Third, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the automated generation method for a data stream accelerator as described in claim 9 or 10.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows: 1) Compared to previous technologies that could only partially eliminate violations, this technology fully eliminates various data flow violations carried by the input application, including both coarse-grained and fine-grained violations. This results in more thorough data flow optimization and a significant improvement in overall performance.

[0023] 2) Previous work often used only a single data transmission or execution method, such as FIFO, ping-pong buffer or serial execution, while this invention provides automatic intelligent decision-making and optimization of on-chip buffers, which can adopt a hybrid solution and greatly improve the efficiency of on-chip data transmission.

[0024] 3) Previous technologies lacked automatic off-chip high bandwidth memory (HBM) management. This invention provides efficient HBM management and improves off-chip data transfer efficiency.

[0025] 4) Previous automated scheduling strategies often only focus on performance, lacking a balance between performance and resources. The strategy proposed in this invention can fully consider the balance between performance and resources. It can be performance-oriented, consuming resources to obtain optimal performance, or resource-oriented, supporting normal model deployment even in resource-constrained scenarios.

[0026] 5) Compared with previous work, in the comprehensive evaluation, CODO achieved a speedup of 3.7 to 33.8 times on DNN models compared with advanced compilers; in the board-level test, CODO achieved an average speedup of 3.6 times on DNN and a speedup of 2.07 times on GPT-2 models. Attached Figure Description

[0027] Figure 1 This is the overall architecture and workflow diagram of the optimized compiler system for dataflow architecture of this invention; Figure 2 This is a schematic diagram of three typical modes and conversion strategies for coarse-grained data stream violation elimination in this invention; wherein, (a) is a single producer-multiple consumer mode; (b) is a multi-producer-single consumer mode; and (c) is a multi-producer-multiple consumer mode.

[0028] Figure 3 This is an example diagram of the reduction operation rewriting involved in the fine-grained data stream violation elimination in this invention; Figure 4This is a schematic diagram of the permutation mapping generation and cyclic exchange process involved in fine-grained data stream violation elimination in this invention; Figure 5 This is an example diagram of the generation and loop reconstruction of the efficient reuse buffers (line buffers and window buffers) in this invention; (a) shows a sliding window of a convolutional kernel on the input feature map; (b) shows the two generated buffer structures: a line buffer (depth = kh) for caching multiple lines of historical data; and a window buffer (size = kh × kw) for maintaining the complete convolutional window; (c) shows the reconstructed loop code structure after integrating FIFO and reuse buffers, where different attributes of the loop dimension are highlighted with different colors (red, orange, green), and compared with... Figure 5 The buffer update logic in (b) corresponds to this; Detailed Implementation The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0029] The compiler system described in this invention (hereinafter referred to as CODO) runs on a general-purpose computing platform, such as a server or workstation equipped with a multi-core CPU. The target hardware platform is an FPGA development board that supports High-Level Synthesis (HLS), such as a Xilinx Alveo series accelerator card equipped with High-Bandwidth Memory (HBM). The system is built on the MLIR (Multi-Level Intermediate Representation) compilation framework, utilizing its scalable dialect system and conversion infrastructure.

[0030] See Figure 1 The CODO compilation process is as follows: S1. Input: The user provides a computation kernel function written in C++ or a PyTorch model definition file; S2. Front-end conversion: Convert to the corresponding MLIR dialects using Polygeist and Torch-MLIR respectively. This step generates a unified, machine-analyzable intermediate representation (IR).

[0031] S3. Collaborative elimination of data flow violations: CODO calls the codo-opt tool to complete the entire process optimization in a single command. Users can also adjust input parameters such as the upper limit of parallelism and the block factor as needed.

[0032] The process begins with two data flow correction phases: a coarse-grained violation elimination phase resolves single producer-consumer constraint violations between tasks (i.e., nodes in the data flow graph); subsequently, a fine-grained violation elimination phase corrects inconsistencies between data access order and counting, laying the foundation for efficient FIFO-based communication. This latter phase embodies our collaborative optimization principle, which not only ensures correctness but also proactively adjusts the code structure to improve communication efficiency and provides guidance for subsequent communication optimization phases.

[0033] S4. Data Communication Optimization: CODO makes communication buffer decisions, choosing between First-In-First-Out (FIFO) and ping-pong buffers, prioritizing the higher-performance FIFO implementation. To further improve communication efficiency, CODO generates efficient reused buffers and re-invokes the correctness verification process to avoid introducing new violations. This process also reveals loop-level parallelism, providing crucial information for subsequent parallel strategy exploration. Afterward, CODO manages off-chip data transmission to improve HBM bandwidth utilization.

[0034] S5. Scheduling Optimization: CODO's automatic scheduling engine determines the partitioning factor, unrolling factor, pipeline strategy, and array partitioning scheme. These parallel decisions are not made in isolation; they can simultaneously affect correctness and communication efficiency. Therefore, through a global inter-task coordinating optimization, CODO coordinates these choices across the entire computation graph, eliminating any newly introduced violations, thereby ensuring the final high-performance design.

[0035] S6. The globally optimized final MLIR is converted through a back-end conversion channel to generate C / C++ code acceptable to the target FPGA platform (such as Xilinx Vitis HLS) or directly to generate an RTL description. Simultaneously, host-side (CPU) code for data handling and kernel control is automatically generated. The final output is a complete, synthesizable, and deployable accelerator design project.

[0036] The specific implementation process of S3-S5 is explained in detail below: S3. Collaborative Elimination of Data Flow Violations: Existing commercial High-Level Synthesis (HLS) tools have limitations in effectively handling data flow violations: they can only report coarse-grained violations through synthesis analysis and cannot automatically transform the code to eliminate the violations. Therefore, we systematically eliminate both coarse-grained and fine-grained data flow violations.

[0037] 3.1 Coarse-grained violation elimination Violation Issue: The input C / C++ code or PyTorch model is first converted into a dataflow graph, where nodes represent computational tasks (such as loops or functions). Current commercial HLS tools enforce that dataflow execution conforms to the single producer-single consumer pattern; therefore, effective techniques are needed to eliminate violations that deviate from this constraint. For example, Figure 2 In (a), Node1 writes the result to buffer 'a', while Node2 and Node3 read from the same buffer, forming a single producer-multiple consumer pattern. Similarly, Figure 2 (b) and (c) describe the multi-producer-single-consumer and multi-producer-multi-consumer patterns, respectively. Although existing work has partially addressed violations of type (a) or (c), it often fails to completely eliminate all violations, leading to a degradation of execution between nodes to sequential execution.

[0038] Pattern-Aware Code Transformation: To thoroughly address these issues, we propose a pattern-aware code transformation algorithm. This algorithm iterates through the input code, detecting data access patterns that may lead to violations, typically stemming from multiple nodes accessing the same buffer. Overall, all access patterns leading to coarse-grained data stream violations can be summarized as follows: Figure 2 The three main categories are shown. Once a violation is identified, CODO will detect its access pattern and apply the appropriate transformations to refactor the code. Figure 2 (a) is a typical bypass mode (common in models with residual structures such as ResNet-18 and GPT-2). CODO first traverses all buffers and collects the nodes that access them. Taking buffer a as an example, its related nodes are Node1–3. CODO analyzes and records the access behavior of each node to buffer a. Figure 2 (a) is then identified as a single producer-multiple consumer pattern. To eliminate this violation, the algorithm inserts an intermediate node (Node1') that reads from buffer a and writes to the copied buffers b and b'. Figure 2 (a)).

[0039] CODO for Figure 2 The three coarse-grained violation patterns shown apply different code transformation strategies: for Figure 2 The multi-producer-single-consumer pattern in (b) (common in combinations of initialization and padding operations) is resolved through node merging. CODO merges circular writes to the same buffer; to maintain correctness, intermediate results from earlier writes are temporarily stored and eventually merged into the last write operation. Figure 2 In the multi-producer-multi-consumer problem in (c), we create a new buffer2 by copying buffer1, ensuring that each buffer is accessed by only one pair of read and write operations.

[0040] 3.2 Fine-grained violation elimination After coarse-grained violation elimination, the HLS tool can allocate ping-pong buffers between nodes by default to support coarse-grained data stream execution based on data blocks. However, for nodes where input and output data can be transmitted sequentially via FIFO and processed with finer granularity, performance may not be maximized. This is because FIFO-based data streams typically offer better performance due to their streaming computation model and lower resource overhead, but they impose strict requirements on the code pattern, thus necessitating fine-grained violation elimination.

[0041] Violation Issues: FIFOs can only be inserted at the connections between all nodes (or loops) if the sequential data access constraints are satisfied and the data access order between adjacent nodes (or loops within a node) matches the access count. Unfortunately, in practical applications, numerous fine-grained read-write inconsistencies exist. As mentioned earlier, violations such as mismatched access counts and inconsistent access orders can lead to deadlocks or calculation errors in the final design. More seriously, existing HLS tools cannot detect these violations during synthesis. Although some problems can be discovered through co-simulation, this process can take days or even weeks, significantly increasing the debugging burden on developers.

[0042] Systematic read / write coordination: To eliminate fine-grained violations and further improve design efficiency, a systematic read-write coordination method is proposed, including: 1) Reduction operation rewriting: solving the data access count mismatch problem while ensuring early writing of FIFO; 2) Permutation map generation: adjusting the access mode of adjacent loops to ensure that their data access order is consistent, thereby solving the problem of inconsistent data access order.

[0043] 1) Reduction Operation Rewriting: Most data access count mismatches stem from reduction operations, such as fully connected layers, max pooling, and normalization. These non-bottleneck operations introduce a circular dimension that doesn't directly correspond to array indices, leading to redundant FIFO accesses during reduction iterations. To address this, we propose a reduction rewriting strategy: identifying reduction regions and aggregating intermediate results using temporary arrays, thereby minimizing unnecessary FIFO transfers and ensuring correct and efficient data flow execution. Figure 3As shown, the producer (max pooling operation) writes to the buffer `out`, while the consumer (initialization operation) reads from the same buffer. The discrepancy between the number of writes and reads leads to a mismatch in data access counts, resulting in a FIFO deadlock. To solve this problem, we categorize the loop dimension corresponding to the FIFO array index as the index dimension, and identify the rest as reduction dimensions and move them to the innermost loop (as shown in the shaded area in the figure). Subsequently, the write operation to `out` is moved out of the reduction region, and a temporary buffer is introduced to accumulate intermediate results. This transformation ensures that the producer's access count matches the consumer's, while guaranteeing that the computation and transmission of intermediate results are performed on demand and in a timely manner, significantly improving data transmission efficiency.

[0044] 2) Permutation Map Generation: Inconsistent data access order is common in practical applications, leading to data flow violations in FIFO-based streaming processing. To address this issue, we propose a permutation map generation method. Specifically, CODO identifies bottleneck loops (such as Q*K in convolution or attention) as reference loops by analyzing the iteration count and computational intensity of each nested loop, and then analyzes their data access patterns (including the access order of input and output arrays). This information serves as the basis for adjusting the data access patterns of its producer and consumer loops (called target loops). CODO employs a mapping-based strategy to efficiently align the data access patterns of the reference and target loops. Figure 4 As shown, the process consists of four steps: Step 1: Establish a mapping relationship between the dimensions of the connection array and the corresponding loop depths for the reference loop and the target loop, respectively.

[0045] Step 2: Apply a loop block of size 1 to the reference loop to align the depths of the reference loop and the target loop, splitting h and w into two loops respectively.

[0046] Step 3: Construct a mapping relationship between the loop depth sets of the reference loop and the target loop (for example, a mapping from 2 to 1 means that the loop with depth 2 in the target loop should be swapped to depth 1).

[0047] Step 4: Based on the depth map generated in Step 3, transform the target loop by adjusting the nesting order.

[0048] S4. Implementation of High-Efficiency Data Communication After eliminating data flow violations, the input algorithm has been transformed into a form that conforms to data flow constraints. Based on this, optimizing on-chip and off-chip data communication is crucial for overall efficiency. To this end, we propose two on-chip optimizations: 1) communication buffer decision-making, prioritizing FIFO implementation for tasks without data flow violations; 2) violation-free multiplexed buffer generation, improving data transmission efficiency while ensuring violation-free design; and one off-chip optimization: 3) off-chip data transmission management, used to improve HBM bandwidth utilization.

[0049] 4.1 On-chip communication buffer decision We employ a FIFO-first strategy to optimize the on-chip communication buffer. For tasks without violations, we prioritize FIFO implementation to maximize performance. If fine-grained violations between loops cannot be eliminated, we opt for a ping-pong buffer implementation. It's important to note that ping-pong buffers have higher resource overhead, with a capacity at least twice that of the transmitted data block, potentially leading to resource overflow risks in large-scale applications.

[0050] Multiplexed buffer generation: Utilizing reuse buffers can effectively improve data reuse rates and memory bandwidth, but integrating them into designs requires careful loop refactoring. Existing HLS frameworks typically rely on user-defined domain-specific languages, requiring developers to manually specify buffer sizes, communication buffer types, and address mappings for reused data. This approach not only demands specialized knowledge from developers but also often leads to suboptimal or infeasible designs, as discussed in the introduction. To address this, we propose a violation-free reuse buffer generation method to automatically discover data reuse opportunities.

[0051] Violation-free multiplexed buffer generation: To automatically generate efficient multiplexed buffers, CODO first analyzes nested loop structures to identify computationally intensive tasks (such as convolution and matrix multiplication). Then, it extracts input / output access patterns to detect FIFO and reduction dimensions, thereby guiding subsequent buffer generation. Figure 5 Taking convolution as an example, each output pixel depends on a small local region of the input feature map, and a large number of input elements are reused between adjacent outputs. These reuse opportunities are aligned with the reduction dimension. CODO automatically analyzes the FIFO index, identifies reduction dimensions that are not related to FIFO, and constructs a reuse buffer accordingly.

[0052] Specifically, CODO constructs row buffers and window buffers based on the iteration domain of the reduction loop (see...). Figure 5(a) and (b)). The row buffer (denoted as lb[n][ci][kh][w]) stores multiple rows of input feature map data, with a depth equal to the convolution kernel height (kh), and retains kh-1 rows of historical data for subsequent computation. Each new input element (input[n][ci][h][w]) is written to the latest position. The window buffer (denoted as wb[n][ci][kh][kw]) maintains a complete kh×kw convolution kernel window. For each new column w in the input, it is updated by horizontally shifting existing content and loading the new column from the row buffer.

[0053] To prevent data flow violations, CODO refactors loops while analyzing FIFO access patterns, ensuring all loop dimensions are used correctly. Specifically, loops involving FIFO access must neither include irrelevant dimensions nor omit necessary ones. For example, in... Figure 5 In (c), the input and output arrays are optimized to FIFO, and the nested loops surrounding them are precisely aligned with the array indices, thus ensuring consistent data access.

[0054] Providing guidance for parallelization exploration: After the reused buffer is generated, the rewritten code is ready for further optimization through loop blocking, pipelining, unrolling, and array partitioning. However, as shown in the figure, the generated loop structure is very complex, making optimization quite challenging. By analyzing the internal computational behavior, we identify different parallelization opportunities: Parallelizing the outermost red loop is unsafe because it expands all three inner regions, introducing complex data dependencies and control issues. The middle orange loops are related to the FIFO index; optimizing them might alter the FIFO access pattern, potentially leading to new violations. The innermost green loop is unrelated to FIFO behavior and can therefore be safely parallelized without introducing new violations. This analysis is crucial for subsequent parallelization exploration because it effectively prunes the large design space associated with large-scale models.

[0055] 4.2 External Data Transmission Management To improve off-chip bandwidth utilization, CODO automatically constructs efficient burst transfers between HBM and on-chip memory. It distributes parameters such as model weights across different HBM channels, enabling parallel access to independent memory regions. CODO provides the `codo-transmit` command, which automatically generates the kernel's host code and burst access operations; users can specify the number of HBM channels allocated.

[0056] S5. Implementation of Optimized Data Flow Scheduling After applying the optimizations described in the preceding stages, data flow violations have been eliminated, and buffers have been inserted into their appropriate positions. Subsequently, CODO performs automatic scheduling to develop parallelism without exceeding the resource budget and coordinates adjacent tasks to avoid introducing new data flow violations.

[0057] Challenges: Parallelization exploration in dataflow accelerators faces significant challenges: 1) Inappropriate parallelization strategies can disrupt the latency balance between tasks, leading to a decline in dataflow performance; 2) Existing methods prioritize performance improvement while neglecting the trade-off between resources and performance, potentially resulting in excessive resource consumption; 3) Some parallelization strategies may alter the FIFO access pattern, thereby introducing new dataflow violations.

[0058] Parallelization Exploration: To address these challenges, we propose a resource-aware, bottleneck-centric design space exploration approach to find optimal parallelization strategies, including loop partitioning, pipelines, unrolling, and array partitioning. The figure illustrates the performance and resource tuning at each stage of the exploration process.

[0059] Phase 1: Initial Parallelization Allocation DSE first constructs a high-quality initial design with initial parallelism. CODO uses a performance profiling-based model to estimate the latency of each loop and allocates parallelism proportionally. Specifically, it estimates the latency of each loop based on the number of iterations and the parallelism strategy, then allocates parallelism proportionally to the latency, setting a minimum parallelism of 1. Subsequently, while maintaining the proportionality, the parallelism of all loops is gradually increased until a user-specified upper limit or hardware resource constraint is reached. This process helps to form a roughly balanced dataflow structure. Unlike methods that randomly parallelize loop dimensions, CODO leverages insights from the early communication optimization phase. It prioritizes the blocking of loop dimensions unrelated to FIFO access to ensure correct and efficient communication, and automatically applies HLS compiler directives such as pipelining, unrolling, and array partitioning to generate the initial design.

[0060] Phase Two: Upward Expansion We then iterate through all bottleneck loops and re-estimate their latency. If the latency of a loop is still at least a fraction of the latency of the lowest-latency loop... If the parallelism is increased by a factor of 1, then its parallelism will be increased to 1. .

[0061] This process is iterative and terminates when the parallelism of all loops stabilizes or the iteration limit is reached. This step aims to maximize the parallelism of each bottleneck loop, but this may increase resource usage.

[0062] Phase 3: Downward Adjustment Since the overall performance of a FIFO-based data stream depends on the longest-latency loop, we introduce a down-adjustment step to adjust for over-optimized loops. If a loop is faster than the longest loop... If the parallelism is multiplied by a factor of 1, it indicates that the loop is over-optimized. In this case, we reduce the parallelism of the loop. This doubles the performance, thereby reducing resource usage while maintaining a balanced data flow. This step may slightly degrade overall performance, but users can configure CODO to enable or disable it based on design goals.

[0063] Inter-task collaborative optimization: After parallel exploration, the bottleneck loop has been effectively optimized. However, if loop partitioning is applied to the dimensions involved in the FIFO, the loops connected to the other end of the FIFO need to adopt the same parallel strategy to maintain consistency in data access behavior. These changes often reshape the loop structure, so we re-invoke the correctness verification process to detect and resolve any newly introduced data flow violations.

[0064] However, conflicts may occur at this stage. For example, consider a data stream consisting of loops A, B, C, and D connected via a FIFO. If loops B and D employ a conflicting parallel strategy, loop C may encounter an unresolved violation. In this case, CODO will downgrade the buffer between loops C and D to a ping-pong buffer implementation, thus preserving the FIFO-based execution from loop A to loop C.

[0065] Through the specific implementations described above, the CODO compiler can automatically convert user-input algorithms into high-performance dataflow accelerators. In implementation examples, CODO achieved a performance improvement of 3.7 to 33.8 times compared to existing advanced HLS compilation schemes on multiple DNN models in comprehensive evaluations, achieving an average speedup of 3.6 times on real FPGA boards, and a speedup of 2.07 times on the GPT-2 model. This verifies the effectiveness and advancement of the technical solution described in this invention in resolving dataflow violations, optimizing communication, and automating scheduling.

[0066] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, or improvements made by those skilled in the art without departing from the principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. An optimized compiler system for a dataflow architecture, characterized in that, include: The front-end conversion module receives computational tasks written in C++ or defined in PyTorch, converts them into a unified intermediate representation, and outputs it to the data flow violation elimination module. The unified intermediate representation preserves the data dependencies and computational semantics of the original algorithm. The data flow violation collaborative elimination module receives the unified intermediate representation, systematically analyzes and eliminates data flow violations therein, and outputs an intermediate representation that conforms to the single producer-single consumer constraint and has consistent fine-grained access to the communication optimization module; at the same time, it feeds back the identified data access pattern and loop structure information to the scheduling optimization module. The communication optimization module receives the intermediate representation after violation elimination, performs on-chip buffer decision-making and automatic generation, manages off-chip data communication, and outputs the intermediate representation with embedded efficient communication structure to the scheduling optimization module; at the same time, it transmits the generated multiplexed buffer structure and its associated loop dimension information to the scheduling optimization module to guide safe parallelization. The scheduling optimization module receives access mode information from the data flow violation elimination module, buffer and dimension information from the communication optimization module, and parallelism constraints from user input. It performs resource-aware automatic parallel scheduling and inter-task collaborative optimization, and outputs a final intermediate representation with a determined parallel strategy to the code generation module. The code generation module receives the globally optimized final intermediate representation and generates hardware description code for the data flow accelerator that can be implemented on the FPGA, as well as the corresponding host-side driver code.

2. The optimized compiler system for dataflow architecture according to claim 1, characterized in that, The data stream violation elimination module includes a coarse-grained violation elimination unit and a fine-grained violation elimination unit cascaded together with signals; The coarse-grained violation elimination unit receives the intermediate representation, identifies and reconstructs the code to eliminate violations between three types of nodes: single producer-multiple consumers, multiple producers-single consumer, and multiple producers-multiple consumers, and outputs an intermediate representation that conforms to the single producer-single consumer constraint to the fine-grained violation elimination unit. The fine-grained violation elimination unit receives the intermediate representation that conforms to the single producer-single consumer constraint, resolves the access count mismatch by rewriting through a reduction operation, resolves the access order inconsistency by generating through a permutation mapping, and outputs the intermediate representation that supports FIFO-based fine-grained pipelined execution to the communication optimization module.

3. The optimized compiler system for dataflow architecture according to claim 2, characterized in that, The coarse-grained violation elimination unit performs mode-aware code transformation, specifically including: When a single producer-multiple consumer violation pattern is detected, an intermediate replication node is inserted in the data flow graph, and the original output buffer is replicated into multiple copies, so that each replication buffer serves only one consumer. When a multi-producer-single-consumer violation pattern is detected, multiple production tasks writing to the same buffer will be merged in a loop and the write operations will be combined through an internal temporary storage mechanism. When a multi-producer-multi-consumer violation pattern is detected, the violation pattern is decoupled into multiple independent single-producer-single-consumer pairs by copying the buffer and reallocating the read-write relationship.

4. The optimized compiler system for dataflow architecture according to claim 2, characterized in that, In the fine-grained violation elimination unit, the reduction operation rewriting includes: Identify the cyclic regions involving reduction operations in the intermediate representation; Separate the index dimension and reduction dimension of the loop, and move the reduction dimension to the innermost layer of the loop; By introducing a temporary accumulation buffer, the multiple write operations to the output FIFO that were originally performed within the reduction loop are transformed into accumulation operations within the temporary buffer. After the reduction is completed, the final result is written to the output FIFO all at once, thus ensuring that the number of FIFO accesses by the producer and consumer are strictly matched.

5. The optimized compiler system for dataflow architecture according to claim 1, characterized in that, In the fine-grained violation elimination unit, the permutation map generation includes: Taking computationally intensive loops as a reference, we analyze their data access order; The goal is to identify the producer or consumer cycle and analyze the current data access order. Construct a mapping relationship from the reference loop iteration space to the target loop iteration space; Based on the mapping relationship, the nested structure of the target loop is transformed by loop exchange, block division, or reordering to align its data output or consumption order with the requirements of the reference loop.

6. The optimized compiler system for dataflow architecture according to claim 1, characterized in that, The communication optimization module includes an on-chip optimization unit for signal interaction and an off-chip management unit; wherein... The on-chip optimization unit receives the intermediate representation with eliminated violations, selects a buffer for the inter-task channel using a FIFO priority strategy, automatically analyzes the data reuse mode to generate a row buffer and a window buffer, and outputs the optimized intermediate representation to the scheduling optimization module. The off-chip management unit works in conjunction with the on-chip optimization unit to analyze the access patterns of model parameters, automatically segment and map them to multiple high-bandwidth memory channels, and generate host-side memory copy instructions to achieve burst transmission.

7. The optimized compiler system for dataflow architecture according to claim 6, characterized in that, In the on-chip optimization unit, the automatically generated multiplexed buffer includes: Analyze the array access patterns in nested loops of computation tasks to identify reduction dimensions where data reuse exists; Based on the size of the iterative domain of the reduction dimension, the depth and width of the reuse buffer are automatically determined; Insert the corresponding buffer declarations and data maintenance logic into the code, and refactor the surrounding loops to ensure that the timing of data filling, sliding, and reading is correct and does not violate existing data flow constraints.

8. The optimized compiler system for dataflow architecture according to claim 1, characterized in that, The scheduling optimization module includes a cascaded initial allocation unit, an iterative optimization unit, and a conflict coordination unit. The initial allocation unit allocates appropriate initial parallelism parameters to each loop based on the dimensional security information and performance estimation model from the communication optimization module. The iterative optimization unit receives the initial allocation result and, with the goal of improving overall throughput, iteratively adjusts the parallelism, block factor, and pipeline depth of each loop, and sends the new loop structure information to the conflict coordination unit after each adjustment. The conflict coordination unit continuously monitors the risk of data flow violations between tasks that may be caused by the adjustment of the parallel strategy. When a conflict that cannot be resolved by adjusting the loop is detected, a degradation instruction is sent to the communication optimization module to request that the buffer implementation of a specific channel be changed from FIFO to ping-pong buffer to ensure global correctness, and the final coordinated strategy is output to the code generation module.

9. A method for automatically generating a data stream accelerator based on the system according to any one of claims 1-8, characterized in that, Includes the following steps: S1: Receives algorithms described in high-level languages ​​or machine learning frameworks, and generates a unified intermediate representation through front-end conversion; S2: Perform data flow violation elimination on the intermediate representation, sequentially perform coarse-grained violation transformation and fine-grained read / write coordination, and generate an intermediate representation that conforms to the data flow execution constraints; S3: Perform communication optimization on the intermediate representation that meets the constraints, including prioritizing the use of FIFO to implement on-chip communication, automatically generating data multiplexing buffers, and optimizing off-chip HBM data transmission; S4: Based on resource constraints, performance model and optimization structure information from step S3, considering the balance between resources and performance, execute automated parallel policy scheduling, and perform inter-task coordination during the scheduling process to eliminate policy conflicts, and generate the final optimization scheme. S5: Compile the final optimized scheme into executable dataflow accelerator hardware code and host control code for the target FPGA platform.

10. The method for automatically generating a data stream accelerator according to claim 9, characterized in that, The inter-task collaboration in step S4 specifically involves: When applying parallelization transformations such as loop partitioning and unrolling to a task, detect the loop structure of its upstream and downstream tasks connected by FIFO. If the transformation causes the FIFO access patterns of the upstream and downstream tasks to no longer match, then attempt to perform a coordinated transformation on the upstream and downstream tasks to restore the match. If the cooperative transformation cannot be implemented due to resource or structural limitations, the affected FIFO channels will be downgraded to ping-pong buffers, thereby locally removing strict sequential access constraints and ensuring the correct execution of the overall data flow graph.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the automated generation method for the data stream accelerator as described in claim 9 or 10.

Images