Dataflow-based field programmable gate array inference acceleration method and related apparatus

Abstract

The application belongs to an edge intelligent computing optimization acceleration method, through multiple traversal of a computation graph, difference storage planning of fixed data and dynamic data is completed, and non-core operators are packaged and combined into an arithmetic logic unit module processing block to reduce data transfer and scheduling overhead. The division and cooperation of the arithmetic logic unit module and the general matrix multiplication module in the FPGA accelerator are used to process non-core operators and core operators respectively, and the smoothness of the calculation process is ensured through the interaction of the working states between the modules. The application optimizes the whole link from data storage, operator reconstruction to hardware execution, solves the problems of frequent data transfer and low hardware resource utilization in traditional schemes, and fully develops the reconfigurable characteristics and parallel computing advantages of FPGA through special module division.

Description

Technical Field

[0001] This application relates to an edge intelligent computing optimization and acceleration method, specifically a data flow-based field-programmable gate array inference acceleration method and related apparatus. Background Technology

[0002] Edge intelligent computing scenarios, such as power equipment operation and maintenance, place high demands on the energy efficiency of computing devices, requiring a balance between hardware efficiency and software flexibility. Traditional computing architectures suffer from a mismatch between data movement and computation speed; data transfer in edge scenarios is time-consuming and energy-intensive, impacting overall performance. Current solutions mainly include reconfigurable computing and accelerator architecture design. A typical reconfigurable computing acceleration method based on TVM / VTA (Tensor Virtual Machine / Versatile Tensor Accelerator) employs a von Neumann architecture. Since the acceleration core only supports a few computational operations such as dense processing, the remaining computational operations of the fault warning model need to be performed on the CPU. This results in intermediate result data needing to be moved back and forth between DRAM (Dynamic Random Access Memory) and BRAM (Block Random Access Memory). Under different network structures, the larger the intermediate result data, the longer the data transmission time and the greater the dynamic energy consumption. In addition, for high-end FPGA (Field-Programmable Gate Array) chips, the utilization rate of BRAM resources is insufficient, and the chip's capabilities cannot be fully utilized. Summary of the Invention

[0003] This application addresses the technical problems of high dynamic energy consumption and insufficient resource utilization in the traditional computing architecture used in current edge intelligent computing scenarios in the power field. It provides a data flow-based field-programmable gate array inference acceleration method and related devices.

[0004] To achieve the above objectives, this application adopts the following technical solution: In a first aspect, this application proposes a dataflow-based method for accelerating inference using a field-programmable gate array (FPGA), comprising: The computation graph of the edge intelligent computing to be processed is traversed multiple times, and the following operations are performed: fixed data in the computation graph is stored in the read-only memory of the field-programmable gate array (FPGA), and storage space is reserved in the block random access memory of the FPGA for dynamic data, which is used to store dynamic data during the computation process when the edge intelligent computing to be processed is performed; non-core operators in the computation graph are packaged into arithmetic logic unit (ALU) processing blocks, and connected subgraphs composed of multiple ALU processing blocks are selected. The ALU processing blocks in a connected subgraph are merged into a total ALU processing block, which is used to batch process non-core operators when the edge intelligent computing to be processed is performed. The edge intelligent computing to be processed is completed by an accelerator with a field-programmable gate array (FPGA). The accelerator includes an arithmetic logic unit (ALU) module and a general matrix multiplication module. The ALU module is used to process non-core operators in the computation graph of the edge intelligent computing to be processed, and the general matrix multiplication module is used to process core operators in the computation graph of the edge intelligent computing to be processed. The ALU module and the general matrix multiplication module interact with each other to obtain the processing progress of core operators and non-core operators.

[0005] Furthermore, the method for exchanging working states between the arithmetic logic unit module and the general matrix multiplication module includes: After the arithmetic logic unit module completes the processing of non-core operators, it sends a message signal to the general matrix multiplication module, including the current working status of the arithmetic logic unit module, the core operators that need to be processed by the general matrix multiplication module, and the storage address of the operands required by the general matrix multiplication module to process the core operators in the on-chip data space of the field programmable gate array. After the general matrix multiplication module completes the core operator processing, it queries the preset state transition table based on the working state of the arithmetic logic unit module received from the arithmetic logic unit module to obtain the working state of the arithmetic logic unit module for the next round of calculation, and sends a message signal including the working state of the arithmetic logic unit module for the next round of calculation to the arithmetic logic unit module.

[0006] Furthermore, the method for reserving storage space for dynamic data in the block random access memory of a field-programmable gate array includes: Based on the data type and data size of the output tensor of each operation in the edge intelligent computing to be processed, the storage space of the largest output tensor under each data type is reserved in the block random access memory of the field programmable gate array. If the output tensor is used as the input of at least two operators simultaneously, an additional (N-1)*Size(Ti) storage space is reserved; where N is the number of operators whose output tensor is used as the input simultaneously, and Ti is the data size of the output tensor.

[0007] Furthermore, after packaging the non-core operators in the computation graph into arithmetic logic unit processing blocks, the method further includes: If the output of an arithmetic logic unit (ALU) processing block is the input of at least two ALU processing blocks, then (N-1) additional operations to write the output to the block random access memory cache are added after the output ALU processing block.

[0008] Furthermore, the method for merging arithmetic logic unit processing blocks in a connected subgraph into a single total arithmetic logic unit processing block includes: According to the traversal order, multiple arithmetic logic unit (ALU) processing blocks are merged into a single ALU processing block and renumbered. During the merging process, for ALU processing blocks with multiple inputs, if they have already been traversed, they are directly read from the block random access memory cache as the input for the subsequent ALU processing block.

[0009] Furthermore, after repeatedly traversing the computation graph of the edge intelligent computing to be processed and performing the following operations, the process further includes: Based on the definition of each operator, the processing logic corresponding to each arithmetic logic unit processing block is converted into hardware description language code supported by the field-programmable gate array.

[0010] Secondly, this application proposes a dataflow-based field-programmable gate array (FPGA) inference acceleration system, comprising: The rules module is used to traverse the computation graph of the edge intelligent computation to be processed multiple times and perform the following operations: store the fixed data in the computation graph into the read-only memory of the field-programmable gate array (FPGA), and reserve storage space for dynamic data in the block random access memory of the FPGA, which is used to store dynamic data during the computation process when the edge intelligent computation to be processed is performed; package the non-core operators in the computation graph into arithmetic logic unit (ALU) processing blocks, filter out the connected subgraphs composed of multiple ALU processing blocks, and merge the ALU processing blocks in a connected subgraph into a total ALU processing block, which is used to batch process the non-core operators when the edge intelligent computation to be processed is performed; The computing module is used to complete the edge intelligent computing to be processed through the accelerator of the field-programmable gate array. The accelerator includes an arithmetic logic unit module and a general matrix multiplication module. The arithmetic logic unit module is used to process non-core operators in the computation graph of the edge intelligent computing to be processed, and the general matrix multiplication module is used to process core operators in the computation graph of the edge intelligent computing to be processed. The arithmetic logic unit module and the general matrix multiplication module interact to obtain the processing progress of core operators and non-core operators.

[0011] Furthermore, the method for exchanging working states between the arithmetic logic unit module and the general matrix multiplication module includes: After the arithmetic logic unit module completes the processing of non-core operators, it sends a message signal to the general matrix multiplication module, including the current working status of the arithmetic logic unit module, the core operators that need to be processed by the general matrix multiplication module, and the storage address of the operands required by the general matrix multiplication module to process the core operators in the on-chip data space of the field programmable gate array. After the general matrix multiplication module completes the core operator processing, it queries the preset state transition table based on the working state of the arithmetic logic unit module received from the arithmetic logic unit module to obtain the working state of the arithmetic logic unit module for the next round of calculation, and sends a message signal including the working state of the arithmetic logic unit module for the next round of calculation to the arithmetic logic unit module.

[0012] Furthermore, the method for reserving storage space for dynamic data in the block random access memory of a field-programmable gate array includes: Based on the data type and data size of the output tensor of each operation in the edge intelligent computing to be processed, the storage space of the largest output tensor under each data type is reserved in the block random access memory of the field programmable gate array. If the output tensor is used as the input of at least two operators simultaneously, an additional (N-1)*Size(Ti) storage space is reserved; where N is the number of operators whose output tensor is used as the input simultaneously, and Ti is the data size of the output tensor.

[0013] Furthermore, after packaging the non-core operators in the computation graph into arithmetic logic unit processing blocks, the method further includes: If the output of an arithmetic logic unit (ALU) processing block is the input of at least two ALU processing blocks, then (N-1) additional operations to write the output to the block random access memory cache are added after the output ALU processing block.

[0014] Furthermore, the method for merging arithmetic logic unit processing blocks in a connected subgraph into a single total arithmetic logic unit processing block includes: According to the traversal order, multiple arithmetic logic unit (ALU) processing blocks are merged into a single ALU processing block and renumbered. During the merging process, for ALU processing blocks with multiple inputs, if they have already been traversed, they are directly read from the block random access memory cache as the input for the subsequent ALU processing block.

[0015] Furthermore, after repeatedly traversing the computation graph of the edge intelligent computing to be processed and performing the following operations, the process further includes: Based on the definition of each operator, the processing logic corresponding to each arithmetic logic unit processing block is converted into hardware description language code supported by the field-programmable gate array.

[0016] Thirdly, this application proposes an electronic device, including: a memory and one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the steps of the above-described data flow-based field-programmable gate array inference acceleration method.

[0017] Fourthly, this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described data flow-based field-programmable gate array inference acceleration method.

[0018] Compared with the prior art, this application has the following beneficial effects: This application proposes a dataflow-based inference acceleration method for Field-Programmable Gate Arrays (FPGAs). By traversing the computation graph multiple times, it achieves differentiated storage planning for fixed and dynamic data. Simultaneously, non-core operators are packaged and merged into arithmetic logic unit (ALU) module processing blocks, reducing data movement and scheduling overhead. Utilizing the collaborative division of labor between the ALU and general matrix multiplication modules in the FPGA accelerator, non-core and core operators are processed separately, and the smoothness of the computation flow is ensured through inter-module interaction. This application optimizes the entire chain from data storage and operator reconfiguration to hardware execution, solving the problems of frequent data movement and low hardware resource utilization in traditional solutions. Furthermore, it fully leverages the reconfigurability and parallel computing advantages of FPGAs through dedicated module division of labor, forming a highly efficient inference acceleration solution adapted to edge intelligent computing scenarios. Based on dataflow execution, it eliminates the need for instruction fetching and decoding operations, resulting in higher efficiency.

[0019] This application also proposes a dataflow-based field-programmable gate array (FPGA) inference acceleration system, an electronic device, and a computer-readable storage medium, which possess all the advantages of the aforementioned FPGA inference acceleration methods. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the first process of the data flow-based field-programmable gate array inference acceleration method of this application; Figure 2This is a schematic diagram of the second process of the data flow-based field-programmable gate array inference acceleration method of this application; Figure 3 This is a schematic diagram of the field-programmable gate array (FPGA) accelerator architecture in an embodiment of this application; Figure 4 This is a calculation diagram of the equipment fault early warning model in the embodiments of this application; Figure 5 This is a schematic diagram of a state transition table in an embodiment of this application; Figure 6 This is a schematic diagram of the dataflow-based field-programmable gate array inference acceleration system of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0023] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0024] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0025] In the description of the embodiments of this application, it should be noted that if terms such as "upper," "lower," "horizontal," or "inner" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use, they are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this application. In addition, terms such as "first" and "second" are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0026] Furthermore, the use of the term "horizontal" does not imply that the component must be absolutely horizontal, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0027] In the description of the embodiments of this application, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0028] With the rapid development of smart grids, industrial IoT, and other fields, power equipment operation and maintenance is gradually becoming more intelligent and refined, and edge intelligent computing is being applied more and more widely in this field. Edge intelligent computing refers to a computing model that performs data processing, analysis, and intelligent decision-making at edge nodes close to the equipment terminal or data generation source. Its core advantages lie in reducing data transmission latency, reducing core network bandwidth consumption, and improving the real-time performance and security of data processing. This is crucial for key operation and maintenance tasks such as power equipment fault early warning and status monitoring. In power equipment operation and maintenance scenarios, edge computing devices need to process massive amounts of equipment operation data, including multi-dimensional monitoring data such as current, voltage, and temperature, and quickly execute computational tasks such as fault diagnosis algorithms and early warning model inference. These scenarios place extremely high demands on the energy efficiency ratio of computing devices, requiring them to ensure rapid response and processing accuracy of computational tasks while minimizing energy consumption to adapt to the actual application situation where edge nodes are often limited by limited power supply resources.

[0029] In edge intelligent computing practices for power equipment operation and maintenance, existing computing devices face a significant performance-energy consumption trade-off. When processing complex fault early warning models, a large amount of intermediate result data needs to be frequently moved between different storage units. This data movement process itself does not generate effective computational value but consumes a significant amount of time and energy. Meanwhile, while high-end Field-Programmable Gate Array (FPGA) chips commonly used in edge computing possess powerful parallel computing capabilities and hardware programmability, their built-in Block Random Access Memory (BRAM) resources are not fully utilized. This results in the chip's hardware potential not being fully realized, leading to a waste of hardware resources and hindering the improvement of overall computing efficiency. Consequently, it is difficult to meet the core requirements of high energy efficiency and high real-time performance for computing devices in power equipment operation and maintenance scenarios.

[0030] To address energy efficiency and performance challenges in edge intelligent computing scenarios, two mainstream technical solutions have emerged in the industry. The first is reconfigurable computing technology, a computing model that combines hardware efficiency with software flexibility. Its core lies in using programmable logic devices to dynamically reconfigure the hardware circuit structure according to the specific computing task requirements, thereby achieving hardware acceleration. Compared to traditional fixed-function hardware and general-purpose processors, reconfigurable computing can maintain near-dedicated hardware performance while providing software-like flexibility, adjusting the hardware architecture according to the computing needs of different fault warning models, thus improving computing efficiency. The second type involves optimizing accelerator architecture design, targeting traditional von Neumann architectures. The existence of the von Neumann architecture The Neumann bottleneck problem has led to the industry proposing dataflow architecture as an alternative. Traditional von Neumann architecture... The von Neumann architecture is an instruction-driven model, requiring the CPU or accelerator to read instructions and data from memory, execute the instructions, and then write the results back to memory. Data movement lags far behind computation speed, resulting in significant time and energy consumption on data transfer. In contrast, the dataflow architecture employs a data-driven computation model. Its core idea is that an operation is executed immediately once all the necessary data is ready, without waiting for instruction scheduling. This reduces data transfer latency and energy consumption, improving computational parallelism and efficiency. Furthermore, reconfigurable computation acceleration methods based on Tensor Virtual Machines (TVM) and Vector Tensor Accelerators (VTA) are increasingly being applied. TVM is an open-source deep learning compilation framework that can convert models defined by different deep learning frameworks into code that can be efficiently executed on various hardware. VTA is a programmable accelerator architecture based on FPGAs. The combination of these two approaches aims to optimize the execution efficiency of edge computing through reconfigurability.

[0031] While existing technologies attempt to address the energy efficiency issues of edge intelligent computing, significant shortcomings remain, failing to fully meet the practical needs of power equipment operation and maintenance scenarios. Among these, reconfigurable computing acceleration methods based on TVM / VTA, although leveraging the flexibility of reconfigurable computing, essentially still employ von Neumann architectures. The Neumann architecture, therefore, cannot fundamentally solve the von Neumann problem. Problems caused by the Neumann bottleneck.

[0032] Based on the above, this application proposes a dataflow-based field-programmable gate array inference acceleration method and related apparatus. The following describes this application in further detail with reference to embodiments and accompanying drawings.

[0033] like Figure 1The diagram shown is a first flowchart of the dataflow-based field-programmable gate array inference acceleration method of this application, which may include: S101, traverse the computation graph of the edge intelligent computing to be processed multiple times and perform the following operations: store the fixed data in the computation graph into the read-only memory of the field-programmable gate array (FPGA), and reserve storage space for dynamic data in the block random access memory of the FPGA, which is used to store dynamic data during the computation process when performing the edge intelligent computing to be processed; package the non-core operators in the computation graph into arithmetic logic unit (ALU) processing blocks, and filter out the connected subgraphs composed of multiple ALU processing blocks, and merge the ALU processing blocks in a connected subgraph into a total ALU processing block, which is used to batch process the non-core operators when performing the edge intelligent computing to be processed.

[0034] This application reduces data movement and hardware scheduling overhead by pre-planning data storage locations and integrating non-core operators. Storing fixed data in ROM (Read-Only Memory) avoids redundant loading, while reserving BRAM space for dynamic data ensures high-speed access. Operator packaging and merging connected subgraphs reduce the scheduling complexity between operators. In practical applications, a graph traversal algorithm can be used to traverse the computation graph and identify fixed and dynamic data. Non-core operators are categorized and packaged based on operator type identification. A graph connectivity analysis algorithm is used to filter connected subgraphs, and a hardware resource adaptation algorithm is used to merge ALU (Arithmetic and Logic Unit) processing blocks in the connected subgraphs into a total operator block. Storing fixed data in ROM reduces redundant data transfer between DDR and FPGA, and reserving dynamic data space in BRAM shortens data access latency. Packing non-core operators and merging connected subgraphs reduces operator scheduling times, lowers hardware control overhead, and lays the foundation for subsequent parallel processing.

[0035] S102 performs edge intelligent computing on the target edge through the accelerator of a field-programmable gate array.

[0036] The accelerator includes an arithmetic logic unit module and a general matrix multiplication module. The arithmetic logic unit module is used to process non-core operators in the computation graph of the edge intelligent computing to be processed, while the general matrix multiplication module is used to process core operators in the computation graph of the edge intelligent computing to be processed. The arithmetic logic unit module and the general matrix multiplication module interact to obtain the processing progress of core and non-core operators.

[0037] This application allocates computational tasks to corresponding dedicated modules within the FPGA accelerator based on the type of operator. Through collaborative work between modules, precise matching and efficient utilization of computing resources are achieved. Core operators are handled by the GEMM (General Matrix Multiplication) module, leveraging its parallel computing advantages. Non-core operators are centrally processed by the ALU module, avoiding the consumption of core computing resources. Simultaneously, the interaction of working status between modules ensures smooth data flow and coordinated task scheduling. In practical applications, the ALU and GEMM modules can be pre-integrated into the FPGA accelerator, with their functional boundaries and data interfaces defined through hardware programming. During computation, the scheduler identifies the type of operator currently being processed in the computation graph. If it is a core operator, it is assigned to the GEMM module; otherwise, it is assigned to the ALU module. The two modules interact in real-time via an internal bus, enabling seamless data transfer and parallel task execution. This application significantly improves the computational efficiency of different types of operators through modular division of labor. The parallel architecture of the GEMM module efficiently handles core matrix operations, while the ALU module quickly completes simple non-core operations. Inter-module interaction avoids data backlog or waiting, ensures the continuity of the computing process, further reduces overall computational latency and energy consumption, and makes full use of FPGA hardware resources, improving chip utilization.

[0038] This application significantly reduces data transfer overhead. By storing fixed data in the FPGA's internal ROM, reserving BRAM space for dynamic data, and reducing data transfer across memory units through operator merging, it completely solves the latency and high energy consumption problems caused by frequent data transfer between DDR and BRAM in traditional solutions. This is particularly suitable for fault early warning models with large amounts of data. The packaging and merging of non-core operators and the division of labor among dedicated modules enable precise matching and efficient utilization of FPGA hardware resources such as BRAM, ALU, and GEMM, avoiding the waste of high-end FPGA chip resources and fully releasing the parallel computing potential of the chip. In addition, the working status interaction mechanism between modules ensures seamless connection between core and non-core operator processing, realizing the continuity and parallelism of the computing process, improving the overall processing speed of inference tasks, and meeting the stringent requirements of high energy efficiency, high real-time performance, and high reliability for computing devices in edge scenarios such as power equipment operation and maintenance.

[0039] like Figure 2 The diagram shown illustrates the second flowchart of the dataflow-based field-programmable gate array (FPGA) inference acceleration method of this application. A more detailed embodiment of this application will be provided to further illustrate the application, specifically the edge intelligent computing scenario for calculating a device fault early warning model. Details are as follows.

[0040] S201, accelerator architecture design.

[0041] like Figure 3 The diagram shown is a schematic of the field-programmable gate array (FPGA) accelerator architecture in this embodiment. The specific design method is as follows: (1) The accelerator starts working from the initial state k=0 of the ALU module and executes the processing logic of k=0 in the ALU module.

[0042] After the accelerator is powered on or started, it directly enters the initial state of the ALU module, k=0. The ALU module will immediately execute the processing logic preset in the k=0 state, such as basic operations like preprocessing and format conversion of the input data for the device fault warning model. This establishes a starting node for the entire calculation process, ensuring the consistency and repeatability of the calculation process by fixing the initial state.

[0043] (2) When the current processing is completed, the ALU module sends a message signal to the GEMM module, including the current state k1, the specific operation (batch_matmul or dense) and the address of the operand in the on-chip data space.

[0044] After the ALU completes the processing logic corresponding to the current state k, it will actively generate a message signal and send it to the GEMM module. Specifically, the message signal can include at least three key pieces of information: ① the current working state k1 of the ALU (i.e., the state number when processing is completed); ② the specific operation that the GEMM needs to perform next, such as batch_matmul, batch matrix multiplication, or dense, fully connected layer operations, etc.; ③ the specific storage address of the operands required for this operation in the FPGA on-chip data space, so that the GEMM can directly read the data without additional lookup.

[0045] This enables task handover between the ALU module and the GEMM module, clearly informing the GEMM module of subsequent computational needs and data locations, preventing the GEMM module from being idle or blindly searching for data, and ensuring efficient collaboration between modules.

[0046] (3) After receiving the message signal from the ALU module and completing the batch_matmul or dense operation, the GEMM module queries the state transition table according to k1, obtains k2, and then sends a message signal containing k2 to the ALU module.

[0047] After receiving and parsing the message signals from the ALU module, the GEMM module reads the operands according to the specified address and completes the batch_matmul or dense operation. After the operation is complete, the GEMM module, based on the current state k1 transmitted from the ALU module, queries a preset state transition table to determine the next working state k2 that the ALU module should execute. Finally, the GEMM module sends a message signal containing k2 back to the ALU module. The preset state transition table records the next hop state corresponding to different states.

[0048] This allows the GEMM module to focus on performing core calculations, while a state transition table ensures a closed loop between calculations and state switching, guaranteeing precise alignment between the subsequent processing logic of the ALU module and the calculation results of the GEMM module.

[0049] (4) The ALU module executes the processing logic of k=k2 according to the message signal received from the GEMM module, and repeats steps (2)-(4) until the calculation ends.

[0050] After receiving the message signal from the GEMM module, the ALU module extracts the state k2, updates its own working state to k2, and immediately executes the processing logic corresponding to k=k2. For example, it performs post-processing and format conversion on the calculation results of the GEMM module, or prepares the operands for the next round of GEMM module calculation. Then, steps (2)-(4) are repeated until the inference calculation of the entire equipment fault early warning model is completed and there is no subsequent state to switch. The loop process of ALU module processing, GEMM module calculation, and ALU module reprocessing is realized, so that the entire calculation process can proceed automatically without manual intervention, ensuring the continuity and integrity of the inference task.

[0051] S202, Accelerator Automated Build.

[0052] Specifically, the following steps can be taken: (1) The equipment fault early warning model calculation graph is traversed for the first time. The weights of the equipment fault early warning model are used as the ROM data of the accelerator. The data type and size of the output tensor Ti of each operation are recorded. On-chip BRAM storage space MAX(Size(Ti)) is reserved for the largest output tensor under each data type. If an output tensor Tj is simultaneously the input of (N>1) operators, (N-1)*Size(Ti) space is reserved for BRAM.

[0053] It should be noted that the weights do not change with calculation, and storing them in ROM ensures data stability and fast read speed. For each data type, the size of the largest tensor under that type is identified, and an on-chip block random access memory (BRAM) space of the same size is reserved for that data type. This can prevent data overflow caused by the tensor size exceeding the storage capacity. If a certain output tensor is simultaneously the input of at least two operators, i.e., a multi-input shared tensor, an additional (N-1)*Size(Ti) space is reserved in the BRAM to cache a copy of that tensor, avoiding conflicts when multiple operators read it simultaneously.

[0054] During the computation process, weight data in ROM is loaded into BRAM or directly provided to the GEMM module for computation. Intermediate results from the ALU processing block and tensor data shared by multiple operators are stored in BRAM, avoiding repeated readings from ROM or external memory. By planning on-chip storage resources in advance, sufficient BRAM space is ensured for tensor data in all computation processes, while avoiding waste of storage resources and laying the foundation for subsequent data reuse and reduced data handling.

[0055] (2) The second traversal of the model computation graph, such as Figure 4 The diagram shown is a computational graph for a device fault early warning model. Operators between dense and batch_matmul operations are merged into single-input, single-output ALU processing blocks, which are then numbered. If the output of an ALU processing block is the input of (N>1) ALU processing blocks, (N-1) operations are added at the end of this processing block to write the results to the BRAM buffer, as shown below. Figure 4 middle ALU As shown.

[0056] The model computation graph is traversed again to filter out dense and batch_matmul operations. All other operators between these two types of operations, such as activation functions, data reshaping, and dimensionality transformations, are integrated into single-input, single-output ALU processing blocks. Each processing block has only one input and one output, facilitating state management and numbering. A unique number is assigned to each ALU processing block. If the output of a certain ALU processing block is the input of N (N>1) other ALU processing blocks, then (N-1) write-to-BRAM operations are added at the end of that ALU processing block to store a copy of the output tensor in BRAM for subsequent ALU processing blocks to read directly, avoiding duplicate computation or data transfer. Integrating non-core operators reduces the number of states in the ALU module and simplifies state transition logic. At the same time, by caching shared tensors in BRAM, the problem of multiple input conflicts is resolved.

[0057] (3) The third post-order traversal of the model computation graph involves merging the connected subgraphs between dense or batch_matmul operations, which contain multiple ALU processing blocks, into the same ALU processing block according to the traversal order, and renumbering them. During the merging process, for ALU processing blocks that are multiple inputs ( Figure 4 middle ALU As shown), if it has already been traversed, directly read the cache written in step (2) as the input of the subsequent ALU processing block.

[0058] The model computation graph is traversed using a post-order traversal, i.e., traversing child nodes first and then parent nodes, to find the connected subgraphs between dense or batch_matmul operations. These subgraphs consist of multiple ALU processing blocks connected in series or parallel and are only connected to these two core operations. All ALU processing blocks in this connected subgraph are merged into a larger ALU processing block according to the traversal order, and a unique number is reassigned to reduce the total number of ALU processing blocks and simplify module management. During the merging process, if an ALU processing block is encountered that has multiple inputs, and its inputs come from other ALU processing blocks that have already been traversed, the cached data of the input tensor written to BRAM in step (2) is directly read without recalculation or transfer from other locations.

[0059] The structure of the ALU processing block has been further optimized to reduce the number of interactions between modules. At the same time, by reusing BRAM cached data, redundant calculations and invalid data transfers are avoided, improving computational efficiency.

[0060] (4) The computation graph of the fourth post-order traversal model, based on Figure 4 The preceding and succeeding relationships of ALU blocks, dense and batch_matmul operations are defined. A signal transmission operation is inserted at the end of the corresponding ALU block to send the current state number to the GEMM module, and a complete state transition table is constructed. For example... Figure 5 The image shown is a schematic diagram of a state transition table.

[0061] A post-order traversal is performed again to analyze the predecessor and successor relationships between each ALU processing block, dense operation, and batch_matmul operation. At the end of each ALU processing block, a message signal sending operation is inserted to ensure that the ALU can accurately send task instructions to the GEMM. Based on the predecessor and successor relationships of all modules and the ALU processing block numbers, a complete state transition table is constructed as the basis for the GEMM module to provide feedback on its state.

[0062] (5) According to the definition of the operator, Figure 3 The operator operations in each ALU processing logic are converted into HLS or Verilog implementations.

[0063] Based on the specific definitions of each operator in the model, such as the calculation logic of the activation function and the rules for data transformation, the processing logic corresponding to each ALU processing block is converted into hardware description language code supported by the FPGA, HLS (High-Level Synthesis) code, or Verilog code. This process transforms the software-level processing logic into executable code for the FPGA hardware, completing the final mapping from software logic to hardware architecture, enabling the accelerator to be physically implemented and run.

[0064] It should be noted that the dense operation and the batch_matmul operation are the core computational operators in the inference process of the equipment fault early warning model, and are the key computational links for the model to complete fault feature extraction and logical judgment. The above embodiments are only used as examples of equipment fault early warning models to illustrate this application. In other embodiments of this application, the data flow-based field-programmable gate array inference acceleration method of this application can also be applied to other edge intelligent computing scenarios. The acceleration principle is the same, only the specific core operators and non-core operators are different.

[0065] The method described in this application accelerates the calculation of fault early warning models for converter station main equipment and enables automatic online updates of the diagnostic system. A comparison is made with FPGA computation acceleration using a TVM / VTA architecture, the original of which was based on the von Neumann architecture. After adopting the method described in this application, inference latency is reduced by 50%, chip power consumption is reduced by 25%, and after model updates, accelerator reconstruction is achieved by traversing the new model computation graph, supporting automatic online updates of the diagnostic system.

[0066] This application addresses the issues of slow computation and high power consumption in FPGA accelerators with VTA architecture. By supporting more on-chip operations and dataflow architecture design, it reduces on-chip and off-chip data exchange, thereby lowering computation latency and chip power consumption. It also proposes an automatic accelerator construction method, which can automatically reconstruct the accelerator after online updates of the edge intelligent model in scenarios such as power equipment operation and maintenance by traversing the computation graph of the device fault early warning model.

[0067] like Figure 6 The diagram shown is a schematic of a dataflow-based field-programmable gate array (FPGA) inference acceleration system according to this application, which may include: The rules module is used to traverse the computation graph of the edge intelligent computation to be processed multiple times and perform the following operations: store the fixed data in the computation graph into the read-only memory of the field-programmable gate array (FPGA), and reserve storage space for dynamic data in the block random access memory of the FPGA, which is used to store dynamic data during the computation process when the edge intelligent computation to be processed is performed; package the non-core operators in the computation graph into arithmetic logic unit (ALU) processing blocks, filter out the connected subgraphs composed of multiple ALU processing blocks, and merge the ALU processing blocks in a connected subgraph into a total ALU processing block, which is used to batch process the non-core operators when the edge intelligent computation to be processed is performed; The computing module is used to complete the edge intelligent computing to be processed through the accelerator of the field-programmable gate array. The accelerator includes an arithmetic logic unit module and a general matrix multiplication module. The arithmetic logic unit module is used to process non-core operators in the computation graph of the edge intelligent computing to be processed, and the general matrix multiplication module is used to process core operators in the computation graph of the edge intelligent computing to be processed. The arithmetic logic unit module and the general matrix multiplication module interact with each other to batch process non-core operators when performing edge intelligent computing to be processed.

[0068] In some embodiments of the dataflow-based field-programmable gate array inference acceleration system of this application, the method for exchanging the working states between the arithmetic logic unit module and the general matrix multiplication module includes: After the arithmetic logic unit module completes the processing of non-core operators, it sends a message signal to the general matrix multiplication module, including the current working status of the arithmetic logic unit module, the core operators that need to be processed by the general matrix multiplication module, and the storage address of the operands required by the general matrix multiplication module to process the core operators in the on-chip data space of the field programmable gate array. After the general matrix multiplication module completes the core operator processing, it queries the preset state transition table based on the working state of the arithmetic logic unit module received from the arithmetic logic unit module to obtain the working state of the arithmetic logic unit module for the next round of calculation, and sends a message signal including the working state of the arithmetic logic unit module for the next round of calculation to the arithmetic logic unit module.

[0069] In some embodiments of the dataflow-based field-programmable gate array (FPGA) inference acceleration system of this application, the method for reserving storage space for dynamic data in the block random access memory of the FPGA includes: Based on the data type and data size of the output tensor of each operation in the edge intelligent computing to be processed, the storage space of the largest output tensor under each data type is reserved in the block random access memory of the field programmable gate array. If the output tensor is used as the input of at least two operators simultaneously, an additional (N-1)*Size(Ti) storage space is reserved; where N is the number of operators whose output tensor is used as the input simultaneously, and Ti is the data size of the output tensor.

[0070] In some embodiments of the dataflow-based field-programmable gate array inference acceleration system of this application, after packaging the non-core operators in the computation graph into arithmetic logic unit processing blocks, the system further includes: If the output of an arithmetic logic unit (ALU) processing block is the input of at least two ALU processing blocks, then (N-1) additional operations to write the output to the block random access memory cache are added after the output ALU processing block.

[0071] In some embodiments of the dataflow-based field-programmable gate array inference acceleration system of this application, the method of merging arithmetic logic unit processing blocks in a connected subgraph into a total arithmetic logic unit processing block includes: According to the traversal order, multiple arithmetic logic unit (ALU) processing blocks are merged into a single ALU processing block and renumbered. During the merging process, for ALU processing blocks with multiple inputs, if they have already been traversed, they are directly read from the block random access memory cache as the input for the subsequent ALU processing block.

[0072] In some embodiments of the dataflow-based field-programmable gate array inference acceleration system of this application, after the multiple traversals of the computation graph of the edge intelligent computing to be processed and the following operations are performed, the system further includes: Based on the definition of each operator, the processing logic corresponding to each arithmetic logic unit processing block is converted into hardware description language code supported by the field-programmable gate array.

[0073] It should be noted that, in the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of each block is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple blocks may be combined or integrated into another device, or some features may be ignored or not executed. The modules described as separate components may or may not be physically separated. The components shown as modules may be one or more physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs.

[0074] Furthermore, in the various embodiments of the present invention, the modules can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional unit.

[0075] This application also provides an electronic device, which may include one or more processors, memory and communication interfaces.

[0076] The memory, communication interface, and processor are coupled together. For example, the memory, communication interface, and processor can be coupled together via a bus.

[0077] The communication interface is used for data transmission with other devices. The memory stores computer program code. This computer program code includes computer instructions, which, when executed by the processor, cause the electronic device to perform the steps of the aforementioned method for determining electricity market position limits.

[0078] The processor can be a processor or controller, such as a Central Processing Unit (CPU), a general-purpose processor, a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The processor can be used to support an electronic device in performing the method steps provided in the above embodiments.

[0079] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. These buses can be categorized as address buses, data buses, control buses, etc.

[0080] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the above-described method for determining the power market position limit.

[0081] The computer-readable storage media involved in this application include random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage media known in the art.

[0082] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A dataflow-based field-programmable gate array (FPGA) inference acceleration method, characterized in that, include: The computation graph of the edge intelligent computing to be processed is traversed multiple times, and the following operations are performed: the fixed data in the computation graph is stored in the read-only memory of the field programmable gate array, and storage space is reserved in the block random access memory of the field programmable gate array for dynamic data, which is used to store the dynamic data in the computation process when performing the edge intelligent computing to be processed. Non-core operators in the computation graph are packaged into arithmetic logic unit processing blocks, and connected subgraphs composed of multiple arithmetic logic unit processing blocks are selected. The arithmetic logic unit processing blocks in a connected subgraph are merged into a total arithmetic logic unit processing block, which is used to batch process non-core operators when performing edge intelligent computing to be processed. The edge intelligent computing to be processed is completed by an accelerator with a field-programmable gate array (FPGA). The accelerator includes an arithmetic logic unit (ALU) module and a general matrix multiplication module. The ALU module is used to process non-core operators in the computation graph of the edge intelligent computing to be processed, and the general matrix multiplication module is used to process core operators in the computation graph of the edge intelligent computing to be processed. The ALU module and the general matrix multiplication module interact with each other to obtain the processing progress of core operators and non-core operators.

2. The data flow-based field-programmable gate array inference acceleration method according to claim 1, characterized in that, The method for exchanging working states between the arithmetic logic unit module and the general matrix multiplication module includes: After the arithmetic logic unit module completes the processing of non-core operators, it sends a message signal to the general matrix multiplication module, including the current working status of the arithmetic logic unit module, the core operators that need to be processed by the general matrix multiplication module, and the storage address of the operands required by the general matrix multiplication module to process the core operators in the on-chip data space of the field programmable gate array. After the general matrix multiplication module completes the core operator processing, it queries the preset state transition table based on the working state of the arithmetic logic unit module received from the arithmetic logic unit module to obtain the working state of the arithmetic logic unit module for the next round of calculation, and sends a message signal including the working state of the arithmetic logic unit module for the next round of calculation to the arithmetic logic unit module.

3. The data flow-based field-programmable gate array inference acceleration method according to claim 1, characterized in that, The method for reserving storage space for dynamic data in the block random access memory of a field-programmable gate array includes: Based on the data type and data size of the output tensor of each operation in the edge intelligent computing to be processed, the storage space of the largest output tensor under each data type is reserved in the block random access memory of the field programmable gate array. If the output tensor is used as the input of at least two operators simultaneously, an additional (N-1)*Size(Ti) storage space is reserved; where N is the number of operators whose output tensor is used as the input simultaneously, and Ti is the data size of the output tensor.

4. The data flow-based field-programmable gate array inference acceleration method according to claim 1, characterized in that, After packaging the non-core operators in the computation graph into arithmetic logic unit processing blocks, the method further includes: If the output of an arithmetic logic unit (ALU) processing block is the input of at least two ALU processing blocks, then (N-1) additional operations to write the output to the block random access memory cache are added after the output ALU processing block.

5. The data flow-based field-programmable gate array inference acceleration method according to claim 4, characterized in that, The method for merging arithmetic logic unit processing blocks in a connected subgraph into a single total arithmetic logic unit processing block includes: According to the traversal order, multiple arithmetic logic unit (ALU) processing blocks are merged into a single ALU processing block and renumbered. During the merging process, for ALU processing blocks with multiple inputs, if they have already been traversed, they are directly read from the block random access memory cache as the input for the subsequent ALU processing block.

6. The dataflow-based field-programmable gate array inference acceleration method according to claim 1, characterized in that, After repeatedly traversing the computation graph of the edge intelligent computing to be processed and performing the following operations, the process further includes: Based on the definition of each operator, the processing logic corresponding to each arithmetic logic unit processing block is converted into hardware description language code supported by the field-programmable gate array.

7. A dataflow-based field-programmable gate array (FPGA) inference acceleration system, characterized in that, include: The rules module is used to traverse the computation graph of the edge intelligent computing to be processed multiple times and perform the following operations: store the fixed data in the computation graph into the read-only memory of the field programmable gate array, and reserve storage space for dynamic data in the block random access memory of the field programmable gate array, which is used to store the dynamic data in the computation process when performing the edge intelligent computing to be processed. Non-core operators in the computation graph are packaged into arithmetic logic unit processing blocks, and connected subgraphs composed of multiple arithmetic logic unit processing blocks are selected. The arithmetic logic unit processing blocks in a connected subgraph are merged into a total arithmetic logic unit processing block, which is used to batch process non-core operators when performing edge intelligent computing to be processed. The computing module is used to complete the edge intelligent computing to be processed through the accelerator of the field-programmable gate array. The accelerator includes an arithmetic logic unit module and a general matrix multiplication module. The arithmetic logic unit module is used to process non-core operators in the computation graph of the edge intelligent computing to be processed, and the general matrix multiplication module is used to process core operators in the computation graph of the edge intelligent computing to be processed. The arithmetic logic unit module and the general matrix multiplication module interact to obtain the processing progress of core operators and non-core operators.

8. The dataflow-based field-programmable gate array inference acceleration system according to claim 7, characterized in that, The method for exchanging working states between the arithmetic logic unit module and the general matrix multiplication module includes: After the arithmetic logic unit module completes the processing of non-core operators, it sends a message signal to the general matrix multiplication module, including the current working status of the arithmetic logic unit module, the core operators that need to be processed by the general matrix multiplication module, and the storage address of the operands required by the general matrix multiplication module to process the core operators in the on-chip data space of the field programmable gate array. After the general matrix multiplication module completes the core operator processing, it queries the preset state transition table based on the working state of the arithmetic logic unit module received from the arithmetic logic unit module to obtain the working state of the arithmetic logic unit module for the next round of calculation, and sends a message signal including the working state of the arithmetic logic unit module for the next round of calculation to the arithmetic logic unit module.

9. The dataflow-based field-programmable gate array inference acceleration system according to claim 7, characterized in that, The method for reserving storage space for dynamic data in the block random access memory of a field-programmable gate array includes: Based on the data type and data size of the output tensor of each operation in the edge intelligent computing to be processed, the storage space of the largest output tensor under each data type is reserved in the block random access memory of the field programmable gate array. If the output tensor is used as the input of at least two operators simultaneously, an additional (N-1)*Size(Ti) storage space is reserved; where N is the number of operators whose output tensor is used as the input simultaneously, and Ti is the data size of the output tensor.

10. The dataflow-based field-programmable gate array inference acceleration system according to claim 7, characterized in that, After packaging the non-core operators in the computation graph into arithmetic logic unit processing blocks, the method further includes: If the output of an arithmetic logic unit (ALU) processing block is the input of at least two ALU processing blocks, then (N-1) additional operations to write the output to the block random access memory cache are added after the output ALU processing block.

11. The dataflow-based field-programmable gate array inference acceleration system according to claim 7, characterized in that, The method for merging arithmetic logic unit processing blocks in a connected subgraph into a single total arithmetic logic unit processing block includes: According to the traversal order, multiple arithmetic logic unit (ALU) processing blocks are merged into a single ALU processing block and renumbered. During the merging process, for ALU processing blocks with multiple inputs, if they have already been traversed, they are directly read from the block random access memory cache as the input for the subsequent ALU processing block.

12. The dataflow-based field-programmable gate array inference acceleration system according to claim 7, characterized in that, After repeatedly traversing the computation graph of the edge intelligent computing to be processed and performing the following operations, the process further includes: Based on the definition of each operator, the processing logic corresponding to each arithmetic logic unit processing block is converted into hardware description language code supported by the field-programmable gate array.

13. An electronic device, characterized in that, include: A memory, one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the steps of the dataflow-based field-programmable gate array inference acceleration method as described in any one of claims 1-6.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the dataflow-based field-programmable gate array inference acceleration method as described in any one of claims 1-6.

Images