Software and hardware collaborative acceleration system and method

Abstract

The invention relates to a software and hardware collaborative acceleration system and method, which are applied to the field of FPGA (Field Programmable Gate Array) and comprise a software side and a hardware side, the software side is used for calling the driving program according to the application program, generating a driving control instruction according to the driving program and sending the driving control instruction; the hardware side is used for generating a streaming data link according to the driving control instruction, realizing a task acceleration function and generating a task result; the hardware side comprises a static area, a dynamic reconfiguration area and a DDR module; the static area is used for generating a streaming data link according to the driving control instruction and connecting the DDR module with the dynamic reconfiguration area in a communication manner; the dynamic reconfiguration area is used for acquiring to-be-processed data from the DDR module according to the driving control instruction, realizing a task acceleration function, generating a task result and storing the task result to the DDR module; and the DDR module is used for storing to-be-processed data and task results. The method has the technical effects that the system flexibility and dynamics are improved, and the data delay is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of FPGA, in particular to a software and hardware cooperative acceleration system and method. BACKGROUND

[0002] With the rapid development of information technology, the demand for real-time and efficient processing of massive data is increasingly urgent. For example, in application scenarios such as real-time analysis of high-definition images, high-code rate audio and video stream encoding and decoding, and artificial intelligence large model inference, it is often necessary to quickly process a large amount of data and obtain real-time processing results. In order to meet this high-performance computing demand, software and hardware acceleration technology has become a key solution. At present, the commonly used acceleration scheme mainly includes a static acceleration scheme based on FPGA. Developers configure the logic resources of the FPGA, and combine the functions of driving, data transmission, running scheduling, etc. on the software side to realize a specific acceleration function.

[0003] However, in the static acceleration scheme based on FPGA, the FPGA configuration mode is usually static; that is, during system operation, the logic function of the entire FPGA is loaded and fixed at one time, and if the acceleration function is to be changed, the entire FPGA must be reconfigured (commonly known as "burning and writing again"), which will cause the service to be interrupted during the process of changing the acceleration function, consume time, and thus may affect the running efficiency of the entire acceleration task and the user experience. SUMMARY

[0004] In order to help solve the problem that the static configuration of FPGA in the static hardware acceleration scheme of FPGA causes the service to be interrupted during the process of changing the acceleration function, consumes time, and affects the running efficiency of the acceleration task and the user experience, the present application provides a software and hardware cooperative acceleration system and method.

[0005] In a first aspect, the present application provides a software and hardware cooperative acceleration system, which adopts the following technical scheme: the system includes a software side and a hardware side; The software side is configured to call a driver program according to an application program, generate a driving control instruction according to the driver program, and transmit the driving control instruction to the hardware side; The hardware side is configured to receive the driving control instruction transmitted by the software side, generate a streaming data link according to the driving control instruction, realize a task acceleration function, and generate a task result; The hardware side includes a static area, a dynamic reconfiguration area, and a DDR module; The static area is configured to generate a streaming data link according to the driving control instruction transmitted by the software side, and communicatively connect the DDR module and the dynamic reconfiguration area; The dynamic reconfiguration area is configured to receive the drive control instruction transmitted by the software side, obtain data to be processed from the DDR module according to the drive control instruction, implement a task acceleration function, generate a task result, and store the task result to the DDR module. The DDR module is configured to store data to be processed and a task result.

[0006] In a specific implementation, the driver includes a link connection driver for a data link connection and a plurality of acceleration function drivers for implementing acceleration functions; and the software side includes a platform device driver module and a logic device driver module. The platform device driver module is configured to call the link connection driver according to an application, generate a link connection control instruction according to the link connection driver, and transmit the link connection control instruction to the static area of the hardware side. The logic device driver module is configured to call a plurality of the acceleration function drivers according to an application, generate a plurality of acceleration control instructions according to the plurality of the acceleration function drivers, and transmit the plurality of the acceleration control instructions to the dynamic reconfiguration area of the hardware side.

[0007] In a specific implementation, the logic device driver module includes a plurality of logic sub-devices, and the dynamic reconfiguration area includes a plurality of dynamic sub-areas. The number of the logic sub-devices is consistent with the number of the dynamic sub-areas, and one logic sub-device corresponds to one dynamic sub-area. The number of the logic sub-devices is consistent with the number of the acceleration function drivers, and one logic sub-device calls a corresponding acceleration function driver and generates a corresponding acceleration control instruction. Each dynamic sub-area receives an acceleration control instruction generated and transmitted by a corresponding logic sub-device, and executes a corresponding acceleration function according to the received acceleration control instruction.

[0008] In a specific implementation, the hardware side further includes a processor, and the static area includes a reconfiguration engine. The processor is configured to generate a corresponding initial configuration instruction or reconfiguration instruction according to an external acceleration instruction or dynamic update instruction, and transmit the initial configuration instruction or the reconfiguration instruction to the reconfiguration engine. The reconfiguration engine is configured to receive the initial configuration instruction or the reconfiguration instruction, and load a preset acceleration core bit stream to the dynamic sub-area according to the initial configuration instruction or the reconfiguration instruction.

[0009] In one specific implementation, the platform device driver module includes an IO connection driver submodule, and the static area includes an IO connection module; The IO connection module receives the link connection control command generated and transmitted by the IO connection driver submodule, and generates the streaming data link according to the received link connection control command.

[0010] In one specific implementation, the IO connection module includes a core switch network; The core switch network is communicatively connected to the DDR module and the dynamic reconfiguration area, and is used to generate the streaming data link according to the link connection control command, so as to form a path for direct data transmission between the several dynamic sub-regions.

[0011] In one specific implementation, several of the dynamic sub-regions are connected to the IO connection module via an AXI interface.

[0012] In one specific implementation, the AXI interface includes flow control logic to control the flow of data transmission at both ends of the AXI interface.

[0013] Secondly, this application provides a software-hardware co-acceleration method, employing the following technical solution: the software-hardware co-acceleration system includes a software side and a hardware side, the software side includes several logical sub-devices and an IO connection driver sub-module, and the hardware side includes several dynamic sub-regions, an IO connection module, and a DDR module; the method includes: The software side acquires preset applications and drivers, the drivers including a link connection driver for data link connection and several acceleration function drivers for implementing acceleration functions; According to the user requirements of the application, several of the logical sub-devices call the corresponding acceleration function driver, generate corresponding acceleration control instructions, and transmit them to the corresponding dynamic sub-region; The IO connection driver submodule calls the corresponding link connection driver program according to the user requirements of the application, generates link connection control instructions, and transmits them to the corresponding IO connection module. The IO connection module generates a streaming data link according to the link connection control command; Several dynamic sub-regions sequentially obtain data to be processed from the DDR module according to the corresponding acceleration control command and the streaming data link, perform accelerated processing on the data to be processed, generate task results, and transmit them to the DDR module.

[0014] In one specific implementation scheme, before the IO connection driver submodule calls the corresponding link connection driver to generate link connection control commands and transmits them to the corresponding IO connection module, the method further includes: Upon receiving a dynamic update instruction from an external source, the hardware side stops several of the currently running dynamic sub-regions; Several of the aforementioned dynamic sub-regions load the updated preset acceleration core bitstream according to the dynamic update instruction; The software side obtains the updated driver program based on the updated acceleration core bitstream. The updated driver program includes an updated link update driver program for data link connection and several updated function update drivers programs for implementing acceleration functions. The IO connection driver submodule calls the corresponding link connection driver to generate link connection control commands and transmits them to the corresponding IO connection module, including: The IO connection driver submodule calls the corresponding link update driver to generate updated link connection control instructions and transmits them to the corresponding IO connection module.

[0015] In summary, this application has the following beneficial technical effects: 1. Improved system flexibility and dynamism: By utilizing the partial dynamic reconfigurability technology of FPGA, multiple hardware acceleration cores can be dynamically loaded, unloaded or replaced during continuous system operation, thereby overcoming the shortcomings of fixed ASIC functions and service interruption caused by traditional FPGA global reconfiguration, and can seamlessly adapt to rapidly changing business needs.

[0016] 2. Improve data processing performance and reduce latency: Through the configurable I / O connection module located in the static area, a direct streaming data channel can be established between multiple accelerated cores, i.e. multiple dynamic sub-regions; data is directly transferred from one core to another in a pipeline manner inside the chip, avoiding the latency and bandwidth bottleneck caused by frequent access to off-chip DDR, thereby achieving throughput and end-to-end processing latency far exceeding traditional solutions.

[0017] 3. Optimized hardware resource utilization and cost: There is no need to deploy dedicated hardware acceleration cards for each application scenario, nor is it necessary to statically reserve all possible functional logic resources in the FPGA. By using time-division multiplexing of a limited dynamic area to carry different acceleration tasks, "one set of hardware, multiple functions" is achieved, which significantly reduces hardware cost, board area and system power consumption. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the hardware and software co-acceleration system in the embodiments of this application; Figure 2 This is a schematic diagram used to illustrate the data processing and data transmission methods of existing approaches; Figure 3 This is a schematic diagram illustrating the data processing and data transmission methods in the embodiments of this application; Figure 4 This is a flowchart of the hardware and software co-acceleration method in the embodiments of this application. Detailed Implementation

[0019] The following combination Figures 1-4 This application will be described in further detail.

[0020] This application discloses a hardware and software collaborative acceleration system. This system enables flexible and dynamic scheduling of hardware acceleration resources and efficient collaborative computing, improving the system's dynamism and flexibility, optimizing the utilization and cost of hardware resources, and simultaneously enhancing data processing performance and reducing latency.

[0021] With the rapid development of information technology, the demand for real-time and efficient processing of massive amounts of data is becoming increasingly urgent. For example, applications such as real-time analysis of high-definition images, high-bitrate audio and video stream encoding and decoding, and large-scale artificial intelligence model inference typically require rapid processing of large amounts of data and obtaining real-time results. To meet this demand for high-performance computing, hardware and software acceleration technologies have become key solutions. Currently, commonly used acceleration solutions mainly include application-specific integrated circuits (ASICs) and FPGAs.

[0022] In acceleration solutions based on application-specific integrated circuits (ASICs), ASICs are chips customized for specific algorithms or functions, exhibiting extremely high performance and energy efficiency when performing target tasks. However, the functionality of an ASIC is fixed after manufacturing and cannot be changed. Once the algorithmic requirements of the application scenario change, or a functional upgrade is needed, the chip must be redesigned, tape-outed, and manufactured. This results in extremely long development cycles, high R&D costs, and zero flexibility, making it unable to adapt to the rapidly iterating algorithm and application requirements.

[0023] FPGA-based acceleration solutions are typically static. Developers configure the FPGA's logic resources and combine this with software-side functions such as drivers, data transmission, and runtime scheduling to implement specific acceleration functions. Due to its programmability, FPGA overcomes the rigidity problem of ASIC to some extent. However, on the one hand, in FPGA-based static hardware acceleration solutions, the FPGA configuration is usually static; that is, during system operation, the entire FPGA's logic function is loaded and fixed all at once. If the acceleration function needs to be changed, the entire FPGA must be reconfigured (commonly known as "reprogramming"), which leads to service interruptions during the process, wasting time and lacking dynamism, potentially affecting the overall efficiency of the acceleration task and impacting the user experience. On the other hand, to cope with various possible application scenarios, developers may need to deploy multiple hardware acceleration cards with different functions, or reserve hardware resources for all possible functions in the FPGA's static logic. This results in wasted hardware resources and increased system complexity.

[0024] Furthermore, to accommodate complex tasks, developers often interconnect existing solutions to achieve multi-task collaboration. For example, an intelligent video analysis workflow might require a "video decoding core," an "image preprocessing core," and an "AI inference core" to work in sequence. In existing technologies, even when using FPGAs, achieving efficient data flow between multiple cores often relies on off-chip memory (such as DDR), leading to frequent data movement between the chip and external storage, resulting in high latency and bandwidth bottlenecks, thus hindering the improvement of overall system performance. Although buses or networks can be designed internally within FPGAs for interconnection, this interconnection structure is fixed under static configuration, making it difficult to flexibly adapt to dynamically loaded acceleration cores with different functions and interfaces, thereby affecting the efficiency of accelerated tasks and user experience. To help improve the flexibility of hardware and software acceleration systems and accelerate user tasks, this application provides a hardware and software co-acceleration system.

[0025] Reference Figure 1 The hardware-software co-acceleration system comprises a software side and a hardware side. The software side is used to invoke drivers based on the application, generate driver control instructions based on the drivers, and transmit these instructions to the hardware side. The hardware side is used to receive the driver control instructions transmitted from the software side, generate streaming data links based on the driver control instructions, implement task acceleration functions, and generate task results.

[0026] Specifically, the hardware side includes a static area, a dynamic reconfiguration area, a DDR module, and a processor. The processor, also known as the processor system in the diagram, controls the operation of the entire hardware system. This processor system can be an internal processor core within the FPGA or an external processor core. The static area is used to generate streaming data links based on the driver control instructions transmitted from the software side and to connect the DDR module and the dynamic reconfiguration area. The static area is a permanently configured logical part of the FPGA, responsible for the core management and data scheduling of the system; its function remains unchanged during system operation.

[0027] The dynamic reconfiguration area receives drive control commands transmitted from the software side, retrieves data to be processed from the DDR module according to the drive control commands, implements task acceleration, generates task results, and stores the task results in the DDR module. The FPGA supports Partial Reconfiguration (PR) technology. Based on PR technology, multiple independent areas are divided within the logic resources of the FPGA's dynamic reconfiguration area. Each area can be dynamically and independently loaded with different hardware acceleration core bitstreams during continuous system operation, thereby achieving "hot switching" of hardware functions. That is, each time a task is executed, there is no need to restart; the acceleration core bitstream is directly updated, and the I / O connection module is reconfigured, allowing the task to continue execution, thus saving time and accelerating the task. The DDR module is used to store the data to be processed and the task results.

[0028] The software side includes a platform device driver module and a logic device driver module. The drivers include a link connection driver for data link connections and several acceleration function drivers for implementing acceleration functions. The platform device driver module is used to call the link connection driver according to the application, generate link connection control commands based on the link connection driver, and transmit the link connection control commands to the static area on the hardware side. The logic device driver module is used to call several acceleration function drivers according to the application, generate several acceleration control commands based on the acceleration function drivers, and transmit several acceleration control commands to the dynamic reconfiguration area on the hardware side. The platform device driver module includes an I / O connection driver module, and the logic device driver module includes several logical sub-devices.

[0029] Specifically, the logic device driver module includes several logic sub-devices, and the dynamic reconfiguration area includes several dynamic sub-regions. The number of logic sub-devices is the same as the number of dynamic sub-regions, and one logic sub-device controls one dynamic sub-region. The number of logic sub-devices is the same as the number of acceleration function drivers. One logic sub-device calls the corresponding acceleration function driver and generates the corresponding acceleration control command. Each dynamic sub-region receives the acceleration control command generated and transmitted by the corresponding logic sub-device and executes the corresponding acceleration function according to the received acceleration control command.

[0030] The platform device driver module includes an IO connection driver submodule, and the static area includes an IO connection module. The IO connection module receives link connection control commands generated and transmitted by the IO connection driver submodule, and generates streaming data links based on the received commands. The IO connection module includes a core switch network; this network communicates with the DDR module and the dynamic reconfiguration area, and is used to generate streaming data links based on the link connection control commands, enabling direct data transmission between several dynamic sub-areas. Several dynamic sub-areas are connected to the IO connection module via an AXI interface. The AXI interface includes flow control logic to control the flow of data transmission at both ends of the AXI interface.

[0031] The dynamic reconfigurable area on the hardware side is divided into several independent dynamic sub-regions. Each dynamic sub-region corresponds to an independent logical sub-device on the software side. The logical sub-device on the software side can also be understood as the driver module corresponding to each dynamic sub-region on the hardware side, responsible for bitstream loading, status monitoring, and task control of the acceleration core in that region. The I / O connection module in the static area can also be abstracted as an independent logical sub-device, corresponding to the I / O connection driver module on the software side, with its driver sub-module responsible for dynamically configuring the data path. This architecture allows applications to configure and control each acceleration core and the data flow connections between them as if they were operating multiple independent hardware devices.

[0032] The hardware-side I / O connection module is crucial for achieving efficient data flow, primarily comprising a core switch network. This core switch network can be understood as a network of switches composed of multiple pathway switches. In this embodiment, the core switch network is implemented through a series of configurable data pathway switches, such as an interconnect structure based on AXI Switches. Each pathway switch within the core switch network connects a general data interface (such as an AXI4-Stream interface) in the static region to the data interfaces of various dynamic sub-regions. By configuring each pathway switch within the core switch network, transmission paths for data flow can be established or altered between different dynamic sub-regions, and between the dynamic reconfiguration region and external high-speed memory (such as a DDR module), ultimately achieving efficient data transmission.

[0033] In this embodiment, the I / O connection module is configured between multiple dynamic sub-regions to establish a direct, point-to-point streaming data path (e.g., using the AXI4-Stream protocol). This streaming data path can also be understood as data flowing directly from the output of one acceleration core to the input of the next, without needing to pass through off-chip memory, thereby significantly reducing latency and increasing throughput. Simultaneously, the I / O connection module also supports configuring access paths to shared DDR memory for dynamically reconfigured regions. This mode is suitable for scenarios requiring large-capacity intermediate data caching or low data reuse rates, providing flexibility and expanding the application scenarios of this solution.

[0034] For example, refer to Figure 2 In currently used methods, each dynamic sub-region needs to retrieve data via DDR for processing, and then store the processed results of each dynamic sub-region back into DDR. While this method is generally effective, it may experience performance bottlenecks when handling high-density tasks due to limitations in the access speed of the DDR storage module.

[0035] In this embodiment, by setting up the I / O connection module, the processing order of the modules can be pre-defined, and the streaming data path can be set. Then, data transmission can proceed directly according to the pre-defined streaming data path. Data can flow directly from the output of one acceleration core to the input of the next acceleration core without passing through off-chip memory. That is, referring to... Figure 3 The first dynamic sub-region A retrieves the data to be processed from the DDR. After processing the data, it can directly transmit the processing result to dynamic sub-region B. Dynamic sub-region B receives the processing result from the previous module, continues processing, and transmits the processed result directly to dynamic sub-region C. After dynamic sub-region C completes processing, it directly stores the processing result in the DDR storage module. Throughout this process, the initial data retrieval is from the DDR, and the final processing result is transmitted to the DDR module. This allows for direct data transfer between different dynamic sub-regions without repeatedly reading and writing data from the DDR module. This supports complex tasks and high-density tasks where the chained path does not pass through the DDR, maximizing system performance. Figure 3 The order of the dynamic sub-regions A, B, and C is preset by the user and can be changed. In this embodiment, only the order of A, B, and C is used for illustration, and no restriction is made here.

[0036] The static region also includes a reconfiguration engine. Through the reconfiguration engine and the processor, the acceleration core bitstream can be loaded into the corresponding dynamic sub-region. Specifically, the processor generates corresponding initial configuration instructions or reconfiguration instructions based on external acceleration instructions or dynamic update instructions, and transmits these instructions to the reconfiguration engine. The reconfiguration engine receives the initial configuration instructions or reconfiguration instructions and loads the preset acceleration core bitstream into the dynamic sub-region according to these instructions. The acceleration core bitstream, which is the acceleration core logic of each dynamic sub-region, is the foundation for the operation of each dynamic sub-region. Before the system runs, the acceleration core logic needs to be loaded into each dynamic sub-region so that the dynamic reconfiguration region can correctly receive and execute instructions issued by the software.

[0037] In addition, the static area on the hardware side also includes clock logic, reset logic, and other static logic. These are routine configurations that need to be performed when the hardware system is running normally, and can all be implemented using existing methods, so they will not be elaborated here.

[0038] In this embodiment, real-time intelligent video analysis is used as an example for illustration. Assuming that this embodiment requires acceleration tasks in a real-time intelligent video analysis system, common functions of real-time intelligent video analysis processing include H.265 decoding of video streams, image scaling and color space conversion, object detection algorithms, OSD overlay of results, and H.265 encoding; these functions are all independent task modules and can be efficiently completed by writing hardware logic.

[0039] On the hardware side, it's necessary to first divide the FPGA chip into a static region, a dynamic reconfigurable region, and a DDR module. Within the FPGA chip, a static region is defined. This static region primarily includes the core I / O connection module, used to configure the data flow connections for the dynamically reconfigurable region, which can be dynamically updated via a configuration interface; necessary clock logic, reset logic, and other static logic required by the user. The dynamically reconfigurable region can be divided into multiple dynamic sub-regions as needed. Here, we take five dynamic sub-regions as an example, each corresponding to the different application function processing tasks mentioned above, including: Dynamic sub-region 1: H.265 decoding core (input is video stream, output is YUV pixel stream); Dynamic sub-region 2: Image preprocessing core (implements scaling and YUV2RGB conversion); Dynamic sub-region 3: YOLOv5 neural network inference core; Dynamic sub-region 4: Result OSD overlay core; Dynamic sub-region 5: H.265 encoding core.

[0040] Each dynamic sub-region uses a standardized AXI4-Stream interface, ensuring seamless integration with the static region's I / O connection module. It should be noted that this embodiment implements all five main processing parts through dynamic sub-regions. In practical applications, if one or more functions are frequently used or are fixed functions, these frequently used or fixed-function modules can be designed and placed in the static region for implementation as fixed functions.

[0041] In the IO connection module, the core switch network can be implemented using digital circuits based on the principle of a programmable crossbar switch; its design goal is to provide flexible, high-speed, and low-latency streaming data connections between multiple data producers and consumers.

[0042] The I / O connectivity module mainly includes: an input interface unit group, an output interface unit group, a core switch network, and routing configuration registers. The input interface unit group contains N standard AXI4-Stream slave interfaces for receiving data streams from the outputs of various dynamically reconfigurable regions or DDR controllers. The output interface unit group contains M standard AXI4-Stream master interfaces for sending data streams to the inputs of various dynamically reconfigurable regions or DDR controllers. The core switch network can be an N x M switching network composed of a series of multiplexers, with each output port connected to a large multiplexer, and all its input ports connected to all input interface units. The routing configuration registers are a set of memory-mapped registers that can be read and written by the processor system via the AXI4-Lite slave bus. Each register controls the selection signal of the multiplexer corresponding to an output port, thereby determining which input port the output port receives data from. The routing configuration registers allow for the configuration of each switch path within the core switch network. Furthermore, considering that each module processes data at different speeds, data overflow may occur during data transmission. Therefore, flow control logic is designed for each AXI bus to handle transmission requests initiated by multiple input streams to the same output port and to perform fair arbitration. At the same time, this logic fully transmits the TVALID and TREADY handshake signals of the AXI4-Stream protocol to ensure that data does not overflow.

[0043] It should be noted that in the actual implementation of the system, during system startup or task switching, the processor system driver pre-assigns an input port to each output port by writing specific values ​​to the routing configuration register set. For example, by setting the routing register of "output port A" corresponding to dynamic sub-region A to the index value of "input port B" corresponding to dynamic sub-region B, a fixed path can be established from the output of dynamic sub-region A to the input of dynamic sub-region B. Once the route is established, data transmission can begin. Throughout the entire process, the processor system does not need to intervene, achieving high-performance streaming transmission at the hardware level.

[0044] In the software-side processor system driver design, this application embodiment is based on the Linux open-source system to design a software stack for a logical device driver framework. The driver architecture includes a platform driver and logical sub-device drivers. The platform driver corresponds to the static area of ​​the hardware side, and the logical sub-device drivers correspond to the dynamic reconfiguration area of ​​the hardware side. The platform driver is responsible for detecting the FPGA hardware and, based on the device tree information, determining the register space resources, interrupt resources, etc., used by the FPGA, registering the static area devices of the FPGA, and providing control functions for I / O connection modules and loading functions for dynamic sub-regions. The logical sub-device drivers adopt a dynamic loading method, combined with the overlay function of the device tree, to dynamically complete the registration and loading of devices during system operation. Each logical sub-device driver only needs to be developed separately according to the hardware module, which has flexibility and independence.

[0045] When a user needs to update a task or replace one or more acceleration functions, the user application first opens the static platform device, loads the required acceleration core logic, and then loads the corresponding device tree overlay file and the corresponding logic sub-device driver module. This enables dynamic functional updates of the hardware logic. When switching hardware acceleration logic, for example, assuming the object detection algorithm is processed by dynamic sub-region C, if the object detection algorithm is modified (e.g., switching from a YOLO inference model to an R-CNN model), the application needs to stop the current data flow, partially reconfigure dynamic sub-region C through the FPGA's static region device interface, switch to the new acceleration core logic, and replace the device tree overlay file and the corresponding logic sub-module driver. Afterward, the data flow can be resumed. If the scenario changes significantly, the IO connection module can be further modified. For example, if H265 decoding and H265 encoding cores are not needed, the application can configure the IO connection module through the FPGA's static region device interface to switch the connection of the system's dynamically reconfigurable region module to adapt to different scenarios.

[0046] It should be noted that when using this system to perform acceleration tasks, if the user needs to update or completely replace the task midway, the user only needs to stop the current program, update the acceleration core logic in the dynamic sub-region, replace the corresponding driver, and reconfigure the core switch network in the IO module according to the user's required data flow mode to generate a new streaming data link. There is no need to restart the system, thus achieving a hot start. This allows for further acceleration on the basis of accelerated data processing in the dynamic sub-region, and improves the system's flexibility, which can be freely configured according to the user's needs.

[0047] In this embodiment, the complete process of using this system includes: First, system initialization is performed. The FPGA static area is configured first, and the system starts up. This can also be understood as initializing and configuring the basic environment and conventional logic of the system to ensure that the entire system can operate normally. Simultaneously, the "empty" interfaces of the IO connection module and each dynamic sub-region are initialized. This can also be understood as the system not receiving any data or instructions during initialization; therefore, the IO connection module and each dynamic sub-region are set to an empty state to facilitate the subsequent reception of data and instructions for task processing. Afterward, the user can configure the current task through the application. For example, the user can pre-define the data flow link, and then call the corresponding driver program through each logic sub-device and the IO connection driver module to generate the corresponding driver instructions. The driver instructions are then transmitted to the corresponding dynamic sub-region and the IO connection module. The IO connection module configures the core switch network according to the received instructions, ensuring that the configured streaming data link is consistent with the data flow link pre-defined by the user. Each dynamic sub-region receives the corresponding task acceleration instruction. When the current dynamic sub-region receives the data that needs to be processed, it performs task acceleration processing and transmits the processed result to the next dynamic sub-region through a streaming data link. After all dynamic sub-region tasks are completed, the final result is transmitted to the DDR module.

[0048] When task requirements change (such as switching from video analytics to audio processing), the application can command the driver to unload the bitstream of certain dynamic regions and load a new bitstream (such as the audio noise reduction core), while reconfiguring the I / O connection module to establish a new data path. The entire process does not require a system restart, and other unaffected cores can continue to operate.

[0049] In this application, regarding the hardware level: 1. Improved system flexibility and dynamism: By utilizing the partially dynamic reconfigurable technology of FPGA, multiple hardware acceleration cores can be dynamically loaded, unloaded, or replaced during continuous system operation. This overcomes the shortcomings of fixed ASIC functions and service interruptions caused by traditional FPGA global reconfiguration, enabling seamless adaptation to rapidly changing business needs. 2. Enhanced data processing performance and reduced latency: Through configurable I / O connection modules located in the static area, direct streaming data channels can be established between multiple acceleration cores, i.e., multiple dynamic sub-regions. Data is piped directly from one core to another within the chip without frequent access to external DDR, avoiding the latency and bandwidth bottlenecks caused by frequent access to external DDR. This achieves throughput and end-to-end processing latency far exceeding traditional solutions. 3. Optimized hardware resource utilization and cost: There is no need to deploy dedicated hardware acceleration cards for each application scenario, nor is it necessary to statically reserve logic resources for all possible functions within the FPGA. By time-division multiplexing a limited dynamic area to carry different acceleration tasks, "one set of hardware, multiple functions" is achieved, significantly reducing hardware costs, board area, and system power consumption.

[0050] Regarding the software layer: 1. Simplified system management and maintainability: The software-side driver architecture abstracts complex physical devices into multiple easily manageable logical objects. Software developers and system administrators can independently configure, control, and upgrade each hardware acceleration function, just like managing software modules, greatly reducing the complexity of system management and maintenance costs. 2. Enhanced software compatibility and development efficiency: This driver architecture provides applications with a unified and standard device operation interface, decoupling the development of upper-layer application software from the specific implementation of the underlying hardware. Application code does not need to concern itself with the details of bitstream loading and data path configuration, improving code portability and development efficiency.

[0051] Regarding hardware and software collaboration: 1. Achieving reconfigurable computing: The system organically integrates dynamic hardware reconfiguration, configurable data paths, and flexible software drivers to construct a complete and responsive reconfigurable computing system. The system can configure the optimal hardware computing path based on real-time task requirements, achieving a precise match between processing power and task demands. 2. Simplified operation and enhanced user experience: The entire system functions as a powerful and flexible acceleration device. Users do not need to worry about hardware restarts or physical card replacements; they can switch between complex acceleration functions simply through software commands, achieving "one-click switching" and fundamentally improving the user experience.

[0052] Based on the above system, this application also discloses a software and hardware collaborative acceleration method.

[0053] Reference Figure 4The method includes the following steps: S10, the software side obtains preset applications and drivers, including link connection drivers for data link connection and several acceleration function drivers for implementing acceleration functions.

[0054] Specifically, firstly, the software side obtains the preset application and driver. The application contains the user's task requirements, and the driver includes a link connection driver for data link connection and several acceleration function drivers for implementing acceleration functions.

[0055] S20: Several logic sub-devices, based on the user requirements of the application, call the corresponding acceleration function driver, generate the corresponding acceleration control command, and transmit it to the corresponding dynamic sub-region.

[0056] Specifically, the software-side logical sub-device calls the acceleration function driver according to the user task requirements in the application, generates corresponding acceleration control instructions, and transmits them to the corresponding dynamic sub-region, so that the dynamic sub-region can execute acceleration tasks according to the corresponding acceleration instructions.

[0057] S30, the IO connection driver submodule calls the corresponding link connection driver program according to the user requirements of the application, generates link connection control instructions and transmits them to the corresponding IO connection module.

[0058] Specifically, the software-side IO connection driver submodule, based on the user's task requirements in the application (which can also be understood as the data processing order set by the user), calls the corresponding link connection driver to generate link connection control instructions and transmits them to the corresponding IO connection module. This enables the hardware-side IO connection module to generate a streaming data link according to the corresponding instructions, allowing data to be processed and transmitted through the data link.

[0059] S40, the IO connection module generates a streaming data link based on the link connection control command.

[0060] Specifically, when the hardware receives the link connection control command sent by the software, it configures each path switch in the core switch network of the IO module to generate the data flow link required by the user.

[0061] S50: Several dynamic sub-regions sequentially obtain data to be processed from the DDR module according to the corresponding acceleration control instructions and streaming data links, perform accelerated processing on the data to be processed, generate task results, and transmit them to the DDR module.

[0062] Specifically, after the streaming data link is generated, the dynamic sub-regions will execute tasks sequentially according to the order of the data link. The first dynamic sub-region in the data link obtains the data to be processed from the DDR module. After completing the acceleration task in this dynamic sub-region, the result of the acceleration processing is transmitted to the second dynamic sub-region on the data link. After the second dynamic sub-region executes the corresponding acceleration task, the result is transmitted to the third dynamic sub-region on the streaming data link. This process continues until all dynamic sub-regions have completed their acceleration tasks. The last dynamic sub-region generates the final task result and stores the task result in the DDR module for users to view.

[0063] In one embodiment, considering the user's need to change tasks, the following steps can be performed before the IO connection driver submodule calls the corresponding link connection driver to generate link connection control instructions and transmits them to the corresponding IO connection module: First, upon receiving a dynamic update instruction from an external source, the hardware stops several currently running dynamic sub-regions. Then, these dynamic sub-regions load the updated preset acceleration core bitstream according to the dynamic update instruction. This can also be understood as follows: in actual operation, through the static region reconfiguration engine and the hardware-side processor system, the updated acceleration core bitstream is loaded into the replaced dynamic sub-region, replacing the original acceleration function with the new one.

[0064] After the hardware-side acceleration core logic is updated, the software side obtains the updated driver program based on the updated acceleration core bitstream. The updated driver program includes an updated link update driver program for data link connections, and several updated function update drivers for implementing acceleration features. The IO connection driver submodule calls the corresponding link connection driver program to generate link connection control commands and transmits them to the corresponding IO connection module. This includes: the IO connection driver submodule calling the corresponding link update driver program to generate updated link connection control commands and transmitting them to the corresponding IO connection module.

[0065] Figure 4 This is a schematic diagram illustrating the software and hardware collaborative acceleration process in one embodiment. It should be understood that, although... Figure 4 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows; unless explicitly stated herein, there is no strict order requirement for the execution of these steps, and they can be executed in other orders; and Figure 4At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0066] This specific embodiment is merely an explanation of the present invention and is not intended to limit the invention. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of the claims of the present invention.

Claims

1. A hardware and software co-acceleration system, characterized in that: The system includes a software side and a hardware side; The software side is used to call the driver according to the application, generate driver control instructions according to the driver, and transmit the driver control instructions to the hardware side; The hardware side is used to receive the drive control instructions transmitted by the software side, generate a streaming data link according to the drive control instructions, implement task acceleration function, and generate task results. The hardware side includes a static area, a dynamic reconfiguration area, and a DDR module; The static region is used to generate a streaming data link according to the drive control instructions transmitted from the software side, and to connect the DDR module and the dynamic reconfiguration region for communication. The dynamic reconfiguration area is used to receive the drive control instructions transmitted by the software side, obtain the data to be processed from the DDR module according to the drive control instructions, realize the task acceleration function, generate task results, and store the task results in the DDR module; The DDR module is used to store data to be processed and task results.

2. The hardware and software collaborative acceleration system according to claim 1, characterized in that: The driver includes a link connection driver for data link connection and several acceleration function drivers for implementing acceleration functions; the software side includes a platform device driver module and a logic device driver module. The platform device driver module is used to call the link connection driver according to the application, generate link connection control instructions according to the link connection driver, and transmit the link connection control instructions to the static area on the hardware side; The logic device driver module is used to call several acceleration function drivers according to the application, generate several acceleration control instructions according to the several acceleration function drivers, and transmit the several acceleration control instructions to the dynamic reconfiguration area on the hardware side.

3. The hardware and software collaborative acceleration system according to claim 2, characterized in that: The logic device driver module includes several logic sub-devices, and the dynamic reconfiguration area includes several dynamic sub-regions. The number of logical sub-devices is the same as the number of dynamic sub-regions, and one logical sub-device controls one dynamic sub-region. The number of logical sub-devices is the same as the number of acceleration function drivers. Each logical sub-device calls the corresponding acceleration function driver and generates a corresponding acceleration control instruction. Each of the dynamic sub-regions receives acceleration control instructions generated and transmitted by the corresponding logical sub-device, and executes the corresponding acceleration function according to the received acceleration control instructions.

4. The hardware and software collaborative acceleration system according to claim 3, characterized in that: The hardware side also includes a processor, and the static area includes a reconfiguration engine; The processor is used to generate corresponding initial configuration instructions or reconfiguration instructions based on external acceleration instructions or dynamic update instructions, and transmit the initial configuration instructions or the reconfiguration instructions to the reconfiguration engine; The reconfiguration engine is used to receive the initial configuration instruction or the reconfiguration instruction, and load the preset acceleration core bitstream into the dynamic sub-region according to the initial configuration instruction or the reconfiguration instruction.

5. The hardware and software collaborative acceleration system according to claim 3, characterized in that: The platform device driver module includes an IO connection driver submodule, and the static area includes an IO connection module; The IO connection module receives the link connection control command generated and transmitted by the IO connection driver submodule, and generates the streaming data link according to the received link connection control command.

6. The hardware and software collaborative acceleration system according to claim 5, characterized in that: The IO connection module includes a core switch network; The core switch network is communicatively connected to the DDR module and the dynamic reconfiguration area, and is used to generate the streaming data link according to the link connection control command, so as to form a path for direct data transmission between the several dynamic sub-regions.

7. The hardware and software collaborative acceleration system according to claim 5, characterized in that: Several of the dynamic sub-regions are connected to the IO connection module via an AXI interface.

8. The hardware and software collaborative acceleration system according to claim 7, characterized in that: The AXI interface includes flow control logic to control the flow of data transmission at both ends of the AXI interface.

9. A software-hardware co-acceleration method, wherein the method is applied to the software-hardware co-acceleration system of claim 1, characterized in that: The hardware-software co-acceleration system includes a software side and a hardware side. The software side includes several logical sub-devices and an I / O connection driver sub-module. The hardware side includes several dynamic sub-regions, an I / O connection module, and a DDR module. The method includes: The software side acquires preset applications and drivers, the drivers including a link connection driver for data link connection and several acceleration function drivers for implementing acceleration functions; According to the user requirements of the application, several of the logical sub-devices call the corresponding acceleration function driver, generate corresponding acceleration control instructions, and transmit them to the corresponding dynamic sub-region; The IO connection driver submodule calls the corresponding link connection driver program according to the user requirements of the application, generates link connection control instructions, and transmits them to the corresponding IO connection module. The IO connection module generates a streaming data link according to the link connection control command; Several dynamic sub-regions sequentially obtain data to be processed from the DDR module according to the corresponding acceleration control command and the streaming data link, perform accelerated processing on the data to be processed, generate task results, and transmit them to the DDR module.

10. The method according to claim 9, characterized in that: Before the IO connection driver submodule calls the corresponding link connection driver to generate link connection control commands and transmits them to the corresponding IO connection module, the following steps are also included: Upon receiving a dynamic update instruction from an external source, the hardware side stops several of the currently running dynamic sub-regions; Several of the aforementioned dynamic sub-regions load the updated preset acceleration core bitstream according to the dynamic update instruction; The software side obtains the updated driver program based on the updated acceleration core bitstream. The updated driver program includes an updated link update driver program for data link connection and several updated function update drivers programs for implementing acceleration functions. The IO connection driver submodule calls the corresponding link connection driver to generate link connection control commands and transmits them to the corresponding IO connection module, including: The IO connection driver submodule calls the corresponding link update driver to generate updated link connection control instructions and transmits them to the corresponding IO connection module.

Images