FPGA-based full-link adaptive parallel data stream acceleration method and system

Abstract

The application provides a full-link adaptive parallel data stream acceleration method and system based on FPGA. Through a FPGA hardware pre-reading mechanism, a PCIe exchange network dynamic bandwidth allocation technology and a configurable protocol stack hardware engine, full-link adaptive data stream acceleration from a physical layer to an application layer is realized. A physical layer direct connection channel of a multi-protocol device and a host system is established by using PS-PL cooperative processing. An adaptive pipeline mechanism and hardware logic are used to analyze data stream characteristics in real time. Parallel pre-reading of multiple data blocks is performed based on an asynchronous request-response decoupling architecture. Data packets are merged and reorganized through a dynamic bandwidth shaping mechanism and a transmission granularity adaptive optimization engine. The protocol overhead can be reduced. The multi-channel intelligent pre-fetch engine directly writes the host memory, and the data stream rate is dynamically adjusted based on the cache water level sensor, so that the delay time is shortened, and the transmission efficiency is significantly improved.

Description

Technical Field

[0001] This application relates to the field of data transmission and communication technology, specifically to an FPGA-based end-to-end adaptive parallel data flow acceleration method and system. Background Technology

[0002] Information records stored in mobile phones and storage media are key to breakthroughs in most criminal cases, cybercrime cases, and some civil cases. However, due to platform limitations, traditional portable electronic evidence collection equipment has low integration and limited interface functionality. Evidence collection from storage media with different interface types is scattered across various evidence collection devices. Moreover, the large amount of information collected from different storage media is time-consuming, which greatly reduces the efficiency of traditional evidence collection work. Furthermore, evidence collection equipment often adopts a fixed interface design, and each device supports only a limited number of interface types. When encountering new interfaces, external adapters are required, which not only increases link latency but may also lead to data loss or data collection failure due to adapter protocol compatibility issues.

[0003] Meanwhile, the core challenge facing current electronic data forensics technology lies in the efficiency bottleneck of data acquisition. This problem stems from the software-driven serial request-response mechanism. After each read request, the forensic host must wait for a response from the device. This sequential, question-and-answer process leads to frequent protocol interactions and low bandwidth utilization. Existing technologies, from device enumeration and protocol negotiation to data reading, are entirely controlled by software instructions in a serial manner. They rely on software dominated by the host CPU for acquisition. After each read request, the forensic host must wait for a response from the device. This sequential, question-and-answer process leads to frequent protocol interactions, high data frame overhead, and low actual bandwidth utilization. Especially with terabyte-level storage capacity and complex data types, operating system scheduling delays and resource contention further exacerbate data transmission latency, severely restricting the integrity and timeliness of forensic evidence.

[0004] To address the aforementioned issues, this application proposes an FPGA-based end-to-end adaptive parallel data flow acceleration method and system. Summary of the Invention

[0005] This application addresses one or more technical deficiencies in the prior art by proposing the following technical solution.

[0006] Based on the first aspect of this application, an FPGA-based end-to-end adaptive parallel data flow acceleration system is proposed, including an FPGA core module, a host, a PCIe expansion module, a SATA controller, and a USB controller. The FPGA core module connects and interacts with the host via a PCIe Gen x8 bus. The FPGA core module includes a data flow monitoring and intelligent reading scheduling module, a data caching and bandwidth smoothing module, a parallel protocol parsing and acceleration processing module, a data flow control and error detection module, and a high-speed interface and data interaction module, which are connected in sequence to perform hardware-level scheduling, protocol parsing and caching control of the data flow. The PCIe expansion module is connected upstream to the FPGA core module, and provides multiple configurable bandwidth PCIe ports downstream. The USB controller and the SATA controller communicate with the FPGA core module through the PCIe ports allocated by the PCIe expansion module. The USB controller and the SATA controller perform parallel acquisition and transmission of multi-protocol data streams, and the FPGA core module performs unified scheduling and processing of the multi-protocol data streams for efficient data reading.

[0007] Furthermore, the data stream monitoring and intelligent reading scheduling module detects the host data request stream status in real time, identifies multi-protocol data stream characteristics, predicts the data blocks accessed by the host, and determines the data target to be read.

[0008] This step can reduce host waiting time, decrease protocol interaction frequency, and improve bandwidth utilization and system response speed.

[0009] Furthermore, the data caching and bandwidth smoothing module integrates multi-protocol data caching and then dynamically allocates bandwidth and aggregates and reassembles it.

[0010] This step optimizes the continuity of data transmission, improves the throughput efficiency of the entire link, and avoids data congestion and interruption.

[0011] Furthermore, the parallel protocol parsing and acceleration processing module performs hardware-based and multi-channel parallel processing of the USB protocol stack, and interacts directly with the host memory through the DMA controller.

[0012] Furthermore, the data flow control and error detection module performs anomaly detection and response retransmission on multi-protocol data streams through a dual-threshold water level monitoring architecture and an ECC automatic repair mechanism.

[0013] This step enhances the reliability and fault tolerance of data stream transmission, preventing data overflow or interruption.

[0014] Furthermore, the high-speed interface and data interaction module use the PS-PL collaborative architecture to reconstruct multi-protocol data stream formats, perform zero-copy pass-through, and synchronize multi-protocol clocks.

[0015] This step can reduce memory copy latency and improve the efficiency of cross-protocol data exchange.

[0016] Based on the second aspect of this application, a method for accelerating end-to-end adaptive parallel data flow according to the system described above is also proposed, comprising: S1: The USB controller and SATA controller acquire multi-protocol data streams and storage device signals from multiple interfaces, and the PCIe expansion module transmits the storage device signals and multi-protocol data streams in parallel to the FPGA core module. S2: The FPGA core module utilizes the PS-side ARM Cortex core and the PL-side programmable logic unit to collaboratively process storage device signals and multi-protocol data streams, establishing a direct physical layer connection channel between multi-interface devices and the host system; S3: The data stream monitoring and intelligent read scheduling module continuously monitors the multi-protocol data stream and identifies data stream characteristics. It adjusts the data stream level through the intelligent prefetch engine, prefetches host requests across protocols in parallel, and constructs an asynchronous decoupled pipeline through the APU core-RPU core dual-core mechanism to continuously monitor the pipeline depth of the data stream. S4: The data caching and bandwidth smoothing module converts the multi-protocol data stream into PCIe protocol format. When the device's continuous output rate exceeds the host's processing capacity, the cache depth expansion unit is activated. When a host processing gap is detected, data transmission is initiated according to the maximum effective bandwidth of the target interface protocol, and the data stream is smoothed out through the write pointer hysteresis control strategy. S5: The parallel protocol parsing and acceleration processing module hardware-encodes the USB protocol and performs full hardware parsing from the physical layer to the link layer. It stores the parsing results of the data stream into the DDR4 controller, performs hardware acceleration through the digital signal processing module DSP of the programmable logic unit, and uses the DMA controller to write the data stream into the host memory. S6: The data flow control and error detection module deploys configurable water level thresholds in the cache matrix through a dual-threshold water level monitoring architecture, and dynamically triggers flow control signals and pre-fetching mechanisms through a real-time rate adjustment engine. When a single-bit error occurs in the data flow, an ECC automatic repair mechanism is triggered. When a multi-bit error occurs in the data flow, a tagged data isolation mechanism is executed, and data retransmission is initiated using a DMA controller. Intelligent retransmission control is performed through a signal quality detection mechanism of the PCIe link. S7: The high-speed interface and data interaction module use the PS-PL collaborative architecture to reconstruct multi-protocol data formats, perform zero-copy pass-through, and synchronize multi-protocol clocks.

[0017] Furthermore, the dynamic triggering of flow control signals and pre-fetching mechanism through the real-time rate adjustment engine specifically includes: When the amount of cached data exceeds the preset high threshold, a flow control signal is triggered and data reading on the device is suspended, while the PCIe switch's dynamic bandwidth allocation mechanism is activated. When the amount of cached data is lower than the preset low threshold, the prefetch acceleration mode is invoked, and intelligent prefetching of multi-protocol data storage is performed in conjunction with the digital signal processing module (DSP).

[0018] This step can dynamically adjust the data stream rate to ensure the stability and efficiency of data stream transmission.

[0019] Furthermore, step S7 specifically includes: constructing a cross-protocol high-speed data path through the PS-PL collaborative architecture; converting multi-protocol data into a unified descriptor format through a protocol decoupling-reassembly engine; using a three-channel parallel processing architecture with a full-link acceleration core to perform data cleaning, address translation, and flow control signal generation on multi-protocol data; accelerating the data flow through a zero-copy pass-through mechanism; and synchronizing the multi-protocol data clock through phase self-calibration technology and a global clock tree.

[0020] Based on a third aspect of this application, a computer program product is also proposed, having one or more computer programs thereon that, when executed by a computer processor, implement the method described above.

[0021] The technical advantages of this application are as follows: Addressing the problems of high overhead, high host latency, and low transmission efficiency in existing serial interface protocols, this application proposes an FPGA-based end-to-end adaptive parallel data flow acceleration method and system. This achieves end-to-end adaptive data flow acceleration from the physical layer to the application layer, reducing the additional overhead caused by multi-protocol data flow transmission. It can dynamically adjust the data flow rate, realize parallel and efficient processing of multiple data flows, significantly improve the integrity and timeliness of electronic evidence acquisition, promote the intelligence and high throughput of electronic forensics technology, greatly shorten the data flow transmission delay time, and significantly improve transmission efficiency. Attached Figure Description

[0022] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0023] Figure 1 This is a block diagram of an FPGA-based end-to-end adaptive parallel data flow acceleration system provided according to an embodiment of this application.

[0024] Figure 2 This is a processing logic diagram of the FPGA core module of an FPGA-based end-to-end adaptive parallel data flow acceleration system provided according to an embodiment of this application.

[0025] Figure 3This is a flowchart of an FPGA-based end-to-end adaptive parallel data flow acceleration method provided according to an embodiment of this application.

[0026] Figure 4 This is a schematic diagram of the structure of a computer system suitable for implementing the electronic devices of the present application embodiments. Detailed Implementation

[0027] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0028] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0029] Figure 1 The FPGA-based end-to-end adaptive parallel data flow acceleration system of this application is shown, including FPGA core module 102, host 101, PCIe expansion module 103, SATA controller 105 and USB controller 104. The FPGA core module 102 is connected to the host 101 and interacts with data via a PCIe Gen x8 bus. The FPGA core module 102 includes a data flow monitoring and intelligent reading scheduling module, a data caching and bandwidth smoothing module, a parallel protocol parsing and acceleration processing module, a data flow control and error detection module, and a high-speed interface and data interaction module connected in sequence, which are used to perform hardware-level scheduling, protocol parsing and cache control of the data flow. The PCIe expansion module 103 is connected upstream to the FPGA core module 102, and provides multiple configurable bandwidth PCIe ports downstream. The USB controller 104 and the SATA controller 105 communicate with the FPGA core module 102 through the PCIe ports allocated by the PCIe expansion module 103. The USB controller 104 and the SATA controller 105 perform parallel acquisition and transmission of multi-protocol data streams, and the FPGA core module 102 performs unified scheduling and processing of the multi-protocol data streams for efficient data reading.

[0030] It should be noted that the data flow monitoring and intelligent reading scheduling module detects the host data request flow status in real time, identifies multi-protocol data flow characteristics, predicts the data blocks accessed by the host, and determines the data target to be read.

[0031] It should be noted that this step can reduce the host's waiting time, decrease the frequency of protocol interactions, and improve bandwidth utilization and system response speed.

[0032] It should be noted that the data caching and bandwidth smoothing module integrates multi-protocol data caching and then dynamically allocates bandwidth and aggregates and reassembles it.

[0033] It should be noted that this step can optimize the continuity of data transmission, improve the throughput efficiency of the entire link, and avoid data congestion and interruption.

[0034] It should be noted that the parallel protocol parsing and acceleration processing module performs hardware-based and multi-channel parallel processing of the USB protocol stack, and interacts directly with the host memory through the DMA controller.

[0035] It should be noted that the data flow control and error detection module uses a dual-threshold water level monitoring architecture and an ECC automatic repair mechanism to perform anomaly detection and response retransmission for multi-protocol data streams.

[0036] It should be noted that this step can enhance the reliability and fault tolerance of data stream transmission, and prevent data overflow or interruption.

[0037] It should be noted that the high-speed interface and data interaction module use the PS-PL collaborative architecture to reconstruct multi-protocol data stream formats, perform zero-copy pass-through, and synchronize multi-protocol clocks.

[0038] It should be noted that this step can reduce memory copy latency and improve the efficiency of cross-protocol data exchange.

[0039] In a specific embodiment, the host communicates directly with the FPGA core module via the PCIe gen3x8 signal and achieves high-speed data writing. At the same time, it sends instructions to the FPGA core module and receives feedback information from the FPGA core module, and makes corresponding prompts. The FPGA core module adopts Xilinx's Zynq UltraScale+ MPSoCs EV series chips for modular design. In the FPGA core module, the PS system integrates two ARM Cortex processors, and the PL system has rich programmable logic units. It communicates with the host through the board connector of high-speed PCIe signal. The FPGA core module internally achieves data prefetching and asynchronous pipeline scheduling, burst data merging, multi-level FIFO bandwidth smoothing, and full hardware processing of USB protocol stack through innovative data prefetching and asynchronous pipeline scheduling, burst data merging, multi-level FIFO bandwidth smoothing, and full hardware processing of USB protocol stack. It also combines PCIe Gen3x8 switching network to realize multi-protocol interface collaborative expansion, breaking through the performance bottleneck of traditional software serial request-response mode. The PCIe expansion module uses ASMedia PCIe, a low-latency, low-cost, low-power 24-channel packet switch that can support up to 12 downstream ports. Its upstream has PCIe gen3x8 bandwidth, which can build various high-speed I / O systems. The downstream PCIe interface has 16 channels of PCIe 3.1 interface, supporting up to 12 PCIe ports. The ports are configured differently according to different bandwidth requirements. In this embodiment, PCIe x2 bandwidth is allocated to the USB controller and M.2 NVME interface, and PCIe x1 bandwidth is allocated to the SATA controller. The USB controller uses an XHCI host controller, bridging a PCIe x2 to two USB 3.1 ports to achieve reliable high-speed bandwidth while remaining backward compatible with traditional USB functions and devices. One USB 3.1 port uses a USB Type-A interface for connecting mobile phones and USB flash drives, while the other USB 3.1 port uses a Type-C interface for connecting high-speed external hard drives. This dual-interface design facilitates direct connection between different types of media and the host, avoiding signal instability and evidence collection issues caused by interface conversion. As a bridge controller between a PCIe host and a storage device, the SATA controller's uplink port is allocated a PCIe x1 signal to the PCIe expansion module, which is then expanded into two SATA signals for connecting storage devices with the SATA protocol, such as the 2.5-inch SATA interface solid-state drive and the M.2 SATA interface solid-state drive in this embodiment.

[0040] It should be noted that the processing logic diagram of the FPGA core module of the system described in this application is as follows: Figure 2 As shown, this application is used for high-speed forensics. It adopts a hardware and software co-processing architecture, with the main processing running on the host side and the FPGA core module serving as the core acceleration unit. It achieves efficient transmission and adaptive scheduling of multi-protocol data with the host through the PCIe Gen3x8 switching network.

[0041] In a specific embodiment, the data flow monitoring and intelligent read scheduling module monitors the data flow status of multiple protocol interfaces (USB 3.2 / NVME / SATA) in real time based on the programmable logic unit of the FPGA core module, and dynamically adjusts the data scheduling strategy in conjunction with the APU / RPU of the processing system. The core of this module is to adopt a two-level control mechanism, including an intelligent prefetch engine and an asynchronous decoupling pipeline. Intelligent Prefetch Engine: The programmable logic unit of the FPGA core module monitors the data buffer level in real time based on the data flow status of the multi-protocol controller through a flow analysis unit built with integrated block RAM. For read requests via the PCIe Gen3 x8 direct connection channel, the intelligent prefetch engine, processed by the ARM Cortex core, dynamically adjusts the high and low water levels. Combined with an UltraRAM cache matrix, it prevents data overflow and performs intelligent scheduling. During cross-protocol prefetch execution, parallel instructions are initiated through the PCIe switch. The prefetched data is converted via USB / NVMe / SATA protocols and stored in a multi-channel cache matrix. A hierarchical FIFO queue is built through the programmable logic unit to ensure that critical data is output first. Asynchronous decoupled pipeline: Based on the heterogeneous computing architecture of MPSoC, a three-stage parallel architecture of "request initiation - data transmission - new request generation" is constructed. The timing logic is reconstructed through the APU core to eliminate idle intervals in the USB / SATA protocol interaction. The multi-level parallel architecture is implemented using the RPU core. During the data transmission stage, new requests are synchronously initiated through the power management unit (PMU). At the same time, asynchronous cache is used to bridge the clock domains of PCIe and M.2 interface. The pipeline depth is dynamically adjusted through the system monitor to maximize the effective bandwidth utilization of the M.2 NVME interface.

[0042] In a specific embodiment, the data caching and bandwidth smoothing module constructs a cross-protocol dynamic bandwidth shaping system based on the heterogeneous architecture of the FPGA core module. Through the collaboration of programmable logic units and the processing system, it achieves an end-to-end throughput efficiency improvement. The FPGA core module uses PCIe Gen3 x8 channels to build a high-speed cache matrix, and performs transparent conversion from USB / SATA / NVME protocol to PCIe protocol through programmable logic units. During the conversion process, UltraRAM is used to retain the addressing mapping relationship between the data payload and the USB / SATA / NVME device. When the device's continuous output rate exceeds the host's processing capacity, the cache depth extension unit is automatically activated. When a host processing gap is detected, burst data transmission is initiated according to the maximum effective bandwidth of the target interface protocol PCIe. At the same time, a write pointer hysteresis control strategy is adopted to smooth the output data stream and eliminate the risk of transmission interruption, thus jointly optimizing transmission efficiency and stability.

[0043] In a specific embodiment, the parallel protocol parsing and acceleration processing module implements hardware-based processing of the USB protocol and parallel transmission with the host based on the heterogeneous acceleration architecture of the FPGA core module: In the programmable logic unit, a dedicated circuit for USB protocol processing is constructed using Verilog HDL. The entire hardware is analyzed from the physical layer to the link layer. The physical layer decoding module uses a differential receiver to directly process USB 3.2 multi-channel signals, perform 8b / 10b decoding and clock recovery, and achieve nanosecond-level response delay control. The link layer processing unit integrates frame header recognition, CRC check and sequence reassembly functions. It uses block RAM for data buffering and builds a multi-channel parallel processing architecture, which can process 4 USB data streams at the same time, with a theoretical throughput of 5Gbps / lane. Multi-protocol collaboration connects the processing system via a PCIe Gen3x8 switching network. The APU core is responsible for protocol status monitoring, the RPU core handles real-time interrupts, and the hardware accelerator and DMA controller work together to directly store the physical layer decoding results into the DDR4 controller with ECC verification. The data is written to the host memory through the multi-channel DMA controller. The transmission process occupies more than 90% of the PCIe Gen3 x8 channel bandwidth. Secure boot is ensured and firmware integrity is guaranteed. The system monitor tracks the status of each processing unit in real time. The data transfer process is hardware-accelerated by the digital signal processing module (DSP) in the programmable logic unit. This process supports AES-256 real-time encryption. The power management unit (PMU) of the processing system dynamically adjusts power consumption, keeping the interface latency below 2μs when working under full load. It connects to the multi-functional expansion module through USB 3.2, TYPE-C, SATA, M.2 SATA, and M.2 NVME interfaces. Its controller is directly mounted on the PCIe switching network, forming a parallel transmission architecture with the main data path. The FPGA core module bypasses the CPU and writes data directly to the host memory through the direct memory access mechanism, which can reduce CPU intervention latency and avoid the host side becoming a bottleneck.

[0044] In a specific embodiment, the data flow control and error detection module establishes a closed-loop reliability assurance system based on the heterogeneous architecture of the FPGA core module, and performs hardware-accelerated fault-tolerant control through programmable logic units, specifically including: Configurable water level thresholds are deployed in the UltraRAM cache matrix (high threshold corresponds to 90% load of PCIe Gen3 x8 bandwidth, low threshold corresponds to 30% load), and a dynamic adjustment baseline is established through the AXI flow control interface; when the cached data volume is higher than the high threshold, the flow control signal is triggered by the RPU core to suspend data reading on the device side, and the dynamic bandwidth allocation mechanism of the PCIe switch is activated at the same time; when the cached data volume is lower than the low threshold, the prefetch acceleration mode is called by the APU core, and intelligent prefetching of USB3.2\TYPE-C\SATA\M.2 SATA\M.2 NVME storage is realized in combination with the digital signal processing module DSP. When a single-bit error occurs, the UltraRAM's ECC automatic repair mechanism is triggered, and the event is recorded by the PS-side security monitoring module. When a multi-bit error occurs, tagged data isolation is performed and the metadata tracking chain is marked, and a retransmission is initiated through the DMA controller. The signal quality of the PCIe link is monitored through eye diagram analysis at the PL-side, and the retransmission time window is dynamically adjusted to configure an enhanced fault tolerance level for critical data channels (M.2 NVME, SATA, and USB device interfaces), increasing the maximum number of retries by 3 times.

[0045] In a specific embodiment, the high-speed interface and data interaction module builds a cross-protocol high-speed data path based on the PS-PL collaborative architecture of the FPGA core module, and realizes lossless protocol conversion through a hardware acceleration engine. The workflow of this module is as follows: Protocol Decoupling-Reassembly Engine: The protocol physical layer processing unit is deployed on the PL side, retaining the electrical characteristics of the NVMe, USB 3.2 and SATA interfaces. The protocol encoding scheme is dynamically stripped through DSP Slice. The link layer processing firmware is run with dual ARM Cortex cores. The payload and AXI address tags of NVMe, USB and SATA storage are extracted and reconstructed into a unified data descriptor format. Uplink transmission (from storage medium to host) improves throughput through the multi-packet aggregation function of PCIe switch, and downlink transmission (from host to storage medium) uses UltraRAM cache to achieve atomic data segmentation. The end-to-end acceleration core integrates a three-channel parallel processing architecture. The data cleaning channel filters protocol residual fields in real time, the tag mapping channel completes the conversion from physical address to host memory address, and the flow control channel generates target protocol flow control signals. This application innovatively adopts a zero-copy pass-through mechanism, where the load data is only updated in the cache header metadata, which can avoid the latency overhead caused by memory copying operations. Heterogeneous Interface Collaborative System: This system resolves multi-clock domain conflicts through phase self-calibration technology, constructs a global clock tree for cross-protocol clock synchronization, designs a unified abstract instruction set to bridge protocol semantic differences, establishes a cross-protocol error code mapping table to coordinate error handling mechanisms, and has the ability to expand to support additional storage interfaces, including frame information structure parsing and command reassembly capabilities for SATA, USB, and NVME interfaces.

[0046] The following is for reference. Figure 3 It illustrates a method for accelerating end-to-end adaptive parallel data flow based on the system described above, including: S1: The USB controller and SATA controller acquire multi-protocol data streams and storage device signals from multiple interfaces, and the PCIe expansion module transmits the storage device signals and multi-protocol data streams in parallel to the FPGA core module. S2: The FPGA core module utilizes the PS-side ARM Cortex core and the PL-side programmable logic unit to collaboratively process storage device signals and multi-protocol data streams, establishing a direct physical layer connection channel between multi-interface devices and the host system; S3: The data stream monitoring and intelligent read scheduling module continuously monitors the multi-protocol data stream and identifies data stream characteristics. It adjusts the data stream level through the intelligent prefetch engine, prefetches host requests across protocols in parallel, and constructs an asynchronous decoupled pipeline through the APU core-RPU core dual-core mechanism to continuously monitor the pipeline depth of the data stream. S4: The data caching and bandwidth smoothing module converts the multi-protocol data stream into PCIe protocol format. When the device's continuous output rate exceeds the host's processing capacity, the cache depth expansion unit is activated. When a host processing gap is detected, data transmission is initiated according to the maximum effective bandwidth of the target interface protocol, and the data stream is smoothed out through the write pointer hysteresis control strategy. S5: The parallel protocol parsing and acceleration processing module hardware-encodes the USB protocol and performs full hardware parsing from the physical layer to the link layer. It stores the parsing results of the data stream into the DDR4 controller, performs hardware acceleration through the digital signal processing module DSP of the programmable logic unit, and uses the DMA controller to write the data stream into the host memory. S6: The data flow control and error detection module deploys configurable water level thresholds in the cache matrix through a dual-threshold water level monitoring architecture, and dynamically triggers flow control signals and pre-fetching mechanisms through a real-time rate adjustment engine. When a single-bit error occurs in the data flow, an ECC automatic repair mechanism is triggered. When a multi-bit error occurs in the data flow, a tagged data isolation mechanism is executed, and data retransmission is initiated using a DMA controller. Intelligent retransmission control is performed through a signal quality detection mechanism of the PCIe link. S7: The high-speed interface and data interaction module use the PS-PL collaborative architecture to reconstruct multi-protocol data formats, perform zero-copy pass-through, and synchronize multi-protocol clocks.

[0047] It should be noted that the dynamic triggering of flow control signals and pre-fetching mechanism through the real-time rate adjustment engine specifically includes: When the amount of cached data exceeds the preset high threshold, a flow control signal is triggered and data reading on the device is suspended, while the PCIe switch's dynamic bandwidth allocation mechanism is activated. When the amount of cached data is lower than the preset low threshold, the prefetch acceleration mode is invoked, and intelligent prefetching of multi-protocol data storage is performed in conjunction with the digital signal processing module (DSP).

[0048] It should be noted that this step can dynamically adjust the data stream rate to ensure the stability and efficiency of data stream transmission.

[0049] It should be noted that step S7 specifically includes: constructing a cross-protocol high-speed data path through the PS-PL collaborative architecture; converting multi-protocol data into a unified descriptor format through a protocol decoupling-reassembly engine; using a three-channel parallel processing architecture with a full-link acceleration core to perform data cleaning, address translation, and flow control signal generation on multi-protocol data; accelerating the data flow through a zero-copy pass-through mechanism; and synchronizing the multi-protocol data clock through phase self-calibration technology and a global clock tree.

[0050] It should be noted that this application addresses the core issues of high overhead, high host latency, and low transmission efficiency of serial interface protocols. It achieves end-to-end adaptive data flow acceleration from the physical layer to the application layer through a hardware prefetching mechanism in the FPGA core module, dynamic bandwidth allocation technology in the PCIe switching network, and a configurable protocol stack hardware engine. First, a heterogeneous protocol processing core is constructed, utilizing the ARM Cortex core on the PS side and the programmable logic unit on the PL side for collaborative processing, establishing a direct physical layer connection channel between multi-protocol devices and the host system. Second, an adaptive pipeline mechanism is adopted, using hardware logic to analyze data flow characteristics in real time and initiating multi-data block prefetching in parallel based on an asynchronous request-response decoupling architecture. Third, dynamic bandwidth shaping is implemented using the PCIe Gen3x8 switching network, and a transmission granularity adaptive optimization engine merges and reassembles data packets to reduce protocol overhead. Simultaneously, multi-protocol processing logic is embedded to achieve hardware acceleration at the physical layer / link layer, combined with a multi-channel intelligent prefetch engine for direct writing to host memory. Finally, a cache level sensor dynamically adjusts the data flow rate, integrates a storage controller with ECC verification, and constructs an end-to-end reliable transmission system, significantly shortening latency and improving transmission efficiency.

[0051] The following is for reference. Figure 4 It shows a schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0052] like Figure 4 As shown, the computer system includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 402 or programs loaded from storage section 408 into random access memory (RAM) 403. RAM 403 also stores various programs and data required for system operation. CPU 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0053] The following components are connected to I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a liquid crystal display (LCD) and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card and a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. Drive 410 is also connected to I / O interface 405 as needed. Removable media 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 410 as needed so that computer programs read from them can be installed into storage section 408 as needed.

[0054] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs the functions defined in the methods of this application. It should be noted that the computer-readable storage medium of this application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0055] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0056] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0057] The modules described in the embodiments of this application can be implemented in software or in hardware.

[0058] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the electronic device, the electronic device comprises: an FPGA core module, a host, a PCIe expansion module, a SATA controller, and a USB controller. The host integrates the FPGA core module, which connects and interacts with the host via a PCIe Gen x8 bus. The FPGA core module includes, in sequence, a data flow monitoring and intelligent read scheduling module, a data caching and bandwidth smoothing module, a parallel protocol parsing and acceleration module, a data flow control and error detection module, and a high-speed interface and data interaction module, used for hardware-level scheduling, protocol parsing, and caching control of the data flow. The upstream of the PCIe expansion module is connected to the FPGA core module, and the downstream provides multiple configurable bandwidth PCIe ports. The USB controller and the SATA controller communicate with the FPGA core module through the PCIe ports allocated by the PCIe expansion module. The USB controller and the SATA controller perform parallel acquisition and transmission of multi-protocol data streams, and the FPGA core module performs unified scheduling and processing of the multi-protocol data streams, realizing collaborative forensics and efficient data reading of the storage device.

[0059] Finally, it should be noted that the above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. An FPGA-based end-to-end adaptive parallel dataflow acceleration system, characterized in that, Includes FPGA core module, host, PCIe expansion module, SATA controller and USB controller; The FPGA core module connects and interacts with the host via a PCIe Genx8 bus. The FPGA core module includes a data flow monitoring and intelligent reading scheduling module, a data caching and bandwidth smoothing module, a parallel protocol parsing and acceleration processing module, a data flow control and error detection module, and a high-speed interface and data interaction module, which are connected in sequence to perform hardware-level scheduling, protocol parsing and caching control of the data flow. The PCIe expansion module is connected upstream to the FPGA core module, and provides multiple configurable bandwidth PCIe ports downstream. The USB controller and the SATA controller communicate with the FPGA core module through the PCIe ports allocated by the PCIe expansion module. The USB controller and the SATA controller perform parallel acquisition and transmission of multi-protocol data streams, and the FPGA core module performs unified scheduling and processing of the multi-protocol data streams for efficient data reading.

2. The system according to claim 1, characterized in that, The data stream monitoring and intelligent reading scheduling module detects the host data request stream status in real time, identifies multi-protocol data stream characteristics, predicts the data blocks accessed by the host, and determines the data target to be read.

3. The system according to claim 1, characterized in that, The data caching and bandwidth smoothing module integrates multi-protocol data caching and then dynamically allocates bandwidth and aggregates and reassembles it.

4. The system according to claim 1, characterized in that, The parallel protocol parsing and acceleration processing module performs hardware-based and multi-channel parallel processing of the USB protocol stack, and interacts directly with the host memory through the DMA controller.

5. The system according to claim 1, characterized in that, The data flow control and error detection module uses a dual-threshold water level monitoring architecture and an ECC automatic repair mechanism to perform anomaly detection and response retransmission for multi-protocol data streams.

6. The system according to claim 1, characterized in that, The high-speed interface and data interaction module use the PS-PL collaborative architecture to reconstruct multi-protocol data stream formats, perform zero-copy pass-through, and synchronize multi-protocol clocks.

7. A method for accelerating end-to-end adaptive parallel data flow according to any one of claims 1-6, characterized in that, include: S1: The USB controller and SATA controller acquire multi-protocol data streams and storage device signals from multiple interfaces, and the PCIe expansion module transmits the storage device signals and multi-protocol data streams in parallel to the FPGA core module. S2: The FPGA core module utilizes the PS-side ARM Cortex core and the PL-side programmable logic unit to collaboratively process storage device signals and multi-protocol data streams, establishing a direct physical layer connection channel between multi-interface devices and the host system; S3: The data stream monitoring and intelligent read scheduling module continuously monitors the multi-protocol data stream and identifies data stream characteristics. It adjusts the data stream level through the intelligent prefetch engine, prefetches host requests across protocols in parallel, and constructs an asynchronous decoupled pipeline through the APU core-RPU core dual-core mechanism to continuously monitor the pipeline depth of the data stream. S4: The data caching and bandwidth smoothing module converts the multi-protocol data stream into PCIe protocol format. When the device's continuous output rate exceeds the host's processing capacity, the cache depth expansion unit is activated. When a host processing gap is detected, data transmission is initiated according to the maximum effective bandwidth of the target interface protocol, and the data stream is smoothed out through the write pointer hysteresis control strategy. S5: The parallel protocol parsing and acceleration processing module hardware-encodes the USB protocol and performs full hardware parsing from the physical layer to the link layer. It stores the parsing results of the data stream into the DDR4 controller, performs hardware acceleration through the digital signal processing module DSP of the programmable logic unit, and uses the DMA controller to write the data stream into the host memory. S6: The data flow control and error detection module deploys configurable water level thresholds in the cache matrix through a dual-threshold water level monitoring architecture, and dynamically triggers flow control signals and pre-fetching mechanisms through a real-time rate adjustment engine. When a single-bit error occurs in the data flow, an ECC automatic repair mechanism is triggered. When a multi-bit error occurs in the data flow, a tagged data isolation mechanism is executed, and data retransmission is initiated using a DMA controller. Intelligent retransmission control is performed through a signal quality detection mechanism of the PCIe link. S7: The high-speed interface and data interaction module use the PS-PL collaborative architecture to reconstruct multi-protocol data formats, perform zero-copy pass-through, and synchronize multi-protocol clocks.

8. The method according to claim 7, characterized in that, The mechanism for dynamically triggering flow control signals and pre-fetching via a real-time rate adjustment engine specifically includes: When the amount of cached data exceeds the preset high threshold, a flow control signal is triggered and data reading on the device is suspended, while the PCIe switch's dynamic bandwidth allocation mechanism is activated. When the amount of cached data is lower than the preset low threshold, the prefetch acceleration mode is invoked, and intelligent prefetching of multi-protocol data storage is performed in conjunction with the digital signal processing module (DSP).

9. The method according to claim 7, characterized in that, Step S7 specifically includes: constructing a cross-protocol high-speed data path through the PS-PL collaborative architecture; converting multi-protocol data into a unified descriptor format through a protocol decoupling-reassembly engine; using a three-channel parallel processing architecture with a full-link acceleration core to perform data cleaning, address translation, and flow control signal generation on multi-protocol data; accelerating the data flow through a zero-copy pass-through mechanism; and synchronizing the multi-protocol data clock through phase self-calibration technology and a global clock tree.

10. A computer program product having one or more computer programs thereon, characterized in that, When the computer program is executed by a computer processor, the method described in any one of claims 7-9 is performed.

Images