Heterogeneous computing storage device supporting on-site data processing

Abstract

The invention discloses a heterogeneous computing storage device supporting on-site data processing. The heterogeneous computing storage device comprises a main control unit, an interface protocol analysis unit, a preprocessing unit, a record storage unit and an unloading unit, the interface protocol analysis unit is respectively connected with the preprocessing unit and the main control unit through a first link and a second link; the preprocessing unit is connected with the main control unit through a third link; the main control unit is respectively connected with the record storage unit and the unloading unit through a fourth link and a fifth link; the preprocessing unit is used for acquiring the multi-source heterogeneous data subjected to protocol analysis and extracting effective information through own computing power for forwarding; according to the invention, the main control unit is taken as a core, other units are combined to construct the full-exchange link, and each unit has independent computing power, so that the communication efficiency of each other is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more particularly to a heterogeneous computing storage device that supports on-site data processing. Background Art

[0002] At present, heterogeneous computing and storage platforms near the data side are mainly used in various types of mobile information processing platforms, especially high-speed and high-mobility application scenarios. This technology aims to collect, process on-site, store in large capacity and process efficiently and intelligently massive amounts of data in real time, providing intelligent data processing capabilities for mobile platforms.

[0003] However, the data processing efficiency and reliability of existing storage devices on the near-data side are relatively low. The reason is that the near-data side is often faced with massive amounts of raw data, with huge amounts of data, various types of data, and different data formats. At the same time, the data is often very time-sensitive. To process the above data, it is often necessary to save it first and then process it offline, which will greatly reduce the timeliness of the intelligence data and even lose its intelligence value. At the same time, offline processing also reduces the data processing efficiency. In addition, due to the limitations of the overall energy consumption supply of the mobile platform, it is often impossible to deploy high-performance and highly reliable computing chips on the near-data side. Therefore, the reliability of data processing and intelligence extraction on the near-data side is often low.

[0004] Therefore, how to provide a high-performance and high-reliability near-data-side storage device is an urgent problem to be solved by those skilled in the art. Summary of the invention

[0005] In view of this, the present invention provides a heterogeneous computing storage device that supports on-site data processing, provides a new processing architecture and mechanism, and improves processing energy efficiency and reliability.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A heterogeneous computing storage device supporting on-site data processing comprises a main control unit, an interface protocol parsing unit, a preprocessing unit, a record storage unit and an unloading unit.

[0008] The interface protocol parsing unit is connected to the preprocessing unit and the main control unit via a first link and a second link respectively; the preprocessing unit is connected to the main control unit via a third link; the main control unit is connected to the record storage unit and the unloading unit via a fourth link and a fifth link respectively.

[0009] The preprocessing unit is used to obtain multi-source heterogeneous data that has been parsed by the protocol and extract effective intelligence through its own computing power for forwarding.

[0010] Preferably, the preprocessing unit includes a data preprocessing module, an intelligence extraction module and a data sending module;

[0011] The preprocessing module temporarily stores the received data according to the format, uses the corresponding preprocessing IP to perform preprocessing, and sends the processing result to the intelligence extraction module or the data sending module;

[0012] The intelligence extraction module stores a software stack for information extraction, which is used to extract key information from different data;

[0013] The data sending module is used to integrate the data transmitted by the preprocessing module and the intelligence extraction module, and output and store them.

[0014] Preferably, the preprocessing module includes an AXI interconnection submodule and a data scheduling submodule, a DDR controller, a heterogeneous data preprocessing submodule and multiple Aurora high-speed interfaces interconnected with the AXI interconnection submodule; the multiple Aurora high-speed interfaces are respectively connected to the effective intelligence extraction module and the data sending module.

[0015] Preferably, the interface protocol parsing unit is connected to the AXI interconnect submodule through the Aurora high-speed interface, and is further stored in a DDR3 chip through the interconnected DDR control.

[0016] Preferably, the preprocessing unit further includes a processing mode configuration interface, and the data scheduling unit selects to send the preprocessing result to the intelligence extraction module or the data sending module according to the signal of the processing mode configuration interface.

[0017] Preferably, the data preprocessing module, the intelligence extraction module and the data sending module are respectively equipped with FPGA chips as cores and are interconnected through a GTH×16 high-speed differential serial link.

[0018] Preferably, the interface protocol parsing unit comprises a physical layer interface module, a multi-type protocol parsing module, an adaptive interface control module and a parsed data sending module which are connected in sequence.

[0019] Preferably, the multi-type protocol analysis includes an SRIO protocol analysis submodule, an FC protocol analysis module and an Ethernet protocol analysis module; the physical layer interface module includes an SRIO interface module, an FC interface and an Ethernet interface module corresponding to various protocols.

[0020] Preferably, the recording storage unit includes a PCIE-NVMe interface, a multi-channel parallel controller and a flash memory controller connected in sequence.

[0021] Preferably, the main control unit adopts a D2000 chip as a core, and the D2000 chip is connected to the preprocessing unit, the record storage unit, the interface protocol unit and the unloading unit through a PEX8796 chip.

[0022] Through the above technical solutions, it can be known that compared with the prior art, the present invention discloses a heterogeneous computing storage device that supports data on-site processing, adopts a new on-site real-time processing architecture and mechanism for edge-oriented massive data flow, and realizes a full switching link based on the PCIE Switch in the main control unit, which significantly improves the processing efficiency. The pre-processing architecture with a multi-FPGA variable interconnection topology can realize real-time streaming processing of various complex calculations; the recording storage unit introduces high-speed interface control through a multi-channel parallel controller to provide broadband guarantee for high-speed data reading and writing. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0024] Figure 1 A schematic diagram of the structure of a heterogeneous computing storage device supporting on-site data processing provided by the present invention.

[0025] Figure 2 Schematic diagram of the structure of the main control unit in an embodiment of the present invention.

[0026] Figure 3 Schematic diagram of the framework of the preprocessing unit in the embodiment of the present invention.

[0027] Figure 4 Schematic diagram of the functional structure of a data preprocessing module in an embodiment of the present invention.

[0028] Figure 5 Schematic diagram of the composition structure of the pre-processing unit in an embodiment of the present invention.

[0029] Figure 6 It is a schematic diagram of the framework of the interface protocol parsing unit in an embodiment of the present invention.

[0030] Figure 7 The figure is a schematic diagram of the composition structure of the interface protocol parsing unit in an embodiment of the present invention.

[0031] Figure 8 Schematic diagram of the storage architecture of the record storage unit in an embodiment of the present invention.

[0032] Fig. 9 Schematic diagram of the composition structure of the recording storage unit in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] like Figure 1 and Figure 2 The embodiment of the present invention discloses a heterogeneous computing storage device supporting on-site data processing, including: a main control unit, an interface protocol parsing unit, a preprocessing unit, a record storage unit and an unloading unit; the interface protocol parsing unit is connected to the preprocessing unit and the main control unit through a first link and a second link respectively; the preprocessing unit is connected to the main control unit through a third link; the main control unit is connected to the record storage unit and the unloading unit through a fourth link and a fifth link respectively; the preprocessing unit is used to obtain multi-source heterogeneous data after protocol parsing and extract effective intelligence through its own computing power for forwarding.

[0035] In this embodiment, the main control unit is composed of a multi-core CPU and a high-performance PCIe switching chip, on which a storage software stack and a record storage unit are run to realize the record storage function. With the PCIe Switch as the hub, the interface protocol parsing unit, the preprocessing unit, the record storage unit and the unloading unit are connected as the EP end of the main control unit. At the same time, each EP unit integrates a high-performance FPGA chip and has independent computing and processing functions for streaming data. In addition, under the PCIe switching chip in the main control unit, a full switching link between each EP end is realized. The communication efficiency between EPs is extremely high, and the mass data movement is not limited by the RC upstream limit, which greatly improves the communication bandwidth.

[0036] In addition, the main control unit integrates a multi-core CPU with moderate performance, runs a streamlined Linux kernel system and user-mode protocols, and customizes the file system for the solid-state storage array. It can implement the NVMe OverFibric high-speed remote communication protocol based on the FC high-speed interface, significantly improving the offload bandwidth.

[0037] Specifically, the hardware platform of the main control unit is based on key components such as the Tengrui D2000 processor and the Feiteng X100 bridge chip. The motherboard of the main control unit follows a modular design. After comprehensively considering the CPU performance, interface characteristics, firmware and operating system operating conditions, the optimal logic implementation method is selected, and detailed logic design is carried out for the motherboard power circuit, clock circuit, reset circuit, power-on circuit, motherboard interface buses and interface units.

[0038] The PCIE SWITCH chip uses the PEX8796 chip. The PEX8796 chip can provide 96 PCI-E3.0 channels. Its upstream and downstream port allocation, port quantity, channel width and other parameter settings are very flexible. It can be set through I2C, external serial EEPROM, setting the level status of the corresponding chip pins and host software settings. There are also a certain number of I / O interfaces that can control key signals and meet the requirements of PCIE bus device expansion.

[0039] A group of PCIE×16 signals on Tengrui D2000 are interconnected with the bridge chip Feiteng X100 for external display devices. Another PCIE×16 signal is used to expand the downstream interface through the PCIE SWITCH chip. PCIE SWITCH divides 1 PCIE×8 signal and interconnects with the WX1820 10G chip for expanding 2 10G networks. 1 PCIE×4 expands M.2 for storage of the main control unit; 1 PCIE×4 expands the Gigabit network port for external connection of the main control unit. In addition, among the downstream interfaces expanded by the PCIE SWITCH chip, 1 PCIE×8 signal is used to achieve high-speed interconnection with the pre-processing unit, 2 PCIE×16 signals are used to achieve high-speed interconnection with 2 record storage units, 1 PCIE×16 signal is connected to the unloading unit, and 1 PCIE×8 signal is connected to the interface protocol parsing unit.

[0040] like Figure 3 , Figure 4 and Figure 5 In order to further implement the above technical solutions, the pre-processing unit is based on a high-performance, large-capacity FPGA and equipped with DDR3 memory to build a hardware platform. The FPGA integrates low-jitter, compatible, and balanced serial transceivers, providing high-speed, low-latency interfaces required for data processing, and can run at high speed on PCIe, 10GbE / 40GbE / 100GbE network ports.

[0041] Specifically, the preprocessing unit includes a data preprocessing module, an intelligence extraction module and a data sending module; the preprocessing module temporarily stores the received data according to the data format, uses the corresponding preprocessing IP for preprocessing, and sends the processing results to the intelligence extraction module or the data sending module; the intelligence extraction module is used to integrate the data transmitted by the preprocessing module and the intelligence extraction module, and output and store them.

[0042] Furthermore, the preprocessing module includes an AXI interconnection submodule and a data scheduling submodule interconnected with the AXI interconnection submodule, a DDR controller, a heterogeneous data preprocessing submodule and multiple Aurora high-speed interfaces; the multiple Aurora high-speed interfaces are respectively connected to the intelligence extraction module and the data sending module.

[0043] Furthermore, the preprocessing unit also includes a processing mode configuration interface, and the data scheduling unit selects to send the preprocessing result to the intelligence extraction module or the data sending module according to the signal of the processing mode configuration interface.

[0044] In this embodiment, the three FPGAs of the data preprocessing module, the effective intelligence extraction module, and the data transmission module are interconnected through the GTH×16 high-speed differential serial link, and each FPGA chip is directly connected to the 4G DDR3 memory particles. The FPGA preprocessing unit is designed with reference to the VPX / OpenVPX specification, and the hardware platform is built with three Fudan Micro JFM7VX690T36 chips as the core. The preprocessing unit is interconnected with the interface protocol parsing unit and the main control unit through the GTH×16 high-speed differential serial link of the heterogeneous data preprocessing module and the PCIe×16 link of the data transmission module.

[0045] Among them, the preprocessing module can realize various preprocessing tasks such as invalid frame filtering and data enhancement. The data preprocessing module designs different preprocessing mechanisms for different types of data, such as radar data, SAR images, infrared data, remote sensing data and voice data. After designing processing mechanisms for different source data, they are stored in different preprocessing IPs; the data preprocessing module receives external data through the interface protocol parsing module, and the data received by the interface protocol parsing module is stored in the DDR3 chip through the Aurora high-speed interface channel 0 via the AXI interconnection module. The data scheduling module immediately starts the corresponding data preprocessing IP for preprocessing calculation according to the received data format. The result of the calculation is sent to the effective intelligence recognition module through the Aurora high-speed interface 1 for the next step of processing or to the data sending module for storage of the collected data according to the processing mode configuration interface signal. That is to say, after the data preprocessing module completes the preprocessing of multi-source heterogeneous data, the processed data can be directly sent to the data sending module, sent to the record storage unit for storage, or transmitted to the effective intelligence extraction module for the next step of online processing.

[0046] The effective intelligence extraction module can store the software stack for information extraction to extract key information from different data. For example, it can configure intelligent recognition programs for voice and image data, complete intelligent recognition and extract key information, and transmit the extracted information to the data transmission module.

[0047] The data sending module integrates the data transmitted by the effective situation extraction module and the data preprocessing module, and finally transmits it to the record storage unit through the PCIe X16 high-speed interface for storage.

[0048] like Figure 6 and Figure 7 In order to further implement the above technical solution, the interface protocol parsing unit includes a physical layer interface module, a multi-type protocol parsing module, an adaptive interface control module and a parsing data sending module connected in sequence. Among them, the multi-type protocol parsing includes an SRIO protocol parsing submodule, an FC protocol parsing module and an Ethernet protocol parsing module; the physical layer interface module includes an SRIO interface module, an FC interface and an Ethernet interface module corresponding to various protocols.

[0049] Among them, the adaptive interface control module is mainly used to automatically parse the packet protocols of various transmission interfaces. This parsing is done by analyzing the packet protocol format and packet header characteristics, independently determining the protocol type and selecting the corresponding parsing components for autonomous and accurate parsing processing. The parsable data packets include SRIO, FC, Ethernet and other data packets. At the same time, the adaptive interface control module can also determine the type of data protocol transmitted according to the interface enable signal, parse and integrate the data packets, and send the processed data information to the parsing data analysis and sending module. The parsing data sending module sends the data to the next subsystem for subsequent processing through the custom protocol in the system.

[0050] In this embodiment, a high-performance FPGA is used to implement multiple high-speed transmission interfaces, and real-time parallel protocol parsing can be achieved between each transmission interface. The interface protocol parsing unit builds a hardware platform with FPGA as the core, which can support multiple interface protocols such as 10G, FC, SRIO, etc. At the same time, the FPGA integrates low-jitter, compatible, and balanced serial transceivers, providing high-speed, low-latency interfaces required for data processing. The GTH×16 high-speed differential serial link of the interface protocol parsing unit is connected to the upstream preprocessing unit through the backplane, and the FPGA chip is directly connected to the 8G DDR3 memory particles. The board voltage and temperature monitoring, key information reporting, fault alarm and other functions of the interface protocol parsing unit are all managed by the whole machine. The high-speed transceiver of the FPGA can support high-speed interface protocols such as PCI-E, 10G network port, SRIO, etc. The GTX transceiver of the interface protocol parsing unit FPGA chip supports line speeds of 500Mb / s to 12.5Gb / s, and the GTH transceiver supports 13.1Gb / s.

[0051] like Figure 8 and Fig. 9 In order to further implement the above technical solution, the recording storage unit includes a PCIE-NVMe interface, a multi-channel parallel controller and a flash memory controller connected in sequence.

[0052] In this embodiment, the recording storage unit is mainly used to efficiently store various types of data, including raw data and pre-processed data. Based on high-performance FPGA and NVMe advanced storage architecture, through the high-speed bus interface, the real-time reception, recording and storage function of external input raw data is completed, and some file system functions are realized through FPGA. For management mode, a standard FTP network access interface is provided to the outside.

[0053] The recording storage unit is designed according to the VPX / OpenVPX specification, with FPGA and SSD firmware as the storage master control, and flash memory particles to achieve the 32TB storage requirement of a single board, which is used to support the recording, playback and dumping of multi-channel high-speed bus data. The PCIE×16 high-speed differential serial link of a single recording storage unit is connected to the upstream master control unit through the backplane.

[0054] The recording storage unit mainly consists of FPGA and peripheral circuits and up to 32TB of 3D NAND FLASH. Two NVMe-based SSD controllers are implemented based on FPGA to achieve high-speed read and write operations and large-capacity storage, while also having the characteristics of low power consumption and fewer peripheral devices.

[0055] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0056] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A heterogeneous computing storage device supporting data in-place processing, characterized in that: It includes a main control unit, an interface protocol parsing unit, a pre-processing unit, a record storage unit and an unloading unit; The interface protocol parsing unit is connected to the pre-processing unit and the main control unit via a first link and a second link respectively; The pre-processing unit is connected to the main control unit via a third link; The main control unit is connected to the record storage unit and the unloading unit via a fourth link and a fifth link respectively; The preprocessing unit is used to obtain multi-source heterogeneous data that has been parsed by the protocol and extract effective intelligence through its own computing power for forwarding.

2. A heterogeneous computing storage device supporting data in-place processing according to claim 1, characterized in that: The preprocessing unit includes a data preprocessing module, an intelligence extraction module and a data sending module; The preprocessing module temporarily stores the received data according to the format, uses the corresponding preprocessing IP to perform preprocessing, and sends the processing result to the intelligence extraction module or the data sending module; The intelligence extraction module stores a software stack for information extraction, which is used to extract key information from different data; The data sending module is used to integrate the data transmitted by the preprocessing module and the intelligence extraction module, and output and store them.

3. A heterogeneous computing storage device supporting data in-place processing according to claim 2, characterized in that: The preprocessing module includes an AXI interconnection submodule and a data scheduling submodule interconnected with the AXI interconnection submodule, a DDR controller, a heterogeneous data preprocessing submodule and multiple Aurora high-speed interfaces; the multiple Aurora high-speed interfaces are respectively connected to the effective intelligence extraction module and the data sending module.

4. A heterogeneous computing storage device supporting data in-place processing according to claim 3, characterized in that: The interface protocol parsing unit is connected to the AXI interconnect submodule through the Aurora high-speed interface, and is further stored in the DDR3 chip through the interconnected DDR control.

5. A heterogeneous computing storage device supporting data in-place processing according to claim 3, characterized in that: The preprocessing unit also includes a processing mode configuration interface, and the data scheduling unit selects to send the preprocessing result to the intelligence extraction module or the data sending module according to the signal of the processing mode configuration interface.

6. A heterogeneous computing storage device supporting data in-place processing according to claim 2, characterized in that: The data preprocessing module, the intelligence extraction module and the data sending module are respectively equipped with FPGA chips as cores and are interconnected through a GTH×16 high-speed differential serial link.

7. A heterogeneous computing storage device supporting data in-place processing according to claim 1, characterized in that: The interface protocol analysis unit comprises a physical layer interface module, a multi-type protocol analysis module, an adaptive interface control module and a analyzed data sending module which are connected in sequence.

8. A heterogeneous computing storage device supporting data in-place processing according to claim 7, characterized in that: The multi-type protocol analysis includes an SRIO protocol analysis submodule, an FC protocol analysis module and an Ethernet protocol analysis module; the physical layer interface module includes an SRIO interface module, an FC interface and an Ethernet interface module corresponding to various protocols.

9. The heterogeneous computing storage device supporting data in-place processing according to claim 1, characterized in that: The recording storage unit includes a PCIE-NVMe interface, a multi-channel parallel controller and a flash memory controller connected in sequence.

10. The heterogeneous computing storage device supporting data in-place processing according to claim 1, characterized in that: The main control unit adopts the D2000 chip as the core, and the D2000 chip is connected to the pre-processing unit, the record storage unit, the interface protocol unit and the unloading unit through the PEX8796 chip.

Images