PCIE-based intelligent heterogeneous soc inference acceleration system and method

Abstract

The application relates to the technical field of computer acceleration, and provides an intelligent heterogeneous SOC inference acceleration system and method based on PCIE, wherein the intelligent heterogeneous SOC is designed in the form of a PCIE device acceleration card, is connected with a host computer in the mode of a PCIE interface, the heterogeneous core memory of the intelligent heterogeneous SOC board card is mapped to a PCIE BAR space by using a PCIE inbound function of the intelligent heterogeneous SOC board card, an interrupt interaction mechanism of the host computer and the intelligent heterogeneous SOC board card is designed, operation is performed by the host computer, scheduling work of the host computer on an NPU, an FFT and a multi-core DSP of the intelligent heterogeneous SOC board card is completed, efficient data transmission can be performed by using a PCIE DMA function of the intelligent heterogeneous SOC board card, and efficient deployment and the efficiency of collaborative work of the multi-core DSP, the NPU and the FFT are improved.

Description

Technical Field

[0001] This invention belongs to the field of computer acceleration technology, and relates to an intelligent heterogeneous SOC inference acceleration system and method based on PCIE. Background Technology

[0002] In recent years, with the development of DSP chip technology and the rapid development of AI chips, ARM cores and dedicated NPU hardware accelerators have been gradually introduced into domestically developed DSP chips, which include FFT hardware accelerators, making their application scenarios more extensive. Correspondingly, problems have also arisen: in the application scenarios of existing intelligent heterogeneous SOC boards as computing, image recognition and inference acceleration boards, how to efficiently deploy them while improving the efficiency of multi-core DSP, NPU and FFT working together has become an unsolved technical problem.

[0003] In embedded systems, DSP board cluster designs typically use SRIO interfaces for interconnection, enabling efficient connection of multiple CPUs, DSPs, and FPGAs without relying on a host computer. When there are a large number of DSP boards, a VPX backplane (containing an SRIO switching module) is also required. Examples include existing VPX chassis data transmission methods and VPX chassis, or a high-speed data stream device and operating method based on the SRIO bus, or a hardware-software integrated parallel reconfigurable real-time image processing system, or a high-bandwidth, multi-channel multi-DSP computing blade device suitable for the VPX architecture. These existing methods are mature, widely used in embedded systems, have numerous application cases, are flexible and versatile, and have strong scalability. However, in application scenarios that emphasize master-slave architecture, centralized and intelligent heterogeneous SOC boards as image recognition acceleration boards, these traditional methods are not suitable solutions. Summary of the Invention

[0004] To address the problems existing in the above-mentioned traditional methods, this invention proposes a PCIE-based intelligent heterogeneous SOC inference acceleration system and a PCIE-based intelligent heterogeneous SOC inference acceleration method. It can be efficiently deployed in application scenarios that emphasize master-slave architecture, centralization, and use intelligent heterogeneous SOC boards as image recognition acceleration boards, while improving the efficiency of multi-core DSP, NPU and FFT collaborative work.

[0005] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions: On the one hand, a PCIE-based intelligent heterogeneous SOC inference acceleration system is provided, including an intelligent heterogeneous SOC board and a main control computer, wherein the intelligent heterogeneous SOC board is connected to the main control computer through a PCIE interface; The device driver for the intelligent heterogeneous SOC board adopts Linux device driver. All data in the intelligent heterogeneous SOC board flows in the DDR memory of the intelligent heterogeneous SOC board, and the input and output data of the intelligent heterogeneous SOC board are transferred by PCIe DMA. Among them, by utilizing the inbound function of the PCIe interface in the intelligent heterogeneous SOC board, the register address space of the FFT hardware accelerator, NPU hardware accelerator, CLC, PSC and Boot ROM of the intelligent heterogeneous SOC board is mapped to the PCIe BAR space. The host computer completes the scheduling of the FFT hardware accelerator, NPU hardware accelerator and multi-core DSP of the intelligent heterogeneous SOC board by operating the PCIe BAR space. The intelligent heterogeneous SOC board generates a PCIE MSI interrupt to the host computer; the host computer requests access to the PCIE interface of the intelligent heterogeneous SOC board through PCIE PIO. After parsing the request, the PCIE interface of the intelligent heterogeneous SOC board accesses the target components on the intelligent heterogeneous SOC board through the on-chip network; the target components include multi-core DSP, ARM core, FFT hardware accelerator, NPU, on-chip large-capacity shared memory SMC and off-chip large-capacity DDR.

[0006] In one embodiment, the PCIe BAR space definition in the intelligent heterogeneous SOC board includes: Base address register BAR0: It is 8MB in size and is used to map to the register address space of the FFT hardware accelerator, NPU hardware accelerator, CLC, PSC and Boot ROM of the smart heterogeneous SOC board; the size of each space used in the base address register BAR0 is 512KB, and the remaining unused space is reserved. Base address register BAR1: 256MB in size, used to map to the SMC Memory and DDR memory space of the intelligent heterogeneous SOC board; Base address register BAR1 serves as shared memory for data interaction between the main control computer and the multi-core DSP in the intelligent heterogeneous SOC board for mathematical calculations, as well as part of the memory used by the NPU for inference in the intelligent heterogeneous SOC board. Base address register BAR2: By default, it is mapped to the register space of the PCIe interface in the intelligent heterogeneous SOC board.

[0007] In one embodiment, the PCIe DMA of the smart heterogeneous SOC board is operated by the device driver of the smart heterogeneous SOC board on the host computer.

[0008] In one embodiment, the intelligent heterogeneous SOC board generates a PCIE MSI interrupt to the host computer CPU by writing to the PCIE MSI interrupt register of the intelligent heterogeneous SOC board.

[0009] In one embodiment, the PCIe MSI interrupt of the intelligent heterogeneous SOC board supports 32 interrupt vectors.

[0010] In one embodiment, after the ARM core in the smart heterogeneous SOC board completes the initialization of board-level hardware, DDR and PCIe in the boot program, it obtains the embedded operating system boot image from the storage device, loads it into the DDR memory, and boots into the embedded operating system. The ARM core registers interrupt handling functions for the FFT hardware accelerator and the NPU hardware accelerator in the kernel of the embedded operating system. After receiving interrupts from the FFT hardware accelerator and the NPU hardware accelerator, the ARM core generates a corresponding PCIE MSI interrupt to the host computer by writing the defined MSI interrupt vector number into the PCIE MSI interrupt register of the intelligent heterogeneous SOC board in the corresponding interrupt handling function.

[0011] In one embodiment, the multi-core DSP in the intelligent heterogeneous SOC board generates PCIE MSI interrupt 6 to the host computer by writing to the PCIE MSI interrupt register of the intelligent heterogeneous SOC board. The definitions of interrupt type messages include: bit-0 indicates a DSP handshake interrupt; bit-1 indicates a normal interruption after the DSP calculation API request is completed; bit-2 represents a DSP exception interrupt.

[0012] In one embodiment, the host computer writes to the IPC interrupt generation register of the intelligent heterogeneous SOC board by operating the PCIE BAR space, thereby generating IPC interrupts for the ARM core and multi-core DSP of the intelligent heterogeneous SOC board; the IPC interrupt generation register provides an interrupt source ID to identify different interrupt sources.

[0013] In one embodiment, the aforementioned PCIE-based intelligent heterogeneous SOC inference acceleration system further includes a PCIE switch board, with multiple intelligent heterogeneous SOC boards connected to the host computer via the PCIE switch board.

[0014] On the other hand, a PCIE-based intelligent heterogeneous SOC inference acceleration method is also provided. Based on the aforementioned PCIE-based intelligent heterogeneous SOC inference acceleration system, the PCIE-based intelligent heterogeneous SOC inference acceleration method includes the following steps: The main control computer operates the PCIE DMA of the intelligent heterogeneous SOC board to copy the input data to the planned DDR memory space of the intelligent heterogeneous SOC board. After the main control computer constructs the FFT computation context, it starts the FFT hardware accelerator to perform frequency domain calculations. While the main control computer waits for the FFT hardware accelerator to complete its calculations, it constructs the multi-core DSP computing context of the intelligent heterogeneous SOC board. After determining that the FFT hardware accelerator has completed the calculation normally, the main control computer generates an IPC interrupt to the multi-core DSP of the intelligent heterogeneous SOC board. While the main control computer waits for the multi-core DSP to complete its calculations, it constructs the NPU inference context of the intelligent heterogeneous SOC board. After receiving the PCIE MSI interrupt from the multi-core DSP of the intelligent heterogeneous SOC board, the main control computer determines that the multi-core DSP has completed its calculation and then starts the NPU inference. While the main control computer waits for the NPU inference to complete, it constructs the multi-core DSP computing context of the intelligent heterogeneous SOC board. After determining that the NPU inference is complete, the main control computer generates an IPC interrupt to the multi-core DSP of the intelligent heterogeneous SOC board; The main control computer waits for the multi-core DSP of the intelligent heterogeneous SOC board to complete the calculation; After the host computer receives the PCIE MSI interrupt sent by the multi-core DSP of the intelligent heterogeneous SOC board, it determines that the multi-core DSP has completed the calculation. Then, the host computer operates the PCIE DMA of the intelligent heterogeneous SOC board to copy the output data to the host computer's memory.

[0015] One of the above technical solutions has the following advantages and beneficial effects: The aforementioned PCIe-based intelligent heterogeneous SOC inference acceleration system and method designs the intelligent heterogeneous SOC as a PCIe device acceleration card, connecting it to the host computer via a PCIe interface. Utilizing the PCIe inbound function of the intelligent heterogeneous SOC card, its FFT, NPU, CLC, SMC, and DDR memory are mapped to the PCIe BAR space. An interrupt interaction mechanism between the host computer and the intelligent heterogeneous SOC card is designed, operated by the host computer. This mechanism schedules the NPU, FFT, and multi-core DSP of the intelligent heterogeneous SOC card, enabling efficient data transfer using the PCIe DMA function of the intelligent heterogeneous SOC card. In application scenarios emphasizing master-slave architecture, centralization, and using the intelligent heterogeneous SOC card as an image recognition acceleration card, this system achieves efficient deployment while improving the collaborative efficiency of the multi-core DSP, NPU, and FFT. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the architecture of a PCIE-based intelligent heterogeneous SOC inference acceleration system in one embodiment. Figure 2 This is a system block diagram of a single embodiment where intelligent heterogeneous SOC boards are clustered and connected to the main control computer via a PCIe interface. Figure 3 This is a block diagram of the overall structure of a smart heterogeneous SOC board connected to the main control computer CPU via a PCIe interface in one embodiment. Figure 4 This is a schematic diagram of the specific address space arrangement of one type of BAR0 in one embodiment; Figure 5 This is a schematic diagram of the specific address space arrangement of one type of BAR1 in one embodiment; Figure 6 This is a schematic diagram of the operation process of the ARM core in a smart heterogeneous SOC board from power-on, as shown in one embodiment. Figure 7 This is a flowchart illustrating the collaborative operation of FFT, multi-core DSP, and NPU on a smart heterogeneous SOC board in one embodiment. Figure 8 This is a flowchart illustrating a PCIE-based intelligent heterogeneous SOC inference acceleration method in one embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0019] It should be noted that, in this document, the reference to "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The presentation of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will understand that the embodiments described herein can be combined with other embodiments. The term "and / or" as used herein refers to any combination of one or more of the associated listed items, and all possible combinations, including such combinations.

[0020] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0021] SRIO interfaces are primarily used in embedded systems, emphasizing peer-to-peer interconnection between multiple DSP / FPGA boards. Their application scope is relatively small, and a unified software specification standard has not been established. Therefore, deploying DSP clusters using SRIO results in high hardware costs and complex development. SRIO technology also requires significant software, including drivers and network protocol stacks, whose development and maintenance costs are also substantial. Furthermore, the lack of a defined standard software model for SRIO technology leads to a high probability of incompatibility between devices from different manufacturers, limiting its deployment to specific areas and hindering its widespread adoption.

[0022] Before formally introducing the technical solution of this invention, it is necessary to point out the meanings of some technical terms used in the specification as follows: CPU stands for Central Processing Unit, DSP for Digital Signal Processor, FFT for Fast Fourier Transform, NPU for Neural Processing Unit, DMA for Direct Memory Access, CLC for Global Control Register, PSC for Power / Sleep Controller, SMC for Shared Memory Controller, and PIO for Programmable Input / Output.

[0023] In one embodiment, such as Figure 1As shown, a PCIe-based intelligent heterogeneous SoC inference acceleration system is provided, including an intelligent heterogeneous SoC board and a host computer CPU. The intelligent heterogeneous SoC board is connected to the host computer CPU via a PCIe interface. The device driver for the intelligent heterogeneous SoC board adopts a Linux device driver. All data in the intelligent heterogeneous SoC board flows in the DDR memory of the intelligent heterogeneous SoC board, and the input and output data of the intelligent heterogeneous SoC board are transferred by PCIe DMA. Specifically, utilizing the inbound function of the PCIe interface in the intelligent heterogeneous SoC board, the register address spaces of the FFT hardware accelerator, NPU hardware accelerator, CLC, PSC, and Boot ROM of the intelligent heterogeneous SoC board are mapped to the PCIe BAR space. The host computer CPU completes the scheduling of the FFT hardware accelerator, NPU hardware accelerator, and multi-core DSP of the intelligent heterogeneous SoC board by operating the PCIe BAR space. The intelligent heterogeneous SOC board generates a PCIE MSI interrupt to the host computer CPU; the host computer CPU requests access to the PCIE interface of the intelligent heterogeneous SOC board through PCIE PIO. After parsing the request, the PCIE interface of the intelligent heterogeneous SOC board accesses the target components on the intelligent heterogeneous SOC board through the on-chip network; the target components include multi-core DSP, ARM core, FFT hardware accelerator, NPU, on-chip large-capacity shared memory SMC and off-chip large-capacity DDR.

[0024] In terms of hardware implementation, the intelligent heterogeneous SOC chip integrates a multi-core DSP, an ARM core, a Fast Fourier Transform (FFT) hardware accelerator, a Neural Processing Unit (NPU), on-chip large-capacity shared memory (SMC), and off-chip large-capacity DDR and PCIe interfaces, similar to existing heterogeneous SOCs. This intelligent heterogeneous SOC chip features the following characteristics: a high-performance DSP core supporting both fixed-point and floating-point operation, a clock frequency exceeding 1 GHz, and strong computational performance; a Neural Processing Unit (NPU) supporting mainstream deep learning algorithms such as CNN, RNN, and LSTM, and possessing strong peak performance; and a Fast Fourier Transform (FFT) hardware accelerator supporting single-precision floating-point or 32-bit fixed-point FFT operations, supporting FFT operations of different scales and data formats, and supporting 32-bit and 64-bit data matrix transposition functions. The integrated ARM core adopts the standard ARM-V7 architecture and has a complete related software ecosystem; it has a high-speed, high-bandwidth data / on-chip network (NOC) that provides high-speed, high-bandwidth concurrent data transmission between devices such as multi-core DSPs, ARM cores, neural network accelerators (NPUs), fast Fourier transforms (FFTs), PCIe interfaces, large-capacity external DDR memory, and large-capacity on-chip shared memory (SMC).

[0025] However, the deployment of intelligent heterogeneous SOC chip (including multi-core DSP, ARM, FFT, and NPU) cluster inference acceleration devices is complex and uses a single method. Currently, when deploying DSP board clusters, SRIO (Serial Rapid IO) interfaces are basically used to achieve interconnection between boards, and SRIO switches are used to connect to the main control computer system. This method has a complex hardware and software structure and is difficult to maintain. The maximum SRIO speed currently supported is GEN3. The SRIO speeds (single lane, line speed) of each generation include: Gen1: 1.25Gbaud, 2.5Gbaud, or 3.125Gbaud (3.125Gbaud is commonly used), Gen2: 5.0 or 6.25Gbaud (5Gbaud and 6.25Gbaud are commonly used), Gen3: 10.3125Gbaud (currently the highest mainstream speed), and Gen4: 12.5Gbaud or 25Gbaud (standard exists, but very few chips / IPs exist).

[0026] In application scenarios emphasizing master-slave architecture, centralized and heterogeneous SOC boards as image recognition and inference accelerator cards, to reduce the difficulty of software development, deployment, maintenance, and upgrades, this embodiment designs the intelligent heterogeneous SOC as a PCIe device accelerator card based on the standard PCIe interface, i.e., an intelligent heterogeneous SOC board, which can be connected to the host computer system through a PCIe slot on the host computer. The driver software conforms to the Linux device driver specification, has a complete software ecosystem, good compatibility, and standardized software models and drivers. It also has a complete AI inference and edge computing ecosystem, making system integration and development more convenient. Specifically, the Linux device driver has a single source code that drives multiple identical PCIe devices; under the Linux operating system, it has a complete software model related to AI edge computing and image recognition inference, and a complete software ecosystem, facilitating rapid deployment.

[0027] In intelligent heterogeneous SOC chips, DSPs, NPUs, and FFTs work collaboratively, each performing its own function, or they can operate independently, with the main control computer's CPU flexibly scheduling according to the application scenario. When multi-core DSPs, NPUs, and FFTs work together, a pipelined operation of FFT->DSP->NPU->DSP can be achieved, with all data flowing within the DDR memory of the intelligent heterogeneous SOC board, improving performance. Input and output data of the intelligent heterogeneous SOC board are transferred via PCIe DMA, improving data transmission efficiency. In the intelligent heterogeneous SOC chip, multi-core DSPs perform pre- and post-processing for image recognition and digital signal processing calculations, while the NPU performs neural network inference, and the FFT performs frequency domain calculations.

[0028] The block diagram of the interconnection system between the intelligent heterogeneous SOC board and the CPU of the main control computer is as follows: Figure 1 As shown, in a host computer with PCIe slots, the intelligent heterogeneous SOC board can be directly connected to a PCIe slot, while other PCIe slots connect to other PCI devices, such as, but not limited to, graphics cards and 10 Gigabit Ethernet cards. The PCIe Root Complex (PCIe RC) is the core hub in the PCIe bus architecture. Currently, the maximum PCIe interface link supported by intelligent heterogeneous SOC chips is x4.

[0029] The aforementioned PCIe-based intelligent heterogeneous SOC inference acceleration system designs the intelligent heterogeneous SOC as a PCIe device accelerator card, connecting it to the host computer via a PCIe interface. Utilizing the PCIe inbound function of the intelligent heterogeneous SOC card, it maps the FFT, NPU, CLC, SMC, and DDR memory of the intelligent heterogeneous SOC card to the PCIe BAR space. An interrupt interaction mechanism between the host computer and the intelligent heterogeneous SOC card is designed, operated by the host computer to schedule the NPU, FFT, and multi-core DSP of the intelligent heterogeneous SOC card. This allows for efficient data transfer using the PCIe DMA function of the intelligent heterogeneous SOC card. In application scenarios emphasizing a master-slave architecture, centralization, and using the intelligent heterogeneous SOC card as an image recognition accelerator, this system achieves efficient deployment while improving the collaborative efficiency of the multi-core DSP, NPU, and FFT.

[0030] The intelligent heterogeneous SOC is designed as a PCIe device accelerator card, supporting plug-and-play functionality and facilitating hardware expansion, upgrades, and maintenance. Within the intelligent heterogeneous SOC board, the ARM core and multi-core DSP each perform their specific functions, eliminating the need for complex control and scheduling logic. Scheduling and control authority rests with the host computer. The ARM core handles board-level hardware initialization, DDR, and PCIe initialization. After initialization, it enters the embedded operating system, registering interrupt handlers for components such as the NPU and FFT. These interrupt handlers generate corresponding PCIe MSI interrupts and send them to the host computer. The multi-core DSP performs pure digital signal processing calculations, focusing on computation to fully leverage its floating-point computing advantages. The NPU, FFT, and other hardware accelerators, as well as the multi-core DSP, are entirely scheduled by the host computer, better maximizing the overall performance of the heterogeneous SOC. The collaborative work of the various heterogeneous cores further enhances the utilization and value of the intelligent heterogeneous SOC.

[0031] In one embodiment, the PCIE-based intelligent heterogeneous SOC inference acceleration system further includes a PCIE switch board, with multiple intelligent heterogeneous SOC boards connected to the main control computer via the PCIE switch board.

[0032] It is understandable that if the application scenario requires multiple intelligent heterogeneous SOC boards to be clustered, then the main control computer CPU can be expanded with PCIe Switch boards to interconnect multiple intelligent heterogeneous SOC boards and realize the rapid deployment of intelligent heterogeneous SOC board cluster hardware.

[0033] Optionally, the system block diagram of intelligent heterogeneous SOC boards being clustered and connected to the main control computer via PCIe interfaces is as follows: Figure 2 As shown, Figure 1 and Figure 2 The memory in the main control computer's CPU is the main memory. The intelligent heterogeneous SOC board connects to slots led out from the PCIe switch board. Other PCIe slots connect to other PCI devices, such as, but not limited to, graphics cards and 10 Gigabit Ethernet cards. The intelligent heterogeneous SOC chip's PCIe interface links support a maximum of x4. In a system where intelligent heterogeneous SOC boards are clustered via PCIe, the number of intelligent heterogeneous SOC boards is not limited to four; the number can be increased or decreased according to the actual application scenario and project requirements.

[0034] More specifically, the overall structure diagram of the intelligent heterogeneous SOC board connected to the host computer CPU via the PCIe interface is as follows: Figure 3 As shown, the intelligent heterogeneous SOC board actively generates a PCIE MSI interrupt to the host computer CPU. The host computer CPU accesses the PCIE interface of the intelligent heterogeneous SOC board through a PCIE PIO request. After parsing the relevant request, the PCIE interface of the intelligent heterogeneous SOC board accesses other target components on the intelligent heterogeneous SOC board through the on-chip network (NOC), such as multi-core DSP, ARM core, Fast Fourier Transform (FFT) hardware accelerator, Neural Processing Unit (NPU), on-chip large-capacity shared memory (SMC), and off-chip large-capacity DDR. The DMA (i.e., PCIE DMA) of the PCIE interface of the intelligent heterogeneous SOC board is used for efficient data transfer between the host computer and the memory of the intelligent heterogeneous SOC board.

[0035] In terms of software implementation, the device driver for the intelligent heterogeneous SOC board adopts the existing Linux device driver. The Linux kernel is widely used, found in desktop computers and servers, and has a complete ecosystem. Following the Linux device driver standard, all code interacting with the underlying hardware of the intelligent heterogeneous SOC board is implemented in the device driver. The application layer only needs to focus on interface calls, without needing to understand the detailed hardware implementation, thus reducing the difficulty of software development.

[0036] The basic functions related to the underlying hardware in the device driver for intelligent heterogeneous SOC boards are described below: 1. Register Mapping: By configuring the PCIe interface's existing inbound function in the intelligent heterogeneous SOC board, the register address spaces of the core components such as FFT and NPU, as well as the registers of CLC, PSC, and Boot ROM (diskless boot ROM interface), are mapped to the PCIe BAR (Base Address Register) space. The host computer CPU directly operates on the corresponding offset addresses in the PCIe BAR space to perform functions such as operating hardware accelerators like FFT and NPU, and calling multi-core DSPs for calculations. The core of the PCIe inbound function is Address Translation (ATU): translating the PCIe bus domain address into a physical address in the memory domain (CPU / system memory), allowing downstream devices to actively access host memory (DMA). Inbound is the "entry translator" of PCIe, responsible for translating the device's bus address into a physical address that the host can understand, and is the foundation for key functions such as DMA and P2P.

[0037] 2. Data Interaction: (1) Through the DMA function of the PCIE interface in the intelligent heterogeneous SOC board, efficient data transfer between the host computer and the memory of the intelligent heterogeneous SOC board is realized. (2) Configure the PCIE inbound related registers of the intelligent heterogeneous SOC board to map the SMC and memory space of the intelligent heterogeneous SOC board to the PCIE BAR space. The host computer CPU realizes PIO read and write of the SMC and memory space of the intelligent heterogeneous SOC board by operating the PCIE BAR space.

[0038] 3. Interrupts: The intelligent heterogeneous SOC board generates an MSI interrupt to the host computer CPU via the PCIE MSI interrupt; the host computer CPU writes to the IPC interrupt register of the intelligent heterogeneous SOC board by operating the PCIE BAR space, thereby generating IPC interrupts to the ARM core and multi-core DSP.

[0039] In one embodiment, the register mapping technique can be implemented as follows: the PCIe BAR space definition in the intelligent heterogeneous SOC board includes: Base address register BAR0: 8MB in size, mapped to the register address spaces of the NPU, FFT, CLC, PSC, and Boot ROM of the intelligent heterogeneous SOC board via the inbound function of the high-speed PCIe interface. Each space is 512KB in size, and the remaining unused space is reserved. This space can be, but is not limited to, used for... Figure 4 The layout can be modified appropriately according to the application scenario requirements. One example of a specific address space arrangement for PCIe BAR0 is... Figure 4 As shown.

[0040] Base Address Register (BAR1): 256MB in size, it is mapped to the SMC Memory (8MB) and DDR memory space of the intelligent heterogeneous SOC board via the PCIe inbound function. This serves as shared memory for data exchange between the main computer CPU and the multi-core DSP in the intelligent heterogeneous SOC board for mathematical calculations, as well as a portion of the memory used for NPU inference in the intelligent heterogeneous SOC board. One specific address space arrangement of PCIe BAR1 is as follows: Figure 5 As shown.

[0041] Base address register BAR2: It is already mapped to the register space of the PCIe interface in the intelligent heterogeneous SOC board by default, and no further mapping is required.

[0042] In one embodiment, the specific technical implementation of data interaction can be as follows: Direct data transfer between the host computer CPU and the memory of the intelligent heterogeneous SOC board is achieved through the DMA function of the PCIe interface in the intelligent heterogeneous SOC board. This DMA supports common full-duplex operations and can simultaneously handle DMA writes, DMA reads, and non-DMA transactions; it can support four read and four write channels each; it supports register-based DMA and linked-list-based DMA, enabling fast and efficient transfer of large amounts of DDR memory data between the host computer and the intelligent heterogeneous SOC board.

[0043] In the intelligent heterogeneous SOC board, the DMA of the PCIe interface is operated by the device driver of the intelligent heterogeneous SOC board under the Linux of the host computer. The ARM and DSP in the intelligent heterogeneous SOC board do not operate the DMA of this PCIe, so as to reduce the overall design difficulty of the software.

[0044] By utilizing the inbound function of PCIe in the intelligent heterogeneous SOC board, PCIe BAR1 is mapped to the SMC and DDR memory space of the intelligent heterogeneous SOC board. The size of BAR1 can be set to 256MB. Figure 5 As shown, the first 8MB is the SMC of the intelligent heterogeneous SOC board, which is used as shared memory for data interaction between the main control computer CPU and the multi-core DSP in the intelligent heterogeneous SOC board for mathematical calculations; the remaining 248MB is the DDR memory (0x80000000 to 0x8F800000) space of the intelligent heterogeneous SOC board, which is used as part of the memory used by the NPU inference in the intelligent heterogeneous SOC board.

[0045] For efficient computation by the multi-core DSP of the intelligent heterogeneous SOC board, the multi-core DSP needs to use contiguous physical memory when acquiring input data and writing result data. When the host computer writes input data to the DDR memory space of the intelligent heterogeneous SOC board through the PCIe DMA, the memory addresses must be contiguous.

[0046] In one embodiment, the specific technical implementation of the interrupt can be as follows: the interrupt implementation includes the main control computer CPU generating an interrupt to the intelligent heterogeneous SOC board and the intelligent heterogeneous SOC board generating an interrupt to the main control computer CPU.

[0047] The scheme for intelligent heterogeneous SoC boards to generate interrupts to the host computer CPU is described as follows: The intelligent heterogeneous SoC board generates MSI interrupts to the host computer CPU via PCIe MSI interrupts. The PCIe MSI interrupts of the intelligent heterogeneous SoC board support 32 interrupt vectors, capable of describing 32 independent interrupt sources, supporting interrupt load balancing, and fully leveraging the multi-core parallel interrupt processing capabilities. The intelligent heterogeneous SoC board generates MSI interrupts to the host computer CPU by writing to its PCIe MSI interrupt register.

[0048] The 32 interrupt vectors are defined as shown in Table 1: Table 1

[0049] The implementation scheme for generating MSI interrupts by the ARM core in the intelligent heterogeneous SOC board is described as follows: After the ARM core in the intelligent heterogeneous SOC board completes the initialization of board-level hardware, DDR, and PCIe in the bootloader (such as U-BOOT), it obtains the existing embedded operating system boot image from the storage device (such as EMIF NOR / NAND flash or SPI flash), loads it into DDR memory, and boots into the embedded operating system. In the kernel of the embedded operating system, interrupt handling functions for components such as NPU and FFT are registered. After the ARM core receives interrupts from components such as NPU and FFT, the corresponding interrupt handling function generates a corresponding PCIe MSI interrupt by writing the defined MSI interrupt vector number into the PCIe MSI interrupt register of the intelligent heterogeneous SOC board. The running process of the ARM core in the intelligent heterogeneous SOC board from power-on is as follows: Figure 6 As shown.

[0050] The implementation scheme for generating MSI interrupts in the DSP core of the intelligent heterogeneous SOC board is described as follows: The multi-core DSP in the intelligent heterogeneous SOC board generates PCIe MSI interrupt 6 by writing to the PCIe MSI interrupt register of the intelligent heterogeneous SOC board. Only one MSI interrupt vector is used, and the interrupt type message is stored in a specific area of ​​the SMC. The definition of the interrupt type message can include: bit-0 represents DSP handshake interrupt, bit-1 represents normal interrupt for DSP calculation API request completion, and bit-2 represents DSP abnormal interrupt (program crash, etc.).

[0051] The scheme for the main control computer CPU to generate interrupts to the intelligent heterogeneous SOC board is described below: The main control computer CPU writes to the IPC interrupt generation register (configured in the CLC register) of the intelligent heterogeneous SOC board by manipulating the PCIe BAR space, thereby generating IPC interrupts for the ARM and DSP. The IPC interrupt generation register provides interrupt source IDs to identify different interrupt sources, and can provide 28 interrupt sources. The 28 interrupt sources for generating IPC interrupts for the multi-core DSP can be defined as shown in Table 2: Table 2

[0052] The process of collaborative operation of FFT, multi-core DSP and NPU in intelligent heterogeneous SOC board is as follows: Figure 7 As shown: (1) The main control computer CPU operates the PCIE DMA of the intelligent heterogeneous SOC board to copy the input data to the planned DDR memory space of the intelligent heterogeneous SOC board.

[0053] (2) After the main control computer CPU constructs the FFT calculation context, it operates the FFT register space of the intelligent heterogeneous SOC board and starts the FFT to perform frequency domain calculations, such as image filtering and image enhancement. It should be noted that the construction of the FFT calculation context in step (2) can be carried out in parallel with step (1), so as to give full play to the parallel performance of the main control computer.

[0054] (3) While the main control computer CPU waits for the FFT calculation to be completed, it constructs the multi-core DSP computing context of the intelligent heterogeneous SOC board, such as the commonly used context for quantization and preprocessing.

[0055] (4) After receiving the FFT transaction interrupt sent by the ARM core of the intelligent heterogeneous SOC board, the main control computer CPU reads the FFT-related status register, determines that the FFT calculation is completed normally, generates an IPC interrupt to the multi-core DSP of the intelligent heterogeneous SOC board, and requests the multi-core DSP to perform the calculation. If the FFT calculation is abnormal, an exception is thrown and the process ends.

[0056] (5) While the main control computer CPU waits for the multi-core DSP to complete the calculation, it constructs the NPU inference context of the intelligent heterogeneous SOC board.

[0057] (6) After receiving the PCIE MSI interrupt sent by the multi-core DSP of the intelligent heterogeneous SOC board, the main control computer CPU reads the corresponding location of the SMC memory space mapped by the BAR1 space through PIO, determines that the multi-core DSP has completed the calculation, and starts NPU inference; if the multi-core DSP is abnormal, it throws an exception and ends the process.

[0058] (7) While the main control computer CPU waits for the NPU inference to complete, it constructs the multi-core DSP computing context of the intelligent heterogeneous SOC board, such as the commonly used context for inverse quantization and post-processing.

[0059] (8) After receiving the NPU transaction interrupt sent by the ARM core of the intelligent heterogeneous SOC board, the main control computer CPU reads the relevant status register of the NPU. After determining that the NPU has completed the inference normally, it generates an IPC interrupt to the multi-core DSP of the intelligent heterogeneous SOC board and requests the multi-core DSP to perform calculation. If the NPU inference is abnormal, an exception is thrown and the process ends.

[0060] (9) The main control computer CPU waits for the multi-core DSP of the intelligent heterogeneous SOC board to complete the calculation.

[0061] (10) After the main control computer CPU receives the PCIE MSI interrupt sent by the multi-core DSP of the intelligent heterogeneous SOC board, it reads the corresponding location of the SMC memory space mapped by the BAR1 space through PIO, determines that the multi-core DSP has completed the calculation, and then proceeds to the next step; if the multi-core DSP is abnormal, it throws an exception and ends the process.

[0062] (11) The main control computer CPU operates the PCIE DMA of the intelligent heterogeneous SOC board, copies the output data to the host memory of the main control computer CPU, and displays the results of image target recognition, etc.

[0063] exist Figure 7 In the process of FFT, multi-core DSP and NPU working together in the intelligent heterogeneous SOC board shown, the original input data is copied to the DDR memory space of the intelligent heterogeneous SOC board through PCIe DMA. After being processed by FFT, multi-core DSP and NPU, the data generated in the process flows in the DDR memory space of the intelligent heterogeneous SOC board, eliminating many intermediate transfer links in the past, improving the data transfer performance and efficiency, and thus improving the overall processing performance.

[0064] It should be noted that the FFT computation context refers to a series of factors that typically need to be considered in FFT computation, such as, but not limited to, signal type, sampling rate, signal length, window function, and zero-padding. The DSP computation context refers to the environment and conditions in which DSP computations or operations are performed, typically including hardware platform, software framework, data processing flow, and specific implementation details. The NPU inference context includes common information such as hardware architecture (e.g., dedicated matrix multiplication units, memory management units, and parallel processing capabilities), inference optimizations (e.g., model compression, graph optimization, and caching strategies), and software tools (e.g., model transformation tools and inference engines) involved in NPU inference. This context information is existing in each computation type and has not been modified in this specification.

[0065] Compared to traditional technologies, the above design, through PCIe data transmission and intelligent heterogeneous SOC board-level high-speed interconnection and cluster deployment technology, enables faster and more efficient deployment of intelligent heterogeneous SOC boards in applications such as image recognition and edge AI computing. It utilizes the multi-core DSP within the intelligent heterogeneous SOC board for image pre- and post-processing, quantization, and inverse quantization; the ARM core within the intelligent heterogeneous SOC board for board-level hardware, DDR, and PCIe initialization, as well as simple interrupt communication with the host computer CPU; the NPU hardware acceleration core within the intelligent heterogeneous SOC board for neural network inference; and the FFT hardware acceleration core within the intelligent heterogeneous SOC board for edge computing, image frequency domain filtering, and feature extraction. This effectively solves the practical problem of difficult cluster deployment of intelligent heterogeneous SOCs in image processing and edge AI computing applications.

[0066] In one embodiment, such as Figure 8 As shown, a PCIe-based intelligent heterogeneous SOC inference acceleration method is also provided. Based on the above-mentioned PCIe-based intelligent heterogeneous SOC inference acceleration system, the PCIe-based intelligent heterogeneous SOC inference acceleration method includes the following steps: S10, the main control computer operates the PCIE DMA of the intelligent heterogeneous SOC board to copy the input data to the planned DDR memory space of the intelligent heterogeneous SOC board; S11, After the main control computer constructs the FFT computation context, it starts the FFT hardware accelerator to perform frequency domain computation; S12, while the main control computer waits for the FFT hardware accelerator to complete its calculations, it builds the multi-core DSP computing context of the intelligent heterogeneous SOC board. S13, after the main control computer determines that the FFT hardware accelerator has completed the calculation normally, it generates an IPC interrupt to the multi-core DSP of the intelligent heterogeneous SOC board. S14, while the main control computer waits for the multi-core DSP to complete the calculation, it builds the NPU inference context of the intelligent heterogeneous SOC board; S15: After receiving the PCIE MSI interrupt sent by the multi-core DSP of the intelligent heterogeneous SOC board, the main control computer determines that the multi-core DSP has completed the calculation and then starts the NPU inference. S16, while the main control computer waits for the NPU inference to complete, it constructs the multi-core DSP computing context of the intelligent heterogeneous SOC board; S17, the main control computer generates an IPC interrupt to the multi-core DSP of the intelligent heterogeneous SOC board after determining that the NPU inference is completed; S18, the main control computer waits for the multi-core DSP of the intelligent heterogeneous SOC board to complete the calculation; S19: After the main control computer receives the PCIE MSI interrupt sent by the multi-core DSP of the intelligent heterogeneous SOC board, it determines that the multi-core DSP has completed the calculation. Then, the main control computer operates the PCIE DMA of the intelligent heterogeneous SOC board to copy the output data to the host memory of the main control computer.

[0067] It is understood that the explanations of the various feature terms in the above-mentioned PCIE-based intelligent heterogeneous SOC inference acceleration method can be understood by referring to the corresponding feature explanations in the various embodiments of the above-mentioned PCIE-based intelligent heterogeneous SOC inference acceleration system, and will not be repeated here.

[0068] The aforementioned PCIe-based intelligent heterogeneous SoC inference acceleration method designs the intelligent heterogeneous SoC as a PCIe device acceleration card, connecting it to the host computer via a PCIe interface. Utilizing the PCIe inbound function of the intelligent heterogeneous SoC card, it maps the FFT, NPU, CLC, SMC, and DDR memory of the intelligent heterogeneous SoC card to the PCIe BAR space. An interrupt interaction mechanism between the host computer and the intelligent heterogeneous SoC card is designed, operated by the host computer to schedule the NPU, FFT, and multi-core DSP of the intelligent heterogeneous SoC card. This allows for efficient data transfer using the PCIe DMA function of the intelligent heterogeneous SoC card. In application scenarios emphasizing master-slave architecture, centralization, and using the intelligent heterogeneous SoC card as an image recognition acceleration card, this method achieves efficient deployment while improving the collaborative efficiency of the multi-core DSP, NPU, and FFT.

[0069] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0070] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of protection of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and all such modifications and improvements fall within the scope of protection of the present invention.

Claims

1. A PCIE-based intelligent heterogeneous SOC inference acceleration system, characterized in that, It includes a smart heterogeneous SOC board and a main control computer. The smart heterogeneous SOC board is connected to the main control computer through a PCIe interface. The device driver for the intelligent heterogeneous SOC board adopts Linux device driver. All data in the intelligent heterogeneous SOC board flows in the DDR memory of the intelligent heterogeneous SOC board, and the input and output data of the intelligent heterogeneous SOC board are transferred by PCIe DMA. Among them, by utilizing the inbound function of the PCIe interface in the intelligent heterogeneous SOC board, the register address space of the FFT hardware accelerator, NPU hardware accelerator, CLC, PSC and Boot ROM of the intelligent heterogeneous SOC board is mapped to the PCIe BAR space. The host computer completes the scheduling of the FFT hardware accelerator, NPU hardware accelerator and multi-core DSP of the intelligent heterogeneous SOC board by operating the PCIe BAR space. The intelligent heterogeneous SOC board generates a PCIE MSI interrupt to the host computer; the host computer requests access to the PCIE interface of the intelligent heterogeneous SOC board through PCIE PIO. After parsing the request, the PCIE interface of the intelligent heterogeneous SOC board accesses the target components on the intelligent heterogeneous SOC board through the on-chip network; the target components include multi-core DSP, ARM core, FFT hardware accelerator, NPU, on-chip large-capacity shared memory SMC and off-chip large-capacity DDR. The PCIe BAR space definition in intelligent heterogeneous SOC boards includes: Base address register BAR0: It is 8MB in size and is used to map to the register address space of the FFT hardware accelerator, NPU hardware accelerator, CLC, PSC and Boot ROM of the smart heterogeneous SOC board; the size of each space used in the base address register BAR0 is 512KB, and the remaining unused space is reserved. Base address register BAR1: 256MB in size, used to map to the SMC Memory and DDR memory space of the intelligent heterogeneous SOC board; Base address register BAR1 serves as shared memory for data interaction between the main control computer and the multi-core DSP in the intelligent heterogeneous SOC board for mathematical calculations, as well as part of the memory used by the NPU for inference in the intelligent heterogeneous SOC board. Base address register BAR2: By default, it is mapped to the register space of the PCIe interface in the intelligent heterogeneous SOC board; In the intelligent heterogeneous SOC board, after the ARM core completes the initialization of board-level hardware, DDR and PCIe in the boot program, it obtains the embedded operating system boot image from the storage device, loads it into DDR memory, and boots into the embedded operating system. The ARM core registers interrupt handling functions for the FFT hardware accelerator and the NPU hardware accelerator in the kernel of the embedded operating system. After receiving interrupts from the FFT hardware accelerator and the NPU hardware accelerator, the ARM core generates a corresponding PCIE MSI interrupt to the host computer by writing the defined MSI interrupt vector number into the PCIE MSI interrupt register of the intelligent heterogeneous SOC board in the corresponding interrupt handling function.

2. The PCIE-based intelligent heterogeneous SOC inference acceleration system according to claim 1, characterized in that, The PCIe DMA of the intelligent heterogeneous SOC board is operated by the device driver of the intelligent heterogeneous SOC board on the host computer.

3. The PCIE-based intelligent heterogeneous SOC inference acceleration system according to claim 1, characterized in that, The intelligent heterogeneous SOC board generates a PCIE MSI interrupt to the host computer CPU by writing to the PCIE MSI interrupt register of the intelligent heterogeneous SOC board.

4. The PCIE-based intelligent heterogeneous SOC inference acceleration system according to claim 3, characterized in that, The PCIe MSI interrupt of the intelligent heterogeneous SOC board supports 32 interrupt vectors.

5. The PCIE-based intelligent heterogeneous SOC inference acceleration system according to claim 3, characterized in that, The multi-core DSP in the intelligent heterogeneous SOC board generates PCIE MSI interrupt 6 to the host computer by writing to the PCIE MSI interrupt register of the intelligent heterogeneous SOC board. The definitions of interrupt type messages include: bit-0 indicates a DSP handshake interrupt; bit-1 indicates a normal interruption after the DSP calculation API request is completed; bit-2 represents a DSP exception interrupt.

6. The PCIE-based intelligent heterogeneous SOC inference acceleration system according to claim 1, characterized in that, The main control computer writes to the IPC interrupt generation register of the intelligent heterogeneous SOC board by operating the PCIE BAR space, thereby generating IPC interrupts for the ARM core and multi-core DSP of the intelligent heterogeneous SOC board. The IPC interrupt generation register provides an interrupt source ID to identify different interrupt sources.

7. The PCIE-based intelligent heterogeneous SOC inference acceleration system according to claim 1, characterized in that, It also includes a PCIe Switch board, with multiple intelligent heterogeneous SOC boards connected to the main control computer via the PCIe Switch board.

8. A method for accelerating inference in intelligent heterogeneous SOCs based on PCIe, characterized in that, Based on the PCIE-based intelligent heterogeneous SOC inference acceleration system according to any one of claims 1 to 7, the PCIE-based intelligent heterogeneous SOC inference acceleration method includes the following steps: The main control computer operates the PCIE DMA of the intelligent heterogeneous SOC board to copy the input data to the planned DDR memory space of the intelligent heterogeneous SOC board. After the main control computer constructs the FFT computation context, it starts the FFT hardware accelerator to perform frequency domain calculations. While the main control computer waits for the FFT hardware accelerator to complete its calculations, it constructs the multi-core DSP computing context of the intelligent heterogeneous SOC board. After determining that the FFT hardware accelerator has completed the calculation normally, the main control computer generates an IPC interrupt to the multi-core DSP of the intelligent heterogeneous SOC board. While the main control computer waits for the multi-core DSP to complete its calculations, it constructs the NPU inference context of the intelligent heterogeneous SOC board. After receiving the PCIE MSI interrupt from the multi-core DSP of the intelligent heterogeneous SOC board, the main control computer determines that the multi-core DSP has completed its calculation and then starts the NPU inference. While the main control computer waits for the NPU inference to complete, it constructs the multi-core DSP computing context of the intelligent heterogeneous SOC board. After determining that the NPU inference is complete, the main control computer generates an IPC interrupt to the multi-core DSP of the intelligent heterogeneous SOC board; The main control computer waits for the multi-core DSP of the intelligent heterogeneous SOC board to complete the calculation; After the host computer receives the PCIE MSI interrupt sent by the multi-core DSP of the intelligent heterogeneous SOC board, it determines that the multi-core DSP has completed the calculation. Then, the host computer operates the PCIE DMA of the intelligent heterogeneous SOC board to copy the output data to the host computer's memory.

Images