High-density integrated computer hardware computing power expansion module

Abstract

The application discloses a high-density integrated computer hardware computing power expansion module and belongs to the technical field of heterogeneous computing and computing power virtualization. The module comprises a bus adaptation unit, an instruction description template library, an operation code matching and analysis unit, a segmented address mapping unit and a task descriptor assembly and distribution unit. In view of the problem that an existing computing power scheduling scheme can only distribute tasks to nodes and cannot uniformly drive different instruction set architecture accelerators in the nodes at a standard bus level, a hardware conversion layer is constructed between a bus transaction layer and an accelerator local command interface, a unified format task control block issued by a host processor is dynamically translated into native instructions and local physical addresses of different accelerators, GPU, NPU, FPGA and other instruction sets incompatible computing power units can be centrally controlled and driven in a unified bus device semantic, so that high-density integration of heterogeneous computing power and system-level elastic expansion are realized.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a high-density integrated computer hardware computing power expansion module. Background Technology

[0002] With the rapid increase in the scale of artificial intelligence model parameters, a single computer system often needs to be configured with multiple types of computing accelerators, such as GPUs, NPUs, and FPGAs. These accelerators come from different manufacturers and have different instruction set architectures, memory addressing methods, and command queue interface formats, posing a great challenge to system integration. In the prior art, Chinese invention patent application CN202410551211.8 discloses a method and system for scheduling computing resources; it achieves preliminary integration of scattered heterogeneous resources by constructing a unified computing resource pool and performing node-level matching and distribution of computing tasks. However, this approach has significant technical flaws: once a task is assigned to a node, the main processor still faces the challenge of directly driving accelerators with multiple instruction set architectures within the node using a unified device control semantics. Under the CN202410551211.8 architecture, this low-level driving task must be delegated to the upper-level software stack, which uses multiple independent device drivers to perform instruction translation and address mapping. This not only leads to a complex software stack and lengthy call paths, but also requires the redevelopment or modification of drivers whenever a new accelerator architecture needs to be integrated, resulting in a severe lack of flexibility and standardization in system expansion. Summary of the Invention

[0003] In view of the above-mentioned technical deficiencies, the purpose of this invention is to provide a high-density integrated computer hardware computing power expansion module to solve the problems mentioned in the background art.

[0004] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: The present invention provides a high-density integrated computer hardware computing power expansion module, including a physical bus adaptation unit, an instruction description template storage unit, an opcode matching and parsing unit, a segmented address mapping unit, and a task descriptor assembly and distribution unit.

[0005] The physical bus adapter unit is used to connect to the standard computer bus and extract bus write transactions targeting this module from the bus transaction layer. The instruction description template storage unit is used to store multiple sets of instruction description templates corresponding to various instruction set architecture accelerators. Each set of templates records the opcode mask, operand extraction rules and addressing mode identifier for each operation type of the corresponding accelerator. The opcode matching and parsing unit is connected to the physical bus adaptation unit and is used to match the opcode in the bus write transaction payload with the opcode mask in the corresponding group according to the target accelerator type, and obtain each operand according to the operand extraction rules in the matched template. The segmented address mapping unit is used to store the mapping table entries that map the bus domain logical address to the local physical address of each accelerator, and to receive operands that are determined to be logical addresses and convert them into the corresponding local physical addresses by looking up the table. The task descriptor assembly and distribution unit is used to assemble a task descriptor in its native format according to the target accelerator's command interface bit field specification, based on the matched operation type and address translation result, and write it into the target accelerator's command queue.

[0006] In this invention, the main processor does not need to be aware of the differences in instruction sets between the back-end accelerators, and the task control blocks it issues always maintain a unified field structure. After the module receives the control block, the internal instruction description template storage unit and opcode matching and parsing unit work together to lock the native opcode semantics of the accelerator corresponding to the current operation within one or a few clock cycles. This significantly shortens the command parsing path, eliminates the dependence on upper-level multi-level driver software, and thus improves the real-time performance and determinism of control under high-density deployment. At the same time, the mapping relationship between the bus domain logical address and the local physical address of different accelerators maintained by the segmented address mapping unit enables the main processor to uniformly address the scattered accelerator memory within a continuous logical address space. This mechanism effectively avoids the complex situation where the address spaces of each accelerator are isolated from each other and must be managed separately by the main processor software. It provides an address domain basis for integrating more accelerator chips within a single module and directly supports the design goal of "high-density integration".

[0007] Preferably, in this invention, the main processor does not need to be aware of the differences in instruction sets of the back-end accelerators, and the task control blocks it issues always maintain a unified field structure. After the module receives the control block, the internal instruction description template storage unit and opcode matching and parsing unit work together to lock the native opcode semantics of the accelerator corresponding to the current operation within one or a few clock cycles. This significantly shortens the command parsing path, eliminates the dependence on upper-level multi-level driver software, and thus improves the real-time performance and determinism of control under high-density deployment. At the same time, the mapping relationship between the bus domain logical address and the local physical address of different accelerators maintained by the segmented address mapping unit enables the main processor to uniformly address the scattered accelerator memory within a continuous logical address space. This mechanism effectively avoids the complex situation where the address spaces of each accelerator are isolated from each other and must be managed separately by the main processor software, providing an address domain basis for integrating more accelerator chips in a single module and directly supporting the design goal of "high-density integration".

[0008] Preferably, when the segmented address mapping unit looks up the table, it matches the received logical address with the bus domain logical base address and range of all mapping table entries. If a match is found, it calculates the difference between the logical address and the logical base address of the matched entry, and then sums the difference with the local physical base address of that entry to obtain the local physical address of the target accelerator. This address translation based on parallel matching and simple arithmetic operations can be completed within one clock cycle, ensuring extremely low and predictable latency throughout the entire pipeline from the control block to the task descriptor submission. For scenarios requiring intensive, small-granularity inference tasks, this low-latency characteristic can significantly improve the throughput of the computing unit.

[0009] Preferably, the segmented address mapping unit maintains a data ready flag for each mapping table entry. When an address is matched but the ready flag is invalid, the unit delays address return and triggers the physical bus adaptation unit to initiate a direct memory access data transfer from system memory to the target accelerator's local memory. The calculated local physical address is returned only after the transfer is complete and the flag is updated to be valid. This mechanism moves the trigger point for data transfer down from the software stack to the hardware logic, achieving automatic connection between task dependencies and data prefetching. The main processor does not need to ensure that the data is already located on the accelerator when submitting a task, which greatly simplifies the programming model and avoids task failures or software waiting overhead caused by data not being ready.

[0010] Preferably, the task descriptor assembly and distribution unit includes a task dependency buffer queue for temporarily storing task descriptors that cannot be submitted immediately due to invalid data readiness flags. Once it is confirmed that all operand dependencies of a temporarily stored task are ready, it is then submitted to the target accelerator command queue. This queue mechanism allows the module to continue receiving and preprocessing subsequent task control blocks while waiting for data prefetching, achieving out-of-order reception and sequential submission management of the task flow, and improving the utilization rate of the internal pipeline and overall throughput capacity of the module.

[0011] Preferably, the instruction description template in the template storage unit is written by the main processor via the standard computer bus during system initialization or when a new accelerator is connected. After the new template is written, the main processor can use the unified control interface to drive the newly connected accelerator. This feature simplifies the hardware adaptation of heterogeneous accelerators from traditional driver development to template data configuration. When a new accelerator architecture needs to be integrated, it is not necessary to modify the operating system kernel or driver program. Template injection and address mapping table updates can be completed in a single bus access, allowing existing applications to transparently call new computing resources. This directly reflects the convenience and system-level scalability of "computing power expansion".

[0012] Preferably, multiple modules are each connected to the same main processor system via a standard computer bus, and each module presents itself as a logical device with the same control interface to the main processor device driver. This symmetrical modular design allows the system's computing power to expand linearly like building blocks; the main processor driver performs unified enumeration and scheduling of all modules, without distinguishing whether the module is equipped with a GPU, NPU, or FPGA, significantly reducing the management complexity in multi-GPU collaborative scenarios.

[0013] Preferably, the main processor's device driver can send the control block of the computing task to a module, which then forwards it via a standard computer bus to another module within the network with the target idle computing resources for processing. This enables direct load balancing between modules; when the task queue of a module is saturated, newly arriving tasks can be transparently routed to other idle modules without secondary intervention from the main processor. This achieves self-organizing resource balancing within the computing power cluster composed of modules, further improving the overall computing power utilization efficiency of the system.

[0014] This invention also provides a method for expanding the computing power of high-density integrated computer hardware, applied to the aforementioned module, comprising the following steps: receiving a task control block issued by the main processor via a bus write transaction, containing the target accelerator type, opcode, and operand addresses; selecting an instruction description template group based on the target accelerator type, matching the opcode with the templates within the group to determine the operation type semantics and operand extraction rules; obtaining each operand according to the extraction rules, converting the logical addresses that need to be converted to the target accelerator's local physical address by looking up a segmented address mapping table; assembling a native task descriptor containing the operation type semantics and local physical address according to the target accelerator's command interface bit field specification, and writing it into its command queue; monitoring the task completion status, and writing the execution result from the target accelerator's local memory back to the main processor's designated memory area via bus DMA. This method fully corresponds to the module hardware workflow, compressing the call path from multi-layered software translation to a hardware operation chain triggered by a single bus transaction, significantly improving the efficiency and predictability of heterogeneous computing power scheduling.

[0015] Preferably, after obtaining the local physical address from the table and before submitting the task descriptor, if the data readiness flag of the corresponding address segment is found to be invalid, a DMA prefetch from system memory to the accelerator's local memory is triggered. The subsequent submission steps continue only after the data is ready. This implicit data dependency handling enables the method implementation to work closely with the hardware mechanism, ensuring that the task is only assigned after all data preparation is complete, thus avoiding execution errors or idle runs caused by data not being ready.

[0016] The beneficial effects of this invention are as follows: The high-density integrated computer hardware computing power expansion module provided by this invention, by completely embedding instruction adaptation and address mapping in the hardware logic of the bus transaction layer, enables the main processor to directly drive accelerators such as GPUs, NPUs, and FPGAs with incompatible instruction sets using a unified bus device access semantics. This achieves higher-density heterogeneous computing resource deployment within a single module, eliminating the dependence on dedicated interconnect hardware and multiple independent driver software. Multiple modules can be linearly expanded through a standard computer bus, presenting themselves as a unified logical computing power device with the same control interface. When adding computing power, it is only necessary to insert the module on the bus and dynamically inject the instruction template to put it into use. At the same time, the control path for task issuance is simplified from multi-layer software translation to a hardware operation chain triggered by a single bus write transaction. Opcode matching and address translation are completed within a few clock cycles, and the scheduling latency has high determinism. The integration of the new architecture accelerator is also changed from kernel-driven development to template data configuration, which significantly shortens the deployment cycle of heterogeneous hardware and fully supports the high-density integration and elastic computing power expansion of the system. Detailed Implementation

[0017] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1: The high-density integrated computer hardware computing power expansion module provided by this invention, in a typical implementation, connects to the main processor via a PCIExpress 4.0 x16 interface and integrates three accelerator chips: a GPU, an NPU, and an FPGA. After the module is powered on, the device driver on the main processor enumerates the module through the configuration space and uses BAR space write operations to write a pre-compiled instruction description template set into the SRAM-type instruction description template storage unit within the module. Taking the GPU as an example, the driver writes the opcode mask and operand positions corresponding to the element-level operations, tensor shrinking, convolution, and other operations it supports one by one. At the same time, the driver also configures mapping table entries in the content-addressable memory of the segmented address mapping unit. For example, it maps the 256MB space from the system bus domain logical address 0x80000000 to 0x8FFFFFFF to the GPU's local video memory base address and sets all the corresponding data ready flags to invalid, indicating that no data has been loaded.

[0019] When the main processor needs to perform a matrix multiplication, the driver constructs a task control block in memory. The header of this control block is filled with the accelerator type number 0x01 (representing the GPU) and the opcode 0x0C (representing matrix multiplication). Then, the logical addresses of the source operand A, source operand B, and destination operand C are filled in according to a predefined order. Afterward, the driver sends this control block to a fixed offset in the module's BAR space via a PCIe memory write transaction. The physical bus adaptation unit inside the module, after detecting that the write operation hits the control address range at the transaction layer, extracts the complete task control block payload and passes it to the opcode matching and parsing unit.

[0020] The opcode matching and parsing unit first reads the accelerator type field 0x01 and selects the template region corresponding to the GPU in the instruction description template storage unit. Then, it performs a parallel comparison between the opcode 0x0C and the opcode mask of all templates in the region. After only one clock cycle, it hits the template corresponding to the matrix multiplication operation. Based on the operand offset array recorded in the template, the parsing unit separates three logical address fields from the control block and determines that they are all logical address indirect addressing based on their addressing mode identifier. Then, it sends these three addresses to the segmented address mapping unit for processing in sequence.

[0021] After receiving the first logical address, the segmented address mapping unit performs a parallel range match with all mapping entries in the content-addressable memory. When an entry is matched, the hardware calculates the segment offset using a subtractor, and then adds the local physical base address of the entry to obtain the actual address in the GPU's local memory using an adder. Before returning the address, the unit checks the data ready flag of the matched entry. Assuming that the data of the source operand A has not yet been loaded into the GPU memory, the flag is invalid, and the address mapping unit will not return the address immediately. Instead, it will send a DMA read request to the physical bus adaptation unit, which carries the starting address and data length of operand A in the system memory. The physical bus adaptation unit then initiates a memory read transaction on the PCIe bus, reads the data block back, and writes it to the corresponding area of ​​the GPU memory. After the write is completed, the address mapping unit sets the data ready flag of the corresponding entry to valid before transmitting the calculated local physical address.

[0022] During the data prefetching process, the task descriptor assembly and distribution unit temporarily stores some information about the task in the task dependency buffer queue and continues to process other control blocks that may arrive later. Once the local physical addresses of all three operands are ready and the corresponding data is valid, the task is released from the dependency queue and enters the assembly phase. The assembly unit, according to the bit field specification of the GPU command queue register, fills the opcode 0x0C into the highest byte of the instruction word and fills the three local addresses into the address fields of the source and destination operands, respectively, generating a task descriptor in native GPU format. Subsequently, the descriptor is written to the GPU's command queue register through a single PCIe memory write transaction.

[0023] After the GPU completes the matrix multiplication, it sets the status register. When the status polling circuit of the task descriptor assembly and distribution unit in the module detects the status bit in the next detection cycle, it immediately notifies the physical bus adaptation unit to initiate a DMA write operation from the GPU memory to the main processor memory, moving the result data of the region where the destination operand C is located to the pre-agreed memory destination in the task control block. After the data transfer is completed, the module notifies the driver to complete the task through the MSI-X interrupt, and a complete heterogeneous computing call ends.

[0024] Example 2: In another typical scenario, three of the aforementioned modules are plugged into the system and connected to the same main processor via a PCIe switching chip. The main processor driver assigns a unique number to each of the three modules and maintains a simplified load table in memory that records the current task queue depth of each module. When image classification inference requests arrive consecutively, the driver checks the load table and finds that the queues of modules A and B are nearing the waterline, while module C is essentially idle. Therefore, it directly writes the control block for the next inference task into the BAR space of module C. After receiving the control block, module C's internal parsing and distribution logic is completely consistent with the aforementioned process, submitting the task to the local NPU for execution and returning the result. The entire process is completely transparent to the upper-layer AI application framework; the scheduling decision is made solely by the driver based on the status information fed back by the modules, without requiring any modification to the application code.

[0025] When integrating a newly emerging RISC-V instruction set coprocessor, users only need to obtain the coprocessor's instruction encoding manual, generate a set of instruction description templates according to a predefined format, covering common operations such as vector loading, vector addition, and vector storage. Then, without shutting down the main system, this set of templates is written to the free address area of ​​the instruction description template storage unit through the interface provided by the module driver, and a mapping table entry is added to the segmented address mapping unit to allocate an unused bus domain logical address segment for the coprocessor's local instruction memory. After writing, the upper-layer application can immediately dispatch tasks to the coprocessor through a unified control block interface. The module hardware automatically completes instruction translation and address mapping without restarting the system or updating any drivers, demonstrating the unique advantages of high-density integrated computer hardware computing power expansion modules in plug-and-play and online expansion.

[0026] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A high-density integrated computer hardware computing power expansion module, characterized in that: include: The physical bus adapter unit is used to connect to the standard computer bus and extract bus write transactions targeting this module from the bus transaction layer. The instruction description template storage unit is used to store multiple sets of instruction description templates corresponding to various instruction set architecture accelerators. Each set of templates records the opcode mask, operand extraction rules and addressing mode identifier for each operation type of the corresponding accelerator. An opcode matching and parsing unit, connected to the physical bus adaptation unit, is used to match the opcode in the bus write transaction payload with the opcode mask in the corresponding group according to the target accelerator type, and obtain each operand according to the operand extraction rules in the matched template. The segmented address mapping unit is used to store the mapping table entries that map the bus domain logical address to the local physical address of each accelerator, and to receive operands that are determined to be logical addresses and convert them into the corresponding local physical addresses by looking up the table. The task descriptor assembly and distribution unit is used to assemble a task descriptor in its native format according to the matched operation type and address translation result, and write it into the command queue of the target accelerator, based on the command interface bit field specification of the target accelerator.

2. The high-density integrated computer hardware computing power expansion module as described in claim 1, characterized in that, The addressing mode identifier in the instruction description template is used to indicate whether the current operand is an immediate value or a logical address that needs to be converted by looking up a table; the opcode matching and parsing unit will directly pass the immediate value or send the logical address to the segmented address mapping unit for processing.

3. The high-density integrated computer hardware computing power expansion module as described in claim 2, characterized in that, When the segmented address mapping unit looks up the table, it matches the received logical address with the bus domain logical base address and range of all mapping table entries. If a match is found, it calculates the difference between the logical address and the logical base address of the matched table entry, and then sums the difference with the local physical base address of the table entry to obtain the local physical address of the target accelerator.

4. The high-density integrated computer hardware computing power expansion module as described in claim 3, characterized in that, The segmented address mapping unit maintains a data ready flag for each mapping table entry. When an address is hit but the ready flag is invalid, the unit delays the address return and triggers the physical bus adaptation unit to initiate a direct memory access data transfer from system memory to the target accelerator's local memory. After the transfer is completed and the flag is updated to be valid, the calculated local physical address is returned.

5. The high-density integrated computer hardware computing power expansion module as described in claim 4, characterized in that, The task descriptor assembly and distribution unit includes a task dependency buffer queue, which is used to temporarily store task descriptors that cannot be submitted immediately due to invalid data readiness flags; after confirming that all operand dependencies of a temporarily stored task are ready, it is then submitted to the target accelerator command queue.

6. A method for expanding the computing power of high-density integrated computer hardware, characterized in that: The high-density integrated computer hardware computing power expansion module according to any one of claims 1 to 5 includes the following steps: receiving a task control block issued by the main processor via a bus write transaction, containing the target accelerator type, opcode, and operand addresses; selecting an instruction description template group based on the target accelerator type, matching the opcode with the templates within the group to determine the operation type semantics and operand extraction rules; obtaining each operand according to the extraction rules, converting the logical addresses that need to be converted to the target accelerator's local physical address by looking up the segmented address mapping table; assembling a native task descriptor containing the operation type semantics and local physical address according to the target accelerator's command interface bit field specification, and writing it into its command queue; monitoring the task completion status, and writing the execution result from the target accelerator's local memory back to the main processor's designated memory area via bus DMA.

7. The method for expanding the computing power of high-density integrated computer hardware as described in claim 6, characterized in that, After obtaining the local physical address from the table and before submitting the task descriptor, if the data readiness flag of the corresponding address segment is found to be invalid, a DMA prefetch from system memory to the accelerator's local memory is triggered. The subsequent submission steps will continue once the data is ready.