Data processing system, method, and computer system

Abstract

The present application relates to the field of computer systems, and discloses a data processing system, a method, and a computer system. In order to solve the problems of long access path and high access delay for accessing an expanded memory, the data processing system comprises a processing board and a memory expansion board. The processing board is provided with a processing core and a first controller, and the memory expansion board is provided with a storage component and a control component. The present application can enable a processing board to directly access a storage component on a memory expansion board, and memory copying does not need to be performed by means of a server host, thereby reducing the degree of coupling between the processing board and the server host when accessing an expanded memory. While effectively expanding the memory of the processing board, an access path of the processing board for the expanded memory is shortened and access delay is reduced, thereby reducing a communication bottleneck in the training process of a pre-trained model.

Description

A data processing system, method and computer system

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411534503.7, filed on October 31, 2024, entitled “A Data Processing System, Method and Computer System”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of computer systems, and in particular to a data processing system, method and computer system. Background Technology

[0004] In the training process of current artificial intelligence models, due to the limited computing power of a single GPU (Graphics Processing Unit), multiple GPUs are required to collaborate on computation. This involves interconnection and communication between multiple GPUs, as well as the transfer of intermediate computational data. To address the issue of insufficient memory, memory expansion cards are typically used.

[0005] Memory expansion cards typically connect to computer systems via PCIe (Peripheral Component Interconnect Express) interfaces, providing GPUs with more memory resources to support the processing of more complex datasets and computational tasks. However, when the GPU accesses this expanded memory, it must perform memory copy operations through the CPU (Central Processing Unit). This lengthens the access path, increases latency, and limits the capacity of the expanded memory, thus creating a communication bottleneck for pre-trained model training. Summary of the Invention

[0006] The purpose of this application is to provide a data processing system, method, and computer system that enables the processing board to directly access the storage components on the memory expansion board without having to copy memory through the server host. This reduces the coupling between the processing board and the server host when accessing the expanded memory, effectively expands the memory of the processing board, shortens the access path of the processing board to the expanded memory, reduces access latency, and thus reduces the communication bottleneck in the training process of the pre-trained model.

[0007] To address the aforementioned technical problems, this application provides a data processing system, including a processing board and a memory expansion board. The processing board includes: a processing core, configured to identify storage components through a bridge driver thread, establish a first mapping relationship between the address space of the processing board and the storage components, and forward internal access requests conforming to the first mapping relationship; the internal access request includes a first internal access request generated by the processing core; a first controller, configured to convert the received internal access request into an extended access request and output it; the memory expansion board includes a storage component and a control component, the control component being configured to perform access operations on the storage component in response to the extended access request.

[0008] The processing board also includes a second controller, which is used to establish a second mapping relationship between the address space of the storage component and the server host, and to forward the first host access request sent by the server host that conforms to the second mapping relationship; the first controller is also used to convert the received first host access request into an extended access request and output it.

[0009] The processing board also includes a system bus for forwarding internal access requests to the first controller and / or forwarding first host access requests to the first controller.

[0010] Specifically, the processing core is used to generate internal access requests, identify storage components through the bridge driver thread, establish a first mapping relationship between the address space of the processing board and the physical address of the storage component, forward internal access requests whose destination address matches the first mapping relationship, and discard internal access requests whose destination address does not match the first mapping relationship.

[0011] The processing board also includes at least one computing core for generating a second internal access request corresponding to the current computing needs; the internal access request also includes a second internal access request.

[0012] The memory expansion board also includes a monitoring and management module, which is used to detect the operating status of the memory expansion board.

[0013] The control components include: an analysis and processing module, used to generate and output a target memory access request based on the access address and access type of the received current access request; the current access request includes an extended access request; and a memory control module, used to perform access operations on the storage components corresponding to the target memory access request.

[0014] The analysis and processing module includes a first hard core module, which is used to generate and output a first memory access request based on the access address and access type of the received extended access request; the target memory access request includes the first memory access request; and the memory control module is specifically used to respond to the received first memory access request and perform corresponding access operations on the storage component.

[0015] The analysis and processing module also includes a second hard core module, which is used to generate and output a second memory access request based on the access address and access type of the received second host access request; the current access request also includes a second host access request sent by the server host, and the target memory access request also includes a second memory access request; the memory control module is specifically used to respond to the received second memory access request and perform corresponding access operations on the storage component.

[0016] The analysis and processing module further includes: a network interface module for receiving remote access requests; a network protocol stack for generating and outputting a third memory access request based on the access address and access type of the remote access request; the target memory access request also includes a third memory access request, and the current access request also includes a remote access request; the memory control module is specifically used to respond to the received third memory access request and perform corresponding access operations on the storage component.

[0017] The analysis and processing module also includes an access arbitration module, which is used to arbitrate the received first memory access request and / or second memory access request and / or third memory access request, and output the target memory access request that was successfully arbitrated.

[0018] The memory expansion board also includes a network optical module connected to the network interface module, used to receive remote access requests sent by remote computing nodes.

[0019] The control component also includes a control register module, which manages the registers in the memory control module and the analysis and processing module.

[0020] The storage component includes a non-volatile storage device; the memory control module includes: a shared memory controller, used to set the access status of the access address of the received target memory access request to a locked state and forward the target memory access request; and a memory access controller, used to perform corresponding access operations on the non-volatile storage device according to the access address and access type of the target memory access request.

[0021] The storage component also includes memory granules; the memory access controller is specifically used to parse and forward access information in the target memory access request; the access information includes the access type and access address corresponding to the target memory access request; the memory control module also includes: a cache controller, used to respond to the access type in the access information as a read operation type, if the access address in the access information hits the cache, read the corresponding response data from the memory granules and return it to the memory access controller, otherwise, forward the access information and cache the response data returned by the storage controller in the memory granules; and a storage controller, used to read the corresponding response data from the non-volatile storage device according to the access address in the access information sent by the cache controller and return it.

[0022] The cache controller is also used to forward the access information in response to the access type being a write operation type; the storage controller is also used to write the access data in the access information to the non-volatile storage device according to the access address in the access information sent by the cache controller.

[0023] Specifically, the storage controller is used to determine the range of read addresses, including the access address, based on the access address in the access information sent by the cache controller, and read the data to be cached from the non-volatile storage device according to the read address range and return it; the cache controller is used to respond to the access type in the access information as a read operation type. If the access address in the access information hits the cache, it reads the corresponding response data from the memory particle and returns it to the memory access controller. Otherwise, it forwards the access information, caches the data to be cached in the memory particle, and returns the response data in the data to be cached to the memory access controller.

[0024] The memory access controller is also used to return response data to the shared memory controller, so that the shared memory controller can return the response data to the sender of the target memory access request; the sender can be a server host, a processing board, or a remote computing node.

[0025] To address the aforementioned technical problems, this application also provides a computer system, including a server host and a data processing system as described above.

[0026] To address the aforementioned technical problems, this application also provides a data processing method applied to any of the data processing systems described above. The data processing system includes a processing board and a memory expansion board equipped with storage components. The data processing method includes:

[0027] The processing core on the processing board identifies the storage component through the bridge driver thread, establishes a first mapping relationship between the address space of the processing board and the storage component, and forwards internal access requests that conform to the first mapping relationship. The internal access request includes the first internal access request generated by the processing core. The first controller on the processing board converts the received internal access request into an extended access request and outputs it. The control component on the memory expansion board responds to the extended access request and performs access operations on the storage component.

[0028] This application provides a data processing system that directly connects a memory expansion board and a processing board via connectors and cables, reducing the coupling between the processing board and the server host when accessing the expanded memory. A bridge driver function is implemented on a processing core of the processing board using a thread to identify and address-map the processing components on the connected memory expansion board, enabling the processing board to directly access the storage components on the memory expansion board without needing to copy memory through the CPU. This effectively expands the processing board's memory while shortening the access path to the expanded memory, reducing access latency, and thus reducing the communication bottleneck during the training process of the pre-trained model.

[0029] This application also provides a data processing method and a computer system, which have the same beneficial effects as the data processing system described above. Attached Figure Description

[0030] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 is a schematic diagram of the structure of the first data processing system provided in this application;

[0032] Figure 2 is a schematic diagram of the structure of the second data processing system provided in this application;

[0033] Figure 3 is a schematic diagram of the structure of a processing plate provided in this application;

[0034] Figure 4 is a schematic diagram of the structure of a memory expansion board provided in this application;

[0035] Figure 5 is a schematic diagram of the structure of a control component provided in this application;

[0036] Figure 6 is a schematic diagram of the structure of the third data processing system provided in this application;

[0037] Figure 7 is a flowchart of the data processing method provided in this application. Detailed Implementation

[0038] The core of this application is to provide a data processing system, method, and computer system that enables the processing board to directly access the storage components on the memory expansion board without having to copy memory through the server host. This reduces the coupling between the processing board and the server host when the processing board accesses the expanded memory. While effectively expanding the memory of the processing board, it shortens the access path of the processing board to the expanded memory, reduces access latency, and thus reduces the communication bottleneck in the training process of the pre-trained model.

[0039] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] In a first aspect, referring to Figure 1, this application provides a data processing system, including a processing board 1 and a memory expansion board 2, wherein: the processing board 1 includes: a processing core 11, used to identify a storage component 21 through a bridge driver thread, establish a first mapping relationship between the address space of the processing board 1 and the storage component 21, and forward internal access requests conforming to the first mapping relationship; the internal access request includes a first internal access request generated by the processing core 11; a first controller 12, used to convert the received internal access request into an extended access request and output it; the memory expansion board 2 includes a storage component 21 and a control component 22, the control component 22 being used to perform access operations on the storage component 21 in response to the extended access request.

[0041] In this embodiment, considering the limited memory accessible by a single GPU, to address the memory shortage issue, memory expansion cards using DDR (Double Data Rate) or PMem (Persistent Memory) memory chips with PCIe (Peripheral Component Interconnect Express) gold finger interfaces can be manufactured. However, when the GPU accesses this expanded memory, it must perform memory copy operations through the CPU (Central Processing Unit), which lengthens the access path, increases latency, and limits the capacity of the expanded memory. Therefore, this embodiment designs a new processing board 1 and a memory expansion board 2, which together form the data processing system.

[0042] The data processing system includes at least one processing board 1 and at least one memory expansion board 2. One processing board 1 can connect to multiple memory expansion boards 2, and one memory expansion board 2 can connect to multiple processing boards 1, depending on the actual engineering needs. To facilitate understanding of the memory expansion scheme provided in this embodiment, this embodiment uses one processing board 1 connected to one memory expansion board 2 as an example. Specifically, the processing board 1 in this embodiment can be a GPU board. The processing board 1 and the memory expansion board 2 are detachably connected, specifically through a connecting device. As one embodiment, referring to FIG2, the connecting device includes, but is not limited to, a first connector 13 disposed on the processing board 1, a second connector 23 disposed on the memory expansion board 2, and a cable for connecting the first connector 13 and the second connector 23. Both the first connector 13 on the processing board 1 and the second connector 23 on the memory expansion board 2 can be MCIO (Mini Cool Edge IO) connectors. It is understood that MCIO is a new type of high-speed cable interface technology designed to meet the high-speed data transmission requirements of data centers and high-performance computing environments. It supports multiple protocols including PCIe and CXL (Compute Express Link), offering good signal integrity and a small package size, making it suitable for long-distance inter-board data transmission. Of course, in addition to connectors and cables, other connection methods can be used to meet the specific engineering needs; this embodiment does not impose specific limitations. Processing board 1 and memory expansion board 2 are directly connected via MCIO high-speed differential cables. This facilitates reducing the coupling between processing board 1 and the server host when using expanded memory, allowing processing board 1 to directly access the mounted expanded memory. Furthermore, it improves system integration and increases system bandwidth while reducing access latency.

[0043] The processing core 11 can specifically be a GPU core on the processing board 1. In this embodiment, the bridge driver utilizes the processing core 11 to run on a GPU thread. The bridge driver can specifically be an RC (Root Complex) bridge driver, meaning that the functions of the RC bridge driver are offloaded from the CPU to the processing core 11 of the processing board 1 to realize the memory expansion function of the processing board 1. At the same time, it can reduce the CPU load, allowing the CPU to focus more on other computing tasks and improve the overall system efficiency. Specifically, when the first connector 13 on the processing board 1 and the second connector 23 on the memory expansion board 2 are connected by a cable, the bridge driver thread on the processing core 11 starts to execute. The bridge driver thread will send signals or perform probing operations to identify the connected memory expansion board 2 and its storage component 21. After identifying the storage component 21, the bridge driver thread establishes a first mapping relationship between the address space of the processing board 1 and the storage component 21. The first mapping relationship defines how the logical address on the processing board 1 corresponds to the physical address of the storage component 21. After the first mapping relationship is established, the processing board 1 can identify the address range of the storage component 21 and use it as an available memory resource. The processing core 11 generates an internal access request based on the current application scenario. In this embodiment, the internal access request generated by the processing core 11 is identified as the first internal access request. The internal access request contains a target address. The bridge driver thread determines whether the internal access request satisfies the first mapping relationship, that is, whether the target address requested by the internal access request matches the address of a storage component 21 on a memory expansion board 2 in the first mapping relationship. If so, it is determined that the internal access request satisfies the first mapping relationship, and the internal access request is forwarded to the first controller 12.

[0044] The first controller 12 converts the internal access request into an extended access request that can be transmitted through the first connector 13, and then transmits it to the second connector 23 through the first connector 13 and the cable. In this embodiment, the first controller 12 can specifically be a CXL RC (Compute Express Link Root Complex) controller, and correspondingly, the first controller 12 is specifically used to convert the internal access request into an extended access request that conforms to the CXL protocol.

[0045] After receiving an expansion access request through the second connector 23, the control component 22 on the memory expansion board 2 performs corresponding operations on the storage component 21 on the memory expansion board 2 based on the expansion access request. In this embodiment, the storage component 21 is connected to the control component 22, and the storage component 21 includes at least one storage device. In this embodiment, the type of storage device is not limited.

[0046] As can be seen, in this embodiment, the memory expansion board 2 and the processing board 1 are directly connected through connectors and cables, which reduces the coupling between the processing board 1 and the server host when accessing the expanded memory. A thread is used on one of the processing cores 11 of the processing board 1 to implement the bridge driver function, which identifies and maps the processing components on the memory expansion board 2 connected to it, so that the processing board 1 can directly access the storage components 21 on the memory expansion board 2 without having to copy memory through the CPU. While effectively expanding the memory of the processing board 1, the access path of the processing board 1 to the expanded memory is shortened, the access latency is reduced, and thus the communication bottleneck in the training process of the pre-trained model is reduced.

[0047] In an exemplary embodiment, referring to FIG3, the processing board 1 further includes a second controller 14, configured to establish a second mapping relationship between the address space of the storage component 21 and the server host, and forward a first host access request sent by the server host that conforms to the second mapping relationship; the first controller 12 is further configured to convert the received first host access request into an extended access request and output it.

[0048] In this embodiment, the second controller 14 can specifically be a CXL EP (End Point) controller, used to receive and send data between the processing board 1 and the CPU of the server host, so as to enable the server host to access the extended memory of the processing board 1. Specifically, the second controller 14 is used to establish a second mapping relationship between the storage component 21 and the address space of the server host. The second mapping relationship allows the CPU of the server host to access the extended memory of the processing board 1 through the address space. When the server host sends a first host access request, the second controller 14 checks whether these first host access requests satisfy the second mapping relationship. If they do, the first host access request is forwarded to the first controller 12. The server host can increase its effective memory capacity through the extended memory of the processing board 1 without directly upgrading the physical memory of the host.

[0049] In one exemplary embodiment, the processing board 1 further includes a system bus 15 for forwarding internal access requests to the first controller 12 and / or forwarding first host access requests to the first controller 12.

[0050] In this embodiment, the processing board 1 also includes a system bus 15. The first controller 12, the second controller 14 and the processing core 11 are all mounted on the system bus 15. Data can be transmitted through the system bus 15, which can reduce the complexity and delay of data transmission between various components, thereby improving transmission efficiency, simplifying board-level design and reducing the number of required hardware interfaces and connectors.

[0051] In one exemplary embodiment, the processing board 1 further includes at least one computing core 16 for generating a second internal access request corresponding to the current computing needs; the internal access request further includes a second internal access request.

[0052] It is understood that, in addition to the processing core 11, the processing board 1 also includes other GPU cores, namely the computing core 16 in this embodiment. The computing core 16 generates a second internal access request based on the current computing needs, and this second internal access request is also included in the internal access request. The second internal access request is transmitted to the processing core 11, which determines whether to forward it. This allows the processing core 11 to coordinate memory access among multiple GPU cores, ensuring that they can efficiently share extended memory resources, avoiding resource conflicts and waste. In addition, it can optimize data paths, reduce data transmission latency between GPU cores, and improve data transmission efficiency. By centrally processing memory access requests through the processing core 11, the system can more easily expand the number of GPU cores without large-scale modifications to the memory management architecture. The processing core 11 can ensure that high-performance computing tasks receive sufficient memory bandwidth and low-latency access, thereby improving computing performance.

[0053] In one exemplary embodiment, referring to FIG3, the processing board 1 further includes a first gold finger interface 17 detachably connected to a first slot on the server host; the server host is configured to allocate server host address space to the processing board 1 after the first gold finger interface 17 is connected to the first slot.

[0054] In this embodiment, the first gold finger interface 17 is a gold finger interface that supports the PCIe communication standard. The PCIe communication standard includes, but is not limited to, PCIe Gen4x16, PCIe Gen5x16, etc. The selection can be based on the compatibility of the server motherboard's controller group, processor, and related devices. This embodiment does not make specific limitations here.

[0055] In one exemplary embodiment, referring to FIG4, the memory expansion board 2 further includes a second gold finger interface 24 that is detachably connected to a second slot on the server host.

[0056] In this embodiment, the second gold finger interface 24 is a gold finger interface that supports the PCIe communication standard. The PCIe communication standard includes, but is not limited to, PCIe Gen4x16 and PCIe Gen5x16 standards. The selection can be based on the compatibility of the server motherboard's controller group, processor, and related devices, and this embodiment does not impose specific limitations. The gold fingers of the two cards are respectively inserted into the corresponding PCIe slots on the server host, facilitating the server host's access to the memory expansion board 2 and the processing board 1.

[0057] In an exemplary embodiment, referring to FIG4, the memory expansion board 2 further includes a monitoring and management module 25 for detecting the operating status of the memory expansion board 2.

[0058] This embodiment also includes a monitoring and management module 25, which can be built from an MCU (Microcontroller Unit) and related sensors to monitor the operating status of the memory expansion board 2, such as voltage, current, and temperature.

[0059] In one exemplary embodiment, referring to FIG4, the memory expansion board 2 further includes a network optical module 26 for receiving remote access requests sent by a remote computing node.

[0060] In one exemplary embodiment, the control component 22 includes: an analysis and processing module, configured to generate and output a target memory access request based on the access address and access type of the received current access request; the current access request includes an extended access request; and a memory control module, configured to perform an access operation on the storage component 21 corresponding to the target memory access request.

[0061] In this embodiment, the control component 22 includes an analysis and processing module and a memory control module. These two modules work together to process received current access requests, which at least include extended access requests sent by the processing board 1. The analysis and processing module parses the access address and access type of the current access request and generates a target memory access request based on the access address and access type, ensuring that valid information in the current access request is transmitted to the memory control module. Alternatively, the analysis and processing module can also perform access permission checks based on the access address and access type. If the sender of the current access request does not have the corresponding access permission, the access request will not be transmitted further, thereby improving system security and ensuring that only valid information is transmitted to the memory control module, thus improving the quality and efficiency of data processing.

[0062] The memory control module interacts directly with the storage component 21. When the memory control module receives a target memory access request generated by the analysis and processing module, it performs specific access operations on the storage component 21 according to the content of the target memory access request, such as reading, writing, or modifying data in the storage component 21. In this embodiment, by dividing the processing of access requests into two stages, parsing and execution, the system design becomes more modular, reducing overall complexity and facilitating maintenance and upgrades.

[0063] Based on the above embodiments, the memory expansion board 2 will be described in detail with reference to FIG5.

[0064] In one exemplary embodiment, the analysis and processing module includes a first hard core module 221, which is used to generate and output a first memory access request based on the access address and access type of the received extended access request; the target memory access request includes the first memory access request; the memory control module is specifically used to perform a corresponding access operation on the storage component 21 in response to the received first memory access request.

[0065] In one exemplary embodiment, the analysis and processing module further includes a second hard core module 222, which is used to generate and output a second memory access request based on the access address and access type of the received second host access request; the current access request also includes a second host access request sent by the server host, and the target memory access request also includes a second memory access request; the memory control module is specifically used to perform corresponding access operations on the storage component 21 in response to the received second memory access request.

[0066] This embodiment also includes a second hard-core module 222 for data interaction with the server host, enabling the server host to directly access the storage component 21 on the memory expansion board 2 without needing other intermediate devices or complex communication paths, thereby reducing access latency. Furthermore, having the hard-core module handle data interaction tasks reduces the burden on the server host's CPU, allowing the CPU to focus more on computational tasks and improving overall system efficiency. Both the first hard-core module 221 and the second hard-core module 222 are hard-core IP (Intellectual Property).

[0067] In this embodiment, both the first hard core module 221 and the second hard core module 222 are in CXL EP mode. In CXL EP mode, the hard core module can be used as a memory expansion device, allowing the server host to access it directly.

[0068] In one exemplary embodiment, the analysis and processing module further includes: a network interface module 223, used to receive a remote access request; a network protocol stack 224, used to generate and output a third memory access request based on the access address and access type of the remote access request; the target memory access request also includes a third memory access request, and the current access request also includes a remote access request; the memory control module is specifically used to perform a corresponding access operation on the storage component 21 in response to the received third memory access request.

[0069] In this embodiment, the analysis and processing module further includes a network interface module 223, including but not limited to network interface modules 223 such as 400G MAC (Media Access Control) 1, 400G MAC 2, and 100G MAC, used to receive remote access requests transmitted by the network optical module 26 on the memory expansion board 2 card. The network protocol stack 224 is specifically a protocol stack capable of remotely accessing the storage component 21, which can be an Ethernet-based RDMA (Remote Direct Memory Access) protocol stack, used to implement the packet encapsulation and decapsulation operations of data packets transmitted between the memory expansion board 2 card and the remote computing node, including but not limited to parsing the access address and access type of the remote access request, and generating and outputting a third memory access request based on the access address and access type of the remote access request. Using an Ethernet-based RDMA protocol stack, the storage component 21 can be remotely initialized and accessed, which greatly improves the shared memory capacity, reduces the data parallel partitioning degree and data communication volume. Compared with the solution based on ordinary network cards, using an Ethernet-based RDMA protocol stack means that the relevant drivers do not need to run on the local server. The FPGA (Field-Programmable Gate Array) and GPU can be configured, managed and initialized on a remote host through the network, and the resource pooling flexibility is higher.

[0070] Of course, in addition to the network protocol stack 224, which can be an Ethernet-based RDMA protocol stack, other options include FCoE (Fibre Channel over Ethernet), iSCSI (Internet Small Computer System Interface), etc. The choice of which network protocol stack 224 to use depends on the specific network environment, performance requirements, cost considerations, and existing infrastructure. This embodiment does not impose any specific limitations on this.

[0071] In one exemplary embodiment, the analysis and processing module further includes an access arbitration module 225, which arbitrates the received first memory access request and / or second memory access request and / or third memory access request, and outputs the target memory access request that was successfully arbitrated.

[0072] In this embodiment, considering that there are multiple senders of the current access request, the analysis and processing module also includes an access arbitration module 225, which is used to arbitrate the received first memory access request and / or second memory access request and / or third memory access request to determine the processing order of each target memory access request. When multiple requests access the same memory resource at the same time, the arbitration module can avoid access conflicts, ensure data consistency and integrity, thereby improving the efficiency of memory access and reducing waiting time.

[0073] Specifically, the access arbitration module 225 determines the priority of the received first memory access request and / or second memory access request and / or third memory access request according to preset rules and strategies. Specifically, the arbitration can be carried out according to the order in which the access arbitration module 225 receives the above memory access requests. After arbitration, the winning request will be forwarded to the corresponding memory control module for subsequent data access operations. This is suitable for high-performance computing scenarios with high concurrency access requests.

[0074] It is understandable that, in different application scenarios, priorities can be set for the senders of the aforementioned memory access requests, such as prioritizing memory access requests sent by the host.

[0075] In one exemplary embodiment, the control component 22 further includes a control register module 226 for managing registers in the memory control module and the analysis and processing module.

[0076] In one exemplary embodiment, the storage component 21 includes a non-volatile storage device 211; the memory control module includes: a shared memory controller 227, configured to set the access status of the access address of the received target memory access request to a locked state and forward the target memory access request; and a memory access controller 228, configured to perform corresponding access operations on the non-volatile storage device 211 according to the access address and access type of the target memory access request.

[0077] In this embodiment, the storage component 21 includes a non-volatile storage device 211. The non-volatile storage device 211 has the characteristic that stored data is not lost after power failure, meaning that even if a power interruption occurs during model training, the loaded weight parameters and training data will not be lost, thus avoiding the need to reload data. Since it is not necessary to reload the initial data at each startup, the GPU's computing resources are utilized more effectively, which helps reduce the overall model training time, eliminates the data reloading step, and simplifies the model training process. In this embodiment, the memory control module includes a shared memory controller 227. The shared memory controller 227 is used to parse the access address of the received target memory access request, set the access address's access state to a locked state, ensuring that other senders cannot operate on the access address simultaneously while processing the target memory access request, maintaining data consistency and integrity. After the access is completed, the unlocked state allows other senders to access, improving system flexibility. And after processing the target memory access request, the access state of the access address is set to an unlocked state so that other senders can access the access address segment. In this embodiment, the shared memory controller 227 can specifically be a CXL.mem-based shared memory controller 227.

[0078] The shared memory controller 227 is also used to forward the target memory access request to the memory access controller 228, so that the memory access controller 228 can parse the access type and access address in the target memory access request, and perform the access operation corresponding to the access type on the non-volatile storage device 211 according to the access address. The memory access controller 228 can specifically be a DMA controller.

[0079] The non-volatile storage device 211 can be an NVMe SSD (Non-Volatile Memory Express Solid State Drive), or an HDD (Hard Disk Drive), NAND Flash Memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), etc., whichever is selected according to the actual engineering needs. This embodiment does not make specific limitations here.

[0080] In one exemplary embodiment, the storage component 21 further includes memory granules 212; the memory access controller 228 is specifically used to parse and forward access information in the target memory access request; the access information includes the access type and access address corresponding to the target memory access request; the memory control module further includes: a cache controller 229, used to, in response to the access type in the access information being a read operation type, if the access address in the access information hits the cache, read the corresponding response data from the memory granules 212 and return it to the memory access controller 228, otherwise, forward the access information and cache the response data returned by the storage controller 230 in the memory granules 212; the storage controller 230 is used to read the corresponding response data from the non-volatile storage device 211 according to the access address in the access information sent by the cache controller 229 and return it.

[0081] In this embodiment, the storage component 21 also includes memory chips 212, which can be DDR DIMM (Dual Inline Memory Module) memory modules. DDR is a type of memory chip 212 that can transmit data twice per clock cycle, i.e., sampling on the rising edge and falling edge respectively. It is usually in DIMM form, with multiple memory chips 212 concentrated on a circuit board. It is used in server hosts or various accelerator cards using address, data and control buses. Similar packaging forms include RDIMM (Registered DIMM), UDIMM (Unbuffered DIMM), Mini-DIMM, etc.

[0082] Correspondingly, the memory control module also includes a cache controller 229 and a storage controller 230. The cache controller 229 is used to cache the non-volatile storage device 211 in the memory chip 212. Specifically, when the cache controller 229 receives access information of the read operation type, it first checks whether the cache is hit, that is, whether the access address is in the memory chip 212. If it is hit, it directly reads the corresponding response data from the memory chip 212 and returns it to the memory access controller 228. If it is not hit, it forwards the request to the storage controller 230 and caches the response data returned by the storage controller 230 to improve the efficiency of subsequent accesses. The storage controller 230 is used to read the corresponding response data from the non-volatile storage device 211 according to the access address and return the data to the cache controller 229, ensuring fast data access.

[0083] It is understandable that by caching the data in the non-volatile storage device 211 to the DDR DIMM memory module, the DMA controller can perform single-byte access, which greatly improves access flexibility. This is especially beneficial for applications requiring random access or modification of small amounts of data. When handling non-contiguous or random data access, single-byte access capability reduces unnecessary block read operations, thereby improving data access efficiency. Considering that frequent data access may cause wear and tear on the non-volatile storage device 211, DDR caching reduces the number of direct accesses to the non-volatile storage device 211, thus extending its lifespan.

[0084] The memory access controller 228 is also used to return response data to the shared memory controller 227 so that the shared memory controller 227 can return the response data to the sender of the target memory access request; the sender is the server host or processing board 1 or a remote computing node.

[0085] In one exemplary embodiment, the cache controller 229 is further configured to forward the access information in response to the access type being a write operation type; the storage controller 230 is further configured to write the access data in the access information to the non-volatile storage device 211 according to the access address in the access information sent by the cache controller 229.

[0086] In this embodiment, if the access type is a write operation, the cache controller 229 and the storage controller 230 work together to write the access data in the access information to the corresponding access address in the non-volatile storage device 211.

[0087] In one exemplary embodiment, the storage controller 230 is specifically configured to determine a range of read addresses including the access address based on the access address in the access information sent by the cache controller 229, read the data to be cached in the non-volatile storage device 211 according to the read address range and return it; the cache controller 229 is specifically configured to respond to the access type in the access information being a read operation type, if the access address in the access information hits the cache, read the corresponding response data from the memory particle 212 and return it to the memory access controller 228, otherwise, forward the access information, cache the data to be cached in the memory particle 212, and return the response data in the data to be cached to the memory access controller 228.

[0088] In this embodiment, when the storage controller 230 reads data from the non-volatile storage device 211, it reads data of a preset length. For example, if the access address is 5, the storage controller 230 determines the read address range to be 0-10. This range is usually based on a preset cache line size or data block size to ensure data continuity and read efficiency. The storage controller 230 reads the cached data from the non-volatile storage device 211 according to the read address range. The cached data includes the response data corresponding to access address 5, as well as other data near that address. The cached data is then returned to the cache controller 229, which stores the received cached data in the memory chip 212 (e.g., DDR). Thus, when subsequent access requests fall within the same read address range, data can be directly retrieved from the cache without needing to access the non-volatile storage device 211 again. By pre-reading and caching data, the system can quickly respond to subsequent access requests. Caching frequently accessed data in DDR can significantly reduce access latency, significantly improving data access efficiency and overall system performance.

[0089] In summary, both the memory expansion board 2 and the processing board 1 in this embodiment are built on FPGA. The GPU card is directly connected to the FPGA-based memory expansion processing board 1 via an MCIO cable, reducing the coupling between the GPU card and the server when using extended memory, allowing the GPU to directly mount and access the extended memory and NVMe SSD. Based on the GPU memory expansion technology of this application, there is no need to go through the server motherboard's PCIe Chipset (PCIe Controller Group) related controllers, resulting in lower access latency on the physical link. Compared to solutions based on ordinary network cards, the FPGA-based RDMA communication scheme does not require running related drivers on the local server, and the FPGA and GPU can be configured, managed, and initialized on a remote host via the network, offering greater flexibility in resource pooling. Using an Ethernet-based RDMA protocol stack, remote initialization and access to the NVMe SSD are possible. The NVMe SSD serves as extended memory accessible to the GPU instance, significantly increasing shared memory capacity and reducing data parallelism and data communication volume. In the FPGA-based GPU instance, a single thread on one GPU core implements the PCIe RC driver function, enabling the GPU instance to directly access the mounted extended memory and NVMe SSD. The memory expansion FPGA board and the GPU instance FPGA board are directly plugged into the standard server PCIe slot. The two cards are directly connected via a high-speed differential cable of MCIO, which is beneficial for system integration and has the characteristics of high bandwidth and low latency. Traditional GPU cards need to communicate with external resources through an RDMA network card and then through a PCIe switch for P2P communication. The FPGA card of this application has a 400G optical network interface, which can access the RoCEv2 network or standard Ethernet to communicate with other computing nodes or storage nodes. The GPU can communicate with external resources through the directly connected FPGA without the need for the traditional solution of going through a PCIe switch and then connecting to an RDMA network card.

[0090] In a specific application, taking the interconnection of a memory expansion card and a processing card as an example, the connection relationship is shown in Figure 6. The data flow when the host accesses the memory expansion card, the processing card accesses the memory expansion card, and the remote compute node accesses the memory expansion card and solid-state storage is as follows: The memory expansion card automatically loads the configuration file and completes initialization upon power-up. The processing card powers on, and the GPU initializes through the host system and driver. GPU core 1 starts GPU thread 1 to load the RC bridge driver, completing the link initialization with the memory expansion EP device. Initial training data or model weight parameters can be loaded locally via PCIe (i.e., training data stored in the local memory of the server host) or stored from the remote compute node via the 400G Ethernet RDMA protocol and stored on the NVMe mounted on the memory expansion card. In the SSD, other GPU cores launch the model training program, accessing the DDR4 extended memory on the memory expansion board 2 via system bus 15 and RC controller. The memory controller loads data from NVMe solid-state storage to DDR4 according to the caching algorithm, and then continuously updates the cached data in DDR4 as the access address changes, so as to realize the transmission of intermediate training data during the model training process. This can reduce the time for loading model weight parameters and training data, effectively expand the GPU computing memory, reduce communication bottlenecks, increase the utilization of GPU computing resources, and reduce model training time.

[0091] Secondly, this application also provides a computer system, including a server host and a data processing system as described in any of the embodiments above.

[0092] For a description of the computer system provided in this application, please refer to the above embodiments; further details will not be repeated here.

[0093] The computer system provided in this application has the same beneficial effects as the data processing system described above.

[0094] Thirdly, referring to Figure 7, this application also provides a data processing method applied to a data processing system as described in any of the embodiments above. The data processing system includes a processing board and a memory expansion board equipped with storage components. The data processing method includes: S101: The processing core on the processing board identifies the storage components through a bridge driver thread, establishes a first mapping relationship between the address space of the processing board and the storage components, and forwards internal access requests that conform to the first mapping relationship; the internal access requests include a first internal access request generated by the processing core; S102: The first controller on the processing board converts the received internal access requests into extended access requests and outputs them; S103: The control component on the memory expansion board responds to the extended access requests and performs access operations on the storage components.

[0095] As can be seen, in this embodiment, this application provides a data processing method that directly connects the memory expansion board and the processing board, reducing the coupling between the processing board and the server host when accessing the expanded memory. A bridging driver function is implemented on a processing core of the processing board using a thread to identify and map the processing components on the memory expansion board connected to it, so that the processing board can directly access the storage components on the memory expansion board without having to copy memory through the CPU. While effectively expanding the memory of the processing board, it shortens the access path of the processing board to the expanded memory, reduces access latency, and thus reduces the communication bottleneck in the training process of the pre-trained model.

[0096] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0097] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing system, characterized in that, Including the processing board and memory expansion board; The processing board includes: The processing core is configured to identify storage components through a bridge driver thread, establish a first mapping relationship between the address space of the processing board and the storage components, and forward internal access requests that conform to the first mapping relationship; the internal access requests include the first internal access requests generated by the processing core. The first controller is configured to convert the received internal access request into an extended access request and output it. The memory expansion board includes the storage component and a control component, wherein the control component is configured to perform an access operation on the storage component in response to the expansion access request.

2. The data processing system according to claim 1, characterized in that, The processing board further includes a second controller configured to establish a second mapping relationship between the address spaces of the storage component and the server host, and to forward a first host access request sent by the server host that conforms to the second mapping relationship; The first controller is also configured to convert the received first host access request into the extended access request and output it.

3. The data processing system according to claim 2, characterized in that, The processing board also includes a system bus configured to forward the internal access request to the first controller, and / or forward the first host access request to the first controller.

4. The data processing system according to claim 1, characterized in that, The processing core is specifically configured to generate internal access requests, identify storage components through a bridge driver thread, establish a first mapping relationship between the address space of the processing board and the physical address of the storage component, forward internal access requests whose destination address matches the first mapping relationship, and discard internal access requests whose destination address does not match the first mapping relationship.

5. The data processing system according to claim 1, characterized in that, The processing board also includes at least one computing core configured to generate a second internal access request corresponding to the current computing needs; the internal access request also includes the second internal access request.

6. The data processing system according to claim 1, characterized in that, The memory expansion board also includes a monitoring and management module, which is configured to detect the operating status of the memory expansion board.

7. The data processing system according to any one of claims 1-6, characterized in that, The control component includes: an analysis and processing module configured to generate and output a target memory access request based on the access address and access type of the received current access request; the current access request includes the extended access request; The memory control module is configured to perform an access operation on the storage component corresponding to the target memory access request.

8. The data processing system according to claim 7, characterized in that, The analysis and processing module includes a first hard core module, which is configured to generate and output a first memory access request based on the access address and access type of the received extended access request; The target memory access request includes the first memory access request; The memory control module is specifically configured to perform a corresponding access operation on the storage component in response to the received first memory access request.

9. The data processing system according to claim 8, characterized in that, The analysis and processing module further includes a second hard-core module, configured to generate and output a second memory access request based on the access address and access type of the received second host access request; the current access request also includes the second host access request sent by the server host. The target memory access request also includes the second memory access request; The memory control module is specifically configured to perform a corresponding access operation on the storage component in response to the received second memory access request.

10. The data processing system according to claim 9, characterized in that, The analysis and processing module further includes: The network interface module is configured to receive remote access requests; The network protocol stack is configured to generate and output a third memory access request based on the access address and access type of the remote access request; the target memory access request also includes the third memory access request, and the current access request also includes the remote access request; The memory control module is specifically configured to perform a corresponding access operation on the storage component in response to the received third memory access request.

11. The data processing system according to claim 10, characterized in that, The analysis and processing module further includes an access arbitration module, configured to arbitrate the received first memory access request and / or the second memory access request and / or the third memory access request, and output the target memory access request that was successfully arbitrated.

12. The data processing system according to claim 10, characterized in that, The memory expansion board also includes a network optical module connected to the network interface module, configured to receive the remote access request sent by the remote computing node.

13. The data processing system according to claim 7, characterized in that, The control component also includes a control register module configured to manage the registers in the memory control module and the analysis and processing module.

14. The data processing system according to claim 7, characterized in that, The storage component includes a non-volatile storage device; The memory control module includes: A shared memory controller is configured to set the access status of the access address of a received target memory access request to a locked state and forward the target memory access request. A memory access controller is configured to perform a corresponding access operation on the non-volatile storage device according to the access address and access type of the target memory access request.

15. The data processing system according to claim 14, characterized in that, The storage component further includes memory granules; the memory access controller is specifically configured to parse and forward access information in the target memory access request; the access information includes the access type and access address corresponding to the target memory access request. The memory control module also includes: The cache controller is configured to respond to the access type in the access information as a read operation type. If the access address in the access information hits the cache, it reads the corresponding response data from the memory particle and returns it to the memory access controller. Otherwise, it forwards the access information and caches the response data returned by the storage controller in the memory particle. The storage controller is configured to read the corresponding response data from the non-volatile storage device and return it according to the access address in the access information sent by the cache controller.

16. The data processing system according to claim 15, characterized in that, The cache controller is also configured to forward the access information in response to the access type being a write operation type in the access information; The storage controller is also configured to write access data in the access information sent by the cache controller to the non-volatile storage device according to the access address in the access information.

17. The data processing system according to claim 15, characterized in that, The storage controller is specifically configured to determine a range of read addresses including the access address based on the access address in the access information sent by the cache controller, read the data to be cached from the non-volatile storage device according to the range of read addresses, and return the data. The cache controller is specifically configured to respond to the access type in the access information as a read operation type. If the access address in the access information hits the cache, the corresponding response data is read from the memory particle and returned to the memory access controller. Otherwise, the access information is forwarded, the data to be cached is cached in the memory particle, and the response data in the data to be cached is returned to the memory access controller.

18. The data processing system according to claim 17, characterized in that, The memory access controller is also configured to return the response data to the shared memory controller, so that the shared memory controller can return the response data to the sender of the target memory access request; the sender is a server host, the processing board, or a remote computing node.

19. A computer system, characterized in that, It includes a server host and a data processing system as described in any one of claims 1-18.

20. A data processing method, characterized in that, The data processing system, applicable to any one of claims 1-18, includes a processing board and a memory expansion board with storage components, and the data processing method includes: The processing core on the processing board identifies the storage component through the bridge driver thread, establishes a first mapping relationship between the address space of the processing board and the storage component, and forwards internal access requests that conform to the first mapping relationship; the internal access request includes the first internal access request generated by the processing core. The first controller on the processing board converts the received internal access request into an extended access request and outputs it. The control component on the memory expansion board responds to the expansion access request and performs an access operation on the storage component.

Images