Memory extension controller, data processing method and electronic equipment
By implementing the interconnect protocol conversion between the CPU and the AI chip through a memory expansion controller, the problem of low communication bandwidth is solved, the data transmission efficiency is improved, and it is suitable for large model training and inference tasks in AI server systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING BAIDU NETCOM SCI & TECH CO LTD
- Filing Date
- 2025-12-09
- Publication Date
- 2026-05-08
AI Technical Summary
In existing technologies, the communication bandwidth between the CPU and the AI chip is low, which cannot effectively reduce data transmission latency, and changing the original design of the CPU and the AI chip will lead to inconvenience in selection.
A memory expansion controller is used, which communicates with the CPU through the first protocol control module and with the AI chip through the second protocol control module. Combined with the data transfer module and the storage control module, the interconnection protocol conversion between the CPU and the AI chip is realized, avoiding the use of the traditional PCIe interconnection protocol.
Without altering the original design of the CPU and AI chip, the communication bandwidth between the CPU and AI chip was increased, data transmission latency was reduced, and the performance of large model training and inference was improved.
Smart Images
Figure CN121996586A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and more particularly to the fields of artificial intelligence technology such as chips and large models. It provides a memory expansion controller, a data processing method, an electronic device, and a readable storage medium. Background Technology
[0002] In artificial intelligence (AI) servers, the central processing unit (CPU) excels at handling complex logic control and serial computing tasks, while AI chips excel at handling large-scale parallel computing tasks. In order to achieve efficient parallel computing, high-speed data exchange is required between the CPU and the AI chip to reduce data transmission latency.
[0003] In existing technologies, if the original design of the CPU and AI chip is not changed, the PCIe interconnect protocol is usually used for communication between the CPU and the AI chip. However, the communication bandwidth of the PCIe interconnect protocol is low, which cannot effectively reduce the latency during data transmission. If the PCIe interconnect protocol is not used for communication between the CPU and the AI chip, it is necessary to ensure that the CPU and the AI chip can implement the same interconnect protocol, that is, there is a binding relationship between the CPU and the AI chip, which brings inconvenience to downstream users when selecting CPU and AI chips.
[0004] Therefore, how to improve the communication bandwidth between the CPU and AI chip without changing the original design of the CPU and AI chip has become an urgent technical problem to be solved. Summary of the Invention
[0005] According to a first aspect of this disclosure, a memory expansion controller is provided, comprising: a first protocol control module, a storage control module, a data transfer module, a second protocol control module, and CPU expansion memory; wherein the first protocol control module communicates with a CPU based on a first interconnection protocol, and the second protocol control module communicates with an AI chip based on a second interconnection protocol.
[0006] According to a second aspect of this disclosure, a data processing method is provided, comprising: after receiving a CPU write command based on a first interconnect protocol sent by a CPU, a memory expansion controller uses a first protocol control module to parse the CPU write command to obtain a data source address of the data to be written by the CPU and a destination address of a first AI chip; using a data transfer module, the controller obtains the local address of the data to be written by the CPU based on the data source address of the data to be written by the CPU; using a storage control module, the controller retrieves the data to be written by the CPU from the CPU extended memory based on the local address of the data to be written by the CPU; using a second protocol control module, the controller encapsulates the data to be written by the CPU and the destination address of the first AI chip based on a second interconnect protocol to obtain a first AI chip write command; and sending the first AI chip write command to the first AI chip so that the first AI chip writes the data to be written by the CPU into the video memory.
[0007] According to a third aspect of this disclosure, a data processing method is provided, comprising: after receiving a CPU read command based on a first interconnect protocol sent by a CPU, a memory expansion controller uses a first protocol control module to parse the CPU read command to obtain a CPU data read request and a destination address of a second AI chip; using a second protocol control module, the CPU data read request and the destination address of the second AI chip are encapsulated based on a second interconnect protocol to obtain a second AI chip read command; after sending the second AI chip read command to the second AI chip, the second protocol control module parses the CPU data read result returned by the second AI chip based on the second interconnect protocol to obtain CPU data to be read; and using a storage controller module to write the CPU data to be read into CPU extended memory, so that the CPU can read the CPU data to be read through the CPU extended memory.
[0008] According to a fourth aspect of this disclosure, a data processing method is provided, comprising: after receiving a write command for a target AI chip based on a second interconnection protocol from a target AI chip, a memory expansion controller uses a second protocol control module to parse the write command to obtain data to be written to the AI chip and a data destination address of the data to be written to the AI chip; using a data transfer module, the local address of the data to be written to the AI chip is obtained according to the data destination address of the data to be written to the AI chip; and using a storage control module, the data to be written to the AI chip is written to the CPU extended memory according to the local address of the data to be written to the AI chip.
[0009] According to a fifth aspect of this disclosure, a data processing method is provided, comprising: after receiving a target AI chip read command based on a second interconnection protocol sent by a target AI chip, a memory expansion controller uses a second protocol control module to parse the target AI chip read command to obtain the data source address of the AI chip data to be read; using a data transfer module, the local address of the AI chip data to be read is obtained according to the data source address of the AI chip data to be read; using a storage control module, the AI chip data to be read is obtained from the CPU extended memory according to the local address of the AI chip data to be read; using the second protocol control module, the AI chip data to be read is encapsulated into an AI chip data read result based on the second interconnection protocol; and sending the AI chip data read result to the target AI chip for the target AI chip to read the AI chip data to be read according to the AI chip data read result.
[0010] According to a sixth aspect of this disclosure, an AI server system is provided, including a CPU, at least one AI chip, and a memory expansion controller provided in the first aspect of this disclosure.
[0011] According to a seventh aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.
[0012] According to an eighth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described above.
[0013] According to a ninth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described above.
[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0015] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:
[0016] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;
[0017] Figure 2This is a schematic diagram according to the second embodiment of the present disclosure;
[0018] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure;
[0019] Figure 4 This is a schematic diagram according to the fourth embodiment of the present disclosure;
[0020] Figure 5 This is a schematic diagram according to the fifth embodiment of the present disclosure;
[0021] Figure 6 This is a schematic diagram according to the sixth embodiment of the present disclosure;
[0022] Figure 7 This is a block diagram of an electronic device used to implement the data processing method of the embodiments of this disclosure. Detailed Implementation
[0023] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and mechanisms are omitted in the following description.
[0024] Figure 1 This is a schematic diagram based on the first embodiment of this disclosure. (See diagram below.) Figure 1 As shown, the memory expansion controller in this embodiment is used for mutual access between a CPU (Central Processing Unit) based on a first interconnection protocol and an AI (Artificial Intelligence) chip based on a second interconnection protocol.
[0025] The memory expansion controller in this embodiment can be implemented using an FPGA or a chip.
[0026] In this embodiment, the first interconnection protocol is the CXL (Compute Express Link) protocol, which is commonly supported by current CPUs; the second interconnection protocol is an interconnection protocol different from the CXL protocol, such as the UALink protocol, NVLink protocol, etc. In this embodiment, the second interconnection protocol can be collectively referred to as the XLINK protocol, which is commonly used by current AI chips.
[0027] In other words, the memory expansion controller provided in this embodiment opens up the path between the CXL interface commonly found in CPUs and the XLINK interface commonly found in AI chips. This enables communication between CPUs and AI chips based on different interconnect protocols without changing the existing CPU and AI chip designs. It eliminates the need to restrict the CPU and AI chips to use the same interconnect protocol, greatly improving the flexibility of CPU and AI chip selection.
[0028] In this embodiment, the CPU and AI chip are located within an AI server system, which can be used for training or inference of large models. The CPU within the AI server system can be used to handle complex logic control and serial computation tasks during large model training or inference, while the AI chip can be used to handle large-scale parallel computation tasks during large model training or inference.
[0029] The memory expansion controller in this embodiment includes a first protocol control module, a storage control module, a data transfer module, a second protocol control module, and CPU expansion memory; wherein, the first protocol control module communicates with the CPU based on a first interconnection protocol, and the second protocol control module communicates with the AI chip based on a second interconnection protocol.
[0030] In this embodiment, the first protocol control module is used by the memory expansion controller to parse the CPU write command or CPU read command sent by the CPU based on the first interconnection protocol (e.g., CXL protocol) to obtain the data source address and AI chip destination address of the corresponding CPU write command or the AI chip destination address of the corresponding CPU read command; that is, in this embodiment, the memory expansion controller presents itself as a CXL device to the CPU through the first protocol control module.
[0031] In this embodiment, the second protocol control module is used, on the one hand, to parse the AI chip write command or AI chip read command sent by the AI chip based on the second interconnect protocol (e.g., XLINK protocol) to obtain the data destination address of the corresponding AI chip write command or the data source address of the corresponding AI chip read command. On the other hand, it is used by the memory expansion controller to encapsulate the AI chip read command based on the AI chip destination address of the corresponding CPU read command, or to encapsulate the AI chip write command based on the second interconnect protocol based on the data to be written and the AI chip destination address of the corresponding CPU write command. That is, in this embodiment, the memory expansion controller presents itself as an XLINK device to the AI chip through the second protocol control module.
[0032] In this embodiment, the data migration module is used by the memory expansion controller to obtain the local address of the corresponding CPU extended memory based on the data source address or data destination address; that is, in this embodiment, the memory expansion controller completes the address mapping of the CPU extended memory through the data migration module. The data migration module in this embodiment can complete the address mapping according to a pre-maintained correspondence table between the source address or destination address and the local address.
[0033] In this embodiment, the storage control module is used by the memory expansion controller to obtain data to be written or data to be read from the CPU extended memory according to the local address.
[0034] In this embodiment, the CPU extended memory can be DDR (Double Data Rate Synchronous Dynamic Random Access Memory), SSD (Solid State Drive), or a combination of DDR and SSD (in which case DDR is the cache of SSD).
[0035] In this embodiment, data stored in the CPU's own memory can be moved to the CPU extended memory in the memory extension controller, so that the CPU extended memory can store the data that was originally in the CPU's own memory.
[0036] In other words, when the memory expansion controller in this embodiment performs mutual access between the CPU and the AI chip, data reading (e.g., data that the CPU wants to write to the AI chip or data that the AI chip wants to read from the CPU) or data writing (e.g., data that the AI chip wants to write to the CPU or data that the CPU wants to read from the AI chip) is completed through the CPU's expanded memory, instead of through the CPU's own memory. This enables mutual access between the CPU and the AI chip even when they do not have the same interconnection protocol.
[0037] Figure 2 This is a schematic diagram according to the second embodiment of this disclosure. (See diagram below.) Figure 2 As shown in the figure, this embodiment illustrates an AI server system based on a memory expansion controller, which includes a memory expansion controller, a CPU, and at least one AI chip.
[0038] In this embodiment, the AI chip is a processor designed specifically for artificial intelligence tasks. By optimizing the architecture and instruction set, it accelerates the operation of algorithms such as machine learning, deep learning, and neural networks. The AI chip in this embodiment may include computing units such as GPUs and TPUs.
[0039] In this embodiment, the CPU communicates with the memory expansion controller based on a first interconnect protocol (e.g., an interface based on the CXL protocol), while the AI chip communicates with the memory expansion controller based on a second interconnect protocol (e.g., an interface based on the XLINK protocol such as UALINK, NVLINK, etc.); that is, the interconnect protocols used by the CPU and the AI chip in this embodiment are different.
[0040] It is understood that the memory expansion controller and the AI chip in this embodiment also include a switching chip based on the second interconnection protocol. That is, the AI chip sends AI chip write commands or AI chip read commands to the memory expansion controller through the switching chip, or receives AI chip write commands or AI chip read commands sent by the memory expansion controller through the switching chip.
[0041] In this embodiment, if the memory expansion controller receives a CPU write command based on the first interconnect protocol sent by the CPU (the CPU write command is used by the CPU to write data in the video memory of the AI chip), the first protocol control module in the memory expansion controller is used to perform the following: parse the CPU write command based on the first interconnect protocol sent by the CPU to obtain the data source address of the data to be written by the CPU and the destination address of the first AI chip; wherein, the data source address of the data to be written by the CPU is the global source address of the data to be written by the CPU, and the destination address of the first AI chip is the address of the AI chip to which the data to be written by the CPU is to be written.
[0042] Accordingly, the data transfer module in the memory expansion controller is used to perform the following: obtain the local address of the data to be written to the CPU based on the data source address of the data to be written to the CPU obtained by the first protocol control module.
[0043] Accordingly, the storage control module in the memory expansion controller is used to perform the following: obtain the CPU data to be written from the CPU expansion memory according to the local address of the CPU data to be written obtained by the data transfer module.
[0044] Accordingly, the second protocol control module in the memory expansion controller is used to perform the following: based on the second interconnection protocol, encapsulate the CPU data to be written obtained by the storage control module and the first AI chip destination address obtained by the first protocol control module to obtain the first AI chip write command.
[0045] In this embodiment, after receiving the write command from the first AI chip, the memory expansion controller can send the write command to the first AI chip through the switching chip. This allows the first AI chip to parse the CPU write data in the first AI write command and write the obtained CPU write data into its video memory, thereby completing the operation of the CPU writing data to the AI chip.
[0046] In other words, this embodiment completes the data writing operation between the CPU and the AI chip through the cooperation between the first protocol control module, the data transfer module, the storage control module and the second protocol control module in the memory expansion controller. During the data writing process, the CPU still uses the first interconnection protocol and the AI chip still uses the second interconnection protocol, avoiding the use of the traditional PCIe (Peripheral Component Interconnect Express) interconnection protocol for communication between the two. This can improve the communication bandwidth between the CPU and the AI chip, reduce the data writing latency, and thus improve the performance when training large models or using large models for inference.
[0047] In this embodiment, if the memory expansion controller receives a CPU read command based on the first interconnect protocol sent by the CPU (the CPU read command is used by the CPU to read data in the video memory of the AI chip), the first protocol control module in the memory expansion controller is used to perform the following: parse the CPU read command based on the first interconnect protocol sent by the CPU to obtain the CPU data read requirement and the destination address of the second AI chip; wherein, the CPU data read requirement is the CPU's requirement to read specific data from the second AI chip, and the destination address of the second AI chip is the address of the second AI chip where the data to be read by the CPU is located.
[0048] Accordingly, the second protocol control module in the memory expansion controller is used to perform the following: based on the second interconnection protocol, it encapsulates the CPU data read request obtained by the first protocol control module with the destination address of the second AI chip to obtain the second AI chip read command.
[0049] In this embodiment, after receiving the read command from the second AI chip, the memory expansion controller can send the read command to the second AI chip through the switching chip, so that the second AI chip can read the data corresponding to the CPU data reading requirement in the second AI read command from the video memory.
[0050] Accordingly, the second protocol control module in the memory expansion controller is also used to perform the following: parse the CPU data read result returned by the second AI chip based on the second interconnection protocol to obtain the CPU data to be read according to the corresponding CPU data read requirement; wherein, after receiving the second AI chip read command, the second AI chip can obtain the corresponding data (such as existing data or real-time generated data) from the video memory according to the CPU data read requirement in the second AI chip read command, and send the obtained data as the CPU data read result to the memory expansion controller.
[0051] Accordingly, the storage control module in the memory expansion controller is used to perform the following: write the CPU data to be read by the CPU obtained by the second protocol control module into the CPU expansion memory, so that the CPU can read the CPU data to be read through the CPU expansion memory.
[0052] In other words, this embodiment completes the operation of the CPU reading data from the AI chip's video memory through the cooperation between the first protocol control module, the storage control module, and the second protocol control module in the memory expansion controller. During the data reading process, the CPU still uses the first interconnection protocol and the AI chip still uses the second interconnection protocol, avoiding the use of the traditional PCIe (Peripheral Component Interconnect Express) interconnection protocol for communication between the two. This can improve the communication bandwidth between the CPU and the AI chip, reduce data reading latency, and thus improve the performance when training large models or using large models for inference.
[0053] In this embodiment, if the memory expansion controller receives a target AI chip write command sent by the AI chip based on the second interconnection protocol (the target AI chip write command is used for the target AI chip to write data in the CPU), the second protocol control module in the memory expansion controller is used to perform the following: parse the target AI chip write command sent by the target AI chip based on the second interconnection protocol to obtain the AI chip to be written data and the data destination address of the AI chip to be written data; wherein, the AI chip to be written data is the data that the target AI chip wants to write into the CPU, and the data destination address is the global destination address of the corresponding AI chip to be written data.
[0054] Accordingly, the data transfer module in the memory expansion controller is used to perform the following: obtain the local address of the data to be written to the AI chip based on the data destination address of the data to be written to the AI chip obtained from the second protocol control module.
[0055] Accordingly, the storage control module in the memory expansion controller is used to perform the following: write the AI chip data to be written, obtained from the second protocol control module, into the CPU extended memory, based on the local address of the AI chip data to be written obtained from the data transfer module.
[0056] In this embodiment, after the memory expansion controller writes the data to be written by the AI chip to the CPU expansion memory through the storage control module, it can complete the operation of the target AI chip writing data to the CPU.
[0057] In other words, this embodiment completes the data writing operation between the target AI chip and the CPU through the cooperation between the data transfer module, the storage control module and the second protocol control module in the memory expansion controller. During the data writing process, the CPU still uses the first interconnect protocol and the target AI chip still uses the second interconnect protocol, avoiding the use of the traditional PCIe (Peripheral Component Interconnect Express) interconnect protocol for communication between the two. This can improve the communication bandwidth between the CPU and the target AI chip, reduce the data writing latency, and thus improve the performance when training large models or using large models for inference.
[0058] In this embodiment, if the memory expansion controller receives a target AI chip read command based on the second interconnection protocol sent by the target AI chip (the target AI chip read command is used by the target AI chip to read data from the CPU), the second protocol control module in the memory expansion controller is used to perform the following: parse the target AI chip read command based on the second interconnection protocol sent by the target AI chip to obtain the data source address of the data to be read by the AI chip; wherein, the data source address of the data to be read by the AI chip is the global source address of the data to be read by the AI chip.
[0059] Accordingly, the data transfer module in the memory expansion controller is used to perform the following: obtain the local address of the data to be read by the AI chip based on the data source address of the data to be read by the AI chip obtained by the second protocol control module.
[0060] Accordingly, the storage control module in the memory expansion controller is used to perform the following: obtain the AI chip data to be read from the CPU extended memory according to the local address of the AI chip data to be read obtained by the data transfer module.
[0061] Accordingly, the second protocol control module in the memory expansion controller is used to perform the following: based on the second interconnection protocol, encapsulate the AI chip data to be read obtained by the storage control module to obtain the AI chip data reading result.
[0062] After obtaining the AI chip data reading result, the memory expansion controller in this embodiment can send the AI chip data reading result to the target AI chip (i.e., the AI chip that sent the AI chip read command) through the switching chip, so that the target AI chip can read the AI chip data to be read in the AI chip data reading result.
[0063] In other words, this embodiment completes the operation of the target AI chip reading CPU data through the cooperation between the data transfer module, storage control module and second protocol control module in the memory expansion controller. During the data reading process, the CPU still uses the first interconnect protocol and the target AI chip still uses the second interconnect protocol, avoiding the use of the traditional PCIe (Peripheral Component Interconnect Express) interconnect protocol for communication between the two. This can improve the communication bandwidth between the CPU and the AI chip, reduce data reading latency, and thus improve the performance when training large models or using large models for inference.
[0064] Figure 3 This is a schematic diagram according to the third embodiment of this disclosure. (See diagram below.) Figure 3 As shown, this embodiment provides a data processing method based on a memory expansion controller, which specifically includes the following steps:
[0065] S301. After receiving a CPU write command based on the first interconnection protocol sent by the CPU, the memory expansion controller uses the first protocol control module to parse the CPU write command and obtain the data source address of the data to be written by the CPU and the destination address of the first AI chip.
[0066] S302. Using the data transfer module, obtain the local address of the data to be written to the CPU based on the data source address of the data to be written to the CPU;
[0067] S303. Using the storage control module, obtain the CPU data to be written from the CPU extended memory according to the local address of the CPU data to be written.
[0068] S304. Using the second protocol control module, the CPU data to be written and the destination address of the first AI chip are encapsulated based on the second interconnection protocol to obtain the first AI chip write command.
[0069] S305. Send the first AI chip write command to the first AI chip so that the first AI chip writes the CPU data to be written to the video memory.
[0070] In other words, this embodiment uses a memory expansion controller to complete the operation of the CPU writing data to the AI chip. Since the CPU still uses the first interconnect protocol and the AI chip still uses the second interconnect protocol during the data writing process, the traditional PCIe interconnect protocol is avoided for communication between the two, which can improve the communication bandwidth between the CPU and the AI chip and thus reduce the data writing latency.
[0071] In this embodiment, the first interconnection protocol is the CXL protocol, and the second interconnection protocol is an XLINK protocol, such as the UALINK protocol or NVLINK protocol, which is different from the CXL protocol.
[0072] In this embodiment, when S305 is executed, the memory expansion controller can send the write command of the first AI chip to the first AI chip through the switching chip.
[0073] Figure 4 This is a schematic diagram according to the fourth embodiment of this disclosure. (See diagram below.) Figure 4 As shown, this embodiment provides a data processing method based on a memory expansion controller, which specifically includes the following steps:
[0074] S401. After receiving a CPU read command based on the first interconnection protocol sent by the CPU, the memory expansion controller uses the first protocol control module to parse the CPU read command and obtain the CPU data reading requirements and the destination address of the second AI chip.
[0075] S402. Using the second protocol control module, the CPU data reading request and the destination address of the second AI chip are encapsulated based on the second interconnection protocol to obtain the second AI chip read command.
[0076] S403. After sending the second AI chip read command to the second AI chip, the second protocol control module is used to parse the CPU data read result returned by the second AI chip based on the second interconnection protocol to obtain the CPU data to be read.
[0077] S404. The storage controller module is used to write the data to be read by the CPU into the CPU extended memory, so that the CPU can read the data to be read by the CPU through the CPU extended memory.
[0078] In other words, this embodiment uses a memory expansion controller to complete the operation of the CPU reading data from the AI chip. Since the CPU still uses the first interconnect protocol and the AI chip still uses the second interconnect protocol during the data reading process, the traditional PCIe interconnect protocol is avoided for communication between the two, which can improve the communication bandwidth between the CPU and the AI chip and thus reduce the data reading latency.
[0079] In this embodiment, the first interconnection protocol is the CXL protocol, and the second interconnection protocol is an XLINK protocol, such as the UALINK protocol or NVLINK protocol, which is different from the CXL protocol.
[0080] Figure 5 This is a schematic diagram according to the fifth embodiment of this disclosure. (See diagram below.) Figure 5As shown, this embodiment provides a data processing method based on a memory expansion controller, which specifically includes the following steps:
[0081] S501 After receiving the target AI chip write command based on the second interconnection protocol sent by the target AI chip, the memory expansion controller uses the second protocol control module to parse the target AI chip write command to obtain the AI chip data to be written and the data destination address of the AI chip data to be written.
[0082] S502. Using the data transfer module, obtain the local address of the data to be written to the AI chip based on the data destination address data of the data to be written to the AI chip;
[0083] S503. Using the storage control module, the AI chip data to be written is written to the CPU extended memory according to the local address of the AI chip data to be written.
[0084] In other words, this embodiment uses a memory expansion controller to complete the operation of writing data from the target AI chip to the CPU. Since the CPU still uses the first interconnect protocol and the target AI chip still uses the second interconnect protocol during the data writing process, the traditional PCIe interconnect protocol is avoided for communication between the two, which can improve the communication bandwidth between the CPU and the target AI chip and thus reduce the data writing latency.
[0085] In this embodiment, the first interconnection protocol is the CXL protocol, and the second interconnection protocol is an XLINK protocol, such as the UALINK protocol or NVLINK protocol, which is different from the CXL protocol.
[0086] After S503 is executed in this embodiment, the CPU can access the CPU extended memory to read and process the AI chip data to be written to the target AI chip.
[0087] Figure 6 This is a schematic diagram according to the sixth embodiment of this disclosure. (See diagram below.) Figure 6 As shown, this embodiment provides a data processing method based on a memory expansion controller, which specifically includes the following steps:
[0088] S601. After receiving a target AI chip read command based on the second interconnection protocol sent by the target AI chip, the memory expansion controller uses the second protocol control module to parse the target AI chip read command and obtain the data source address of the data to be read by the AI chip.
[0089] S602. Using the data transfer module, obtain the local address of the data to be read by the AI chip based on the data source address of the data to be read by the AI chip;
[0090] S603. Using the storage control module, the AI chip data to be read is obtained from the CPU extended memory according to the local address of the AI chip data to be read.
[0091] S604. Using the second protocol control module, the AI chip data to be read is encapsulated into an AI chip data reading result based on the second interconnection protocol;
[0092] S605. The AI chip data reading result is sent to the target AI chip so that the target AI chip can read the AI chip data to be read based on the AI chip data reading result.
[0093] In other words, this embodiment uses a memory expansion controller to complete the operation of the target AI chip reading data from the CPU. Since the CPU still uses the first interconnect protocol and the target AI chip still uses the second interconnect protocol during the data reading process, the traditional PCIe interconnect protocol is avoided for communication between the two, which can improve the communication bandwidth between the CPU and the target AI chip and thus reduce the data reading latency.
[0094] In this embodiment, the first interconnection protocol is the CXL protocol, and the second interconnection protocol is an XLINK protocol, such as the UALINK protocol or NVLINK protocol, which is different from the CXL protocol.
[0095] In this embodiment, when S605 is executed, the memory expansion controller can send the AI chip data reading result to the target AI chip through the switching chip.
[0096] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0097] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0098] like Figure 7 The diagram shown is a block diagram of an electronic device for a data processing method according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0099] like Figure 7As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.
[0100] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of displays, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0101] Computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 701 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing methods may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 708.
[0102] In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by computing unit 701, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, computing unit 701 may be configured to perform the data processing method by any other suitable means (e.g., by means of firmware).
[0103] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.
[0104] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0105] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0106] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for showing information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0107] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0108] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.
[0109] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.
[0110] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.
Claims
1. A memory expansion controller, comprising: The system comprises a first protocol control module, a storage control module, a data transfer module, a second protocol control module, and CPU extended memory. The first protocol control module communicates with the CPU based on the first interconnection protocol, and the second protocol control module communicates with the AI chip based on the second interconnection protocol.
2. The memory expansion controller according to claim 1, wherein, The first protocol control module is used for: The CPU write command sent by the CPU based on the first interconnection protocol is parsed to obtain the data source address of the data to be written by the CPU and the destination address of the first AI chip.
3. The memory expansion controller according to claim 2, wherein, The data transfer module is used for: Based on the data source address of the CPU data to be written obtained from the first protocol control module, the local address of the CPU data to be written is obtained.
4. The memory expansion controller according to claim 3, wherein, The storage control module is used for: Based on the local address of the CPU data to be written obtained by the data transfer module, the CPU data to be written is retrieved from the CPU extended memory.
5. The memory expansion controller according to claim 4, wherein, The second protocol control module is used for: Based on the second interconnection protocol, the CPU data to be written obtained by the storage control module and the first AI chip destination address obtained by the first protocol control module are encapsulated to obtain the first AI chip write command. The first AI chip write command is used by the first AI chip to write the CPU data to be written to the first AI chip.
6. The memory expansion controller according to claim 1, wherein, The first protocol control module is also used for: The CPU read command sent by the CPU based on the first interconnection protocol is parsed to obtain the CPU data read request and the destination address of the second AI chip.
7. The memory expansion controller according to claim 6, wherein, The second protocol control module is also used for: Based on the second interconnection protocol, the CPU data reading request obtained by the first protocol control module and the destination address of the second AI chip are encapsulated to obtain the second AI chip read command. The second AI chip read command is used by the second AI chip to obtain the CPU data read result; The CPU data read results returned by the second AI chip based on the second interconnection protocol are analyzed to obtain the CPU data to be read corresponding to the CPU data read requirements.
8. The memory expansion controller according to claim 7, wherein, The storage control module is also used for: The CPU data to be read obtained by the second protocol control module is written into the CPU extended memory so that the CPU can read the CPU data through the CPU extended memory.
9. The memory expansion controller according to claim 1, wherein, The second protocol control module is also used for: The write command sent by the target AI chip based on the second interconnection protocol is parsed to obtain the data to be written to the AI chip and the data destination address of the data to be written to the AI chip.
10. The memory expansion controller according to claim 9, wherein, The data transfer module is also used for: Based on the data destination address of the AI chip to be written obtained from the second protocol control module, the local address of the AI chip to be written is obtained.
11. The memory expansion controller according to claim 10, wherein, The storage control module is also used for: Based on the local address of the AI chip data to be written obtained by the data transfer module, the AI chip data to be written obtained by the second protocol control module is written into the CPU extended memory.
12. The memory expansion controller according to claim 1, wherein, The second protocol control module is also used for: The target AI chip read command sent by the target AI chip based on the second interconnection protocol is parsed to obtain the data source address of the data to be read by the AI chip.
13. The memory expansion controller according to claim 12, wherein, The data transfer module is also used for: Based on the data source address of the AI chip to be read obtained from the second protocol control module, the local address of the AI chip to be read is obtained.
14. The memory expansion controller according to claim 13, wherein, The storage control module is also used for: Based on the local address of the AI chip data to be read obtained by the data transfer module, the AI chip data to be read is retrieved from the CPU extended memory.
15. The memory expansion controller according to claim 14, wherein, The second protocol control module is also used for: Based on the second interconnection protocol, the AI chip data to be read obtained by the storage control module is encapsulated to obtain the AI chip data reading result; The AI chip data reading result is sent to the target AI chip so that the target AI chip can read the AI data to be read based on the AI chip data reading result.
16. A data processing method based on a memory expansion controller according to any one of claims 1-15, comprising: After receiving a CPU write command based on the first interconnection protocol sent by the CPU, the memory expansion controller uses the first protocol control module to parse the CPU write command and obtain the data source address of the data to be written by the CPU and the destination address of the first AI chip. Using the data transfer module, the local address of the data to be written to the CPU is obtained based on the data source address of the data to be written to the CPU. The storage control module is used to retrieve the data to be written to the CPU from the CPU extended memory based on the local address of the data to be written to the CPU. Using the second protocol control module, the data to be written to the CPU and the destination address of the first AI chip are encapsulated based on the second interconnection protocol to obtain the write command of the first AI chip; The first AI chip write command is sent to the first AI chip so that the first AI chip writes the data to be written by the CPU into the video memory.
17. A data processing method based on a memory expansion controller according to any one of claims 1-15, comprising: After receiving a CPU read command based on the first interconnection protocol sent by the CPU, the memory expansion controller uses the first protocol control module to parse the CPU read command and obtain the CPU data reading requirements and the destination address of the second AI chip. Using the second protocol control module, the CPU data reading request and the destination address of the second AI chip are encapsulated based on the second interconnection protocol to obtain the second AI chip read command; After sending the second AI chip read command to the second AI chip, the second protocol control module is used to parse the CPU data read result returned by the second AI chip based on the second interconnection protocol to obtain the CPU data to be read. The storage controller module writes the data to be read by the CPU into the CPU extended memory, so that the CPU can read the data to be read through the CPU extended memory.
18. A data processing method based on a memory expansion controller according to any one of claims 1-15, comprising: After receiving a write command from the target AI chip based on the second interconnection protocol, the memory expansion controller uses the second protocol control module to parse the write command and obtain the data to be written to the AI chip and the data destination address of the data to be written to the AI chip. Using the data transfer module, the local address of the data to be written to the AI chip is obtained based on the data destination address of the data to be written to the AI chip; Using the storage control module, the data to be written to the AI chip is written to the CPU extended memory according to the local address of the data to be written to the AI chip.
19. A data processing method based on a memory expansion controller according to any one of claims 1-15, comprising: After receiving a target AI chip read command based on the second interconnection protocol sent by the target AI chip, the memory expansion controller uses the second protocol control module to parse the target AI chip read command and obtain the data source address of the data to be read by the AI chip; Using the data transfer module, the local address of the data to be read by the AI chip is obtained based on the data source address of the data to be read by the AI chip; The storage control module is used to retrieve the AI chip data to be read from the CPU extended memory according to the local address of the data to be read by the AI chip. Using the second protocol control module, the AI chip data to be read is encapsulated into an AI chip data reading result based on the second interconnection protocol; The AI chip data reading result is sent to the target AI chip so that the target AI chip can read the data to be read from the AI chip based on the AI chip data reading result.
20. An AI server system, comprising a CPU, at least one AI chip, and a memory expansion controller as described in any one of claims 1-15.
21. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 16-19.
22. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 16-19.
23. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 16-19.