AI computing terminal and method based on RISCV processor and Smart NIC

Abstract

The invention provides an AI computing terminal and method based on an RISCV processor and a Smart NIC, and relates to the field of computing terminals. The AI computing terminal comprises an integrated storage computing chip based on RISC-V, an NPU (Network Processing Unit) and a UCIE memory, a Smart NIC (Network Interface Card) and RDMA (Remote Direct Memory Access) interconnection chip and a DDR5 (Double Data Rate Receptor) RDIMM (Radio Data In-line Memory Module) memory bank socket; the integrated storage and calculation chip is internally provided with two calculation dies and two UCIE memories, each calculation die comprises an RISC-V and an NPU, the speed of each UCIE memory is higher than the specified speed, the calculation dies and the UCIE memories are integrated on one chip in an MCM multi-die sealing mode, the two calculation dies are interconnected through a UCIE bus, and the calculation dies and the UCIE memories are interconnected through UCIE and used for providing memory bandwidth higher than the specified speed; and a larger AI model can be loaded and operated.

Description

Technical Field

[0001] This application relates to the field of computing terminal technology, and in particular to an AI computing terminal and method based on a RISC-CV processor and a Smart NIC. Background Technology

[0002] Currently, existing AI computing terminals mainly adopt x86 or Arm architecture, integrating an on-chip NPU to provide computing power, using LPDDR5X memory chips packaged via MCM to provide computing memory, and employing an external Mellanox network card via PCIe bus for interconnection and communication. Some AI computing terminals use traditional x86 architecture CPUs, connecting an external GPU computing card via PCIe bus to provide computing power, utilizing the GPU's built-in video memory as computing memory, and similarly using Mellanox CX series network cards for interconnection. However, the computing memory capacity and bandwidth of these current AI computing terminals are relatively small, resulting in an inability to load and run larger AI models. Summary of the Invention

[0003] The purpose of this invention is to provide an AI computing terminal and method based on a RISCV processor and a Smart NIC, so as to solve the technical problem that the current AI computing terminals have small computing memory capacity and low bandwidth, which makes it impossible to load and run larger AI models.

[0004] In a first aspect, this application provides an AI (Artificial Intelligence) computing terminal based on a RISC-V (Reduced Instruction Set Computer V, an open-source instruction set architecture) processor and a Smart NIC (Network Interface Controller). The AI ​​computing terminal includes: an integrated storage computing chip based on RISC-V, an NPU (Neural-network Processing Unit), and UCIE (Universal Chiplet Interconnect Express) memory; a Smart NIC and an RDMA (remote direct memory access) interconnect chip; and a DDR5 RDIMM (Registered Dual In-line Memory Module) memory module socket. The integrated storage computing chip is provided with two computing dies, each containing the RISC-V and the NPU, and two UCIE memories with a speed higher than the specified speed. The computing dies and the UCIE memories are all integrated on a single chip through MCM (Multi-Chip Module) multi-die packaging. The two computing dies are interconnected through a UCIE bus, and the computing dies and the UCIE memories are interconnected through UCIE, which is used to provide memory bandwidth higher than the specified speed. The integrated storage computing chip has a high-speed external bus called PCIe Gen5 RC and two standard DDR5 memory interfaces corresponding to the DDR5 RDIMM memory module socket. The Smart NIC dual-port interconnection interface in the Smart NIC and RDMA interconnection chip supports RDMA. The maximum bandwidth of the external interconnection switch of the AI ​​computing terminal is a specified bandwidth. Multiple AI computing terminals are connected in series at a rate of half the specified bandwidth, so that multiple AI computing terminals can jointly run an AI model through collaborative work and memory sharing.

[0005] In one possible implementation, the AI ​​computing terminal further includes: Multi-phase power supply module, reserved M.2 hard drive storage interface, and external PCIe switch device slot; The M.2 hard drive storage interface corresponds to an M.2 solid-state drive (SSD) with two M.2 slots. The M.2 SSD supports standard SSD specifications, NVMe, and PCIe Gen5 x4, and is used for installing operating systems and storing file data.

[0006] In one possible implementation, the Smart NIC dual-port interconnect interface in the RDMA interconnect chip includes an interface 2xQSFP; The integrated storage computing chip, the Smart NIC and RDMA interconnect chip, the DDR5 RDIMM memory module socket, the multi-phase power supply module, the M.2 hard drive storage interface, and the components corresponding to the external PCIe Switch device slot are all mounted on a single PCB by soldering.

[0007] In one possible implementation, the AI ​​computing terminal further includes a heat dissipation component; the heat dissipation component includes a cold plate type liquid cooling main component and an air cooling auxiliary component; The cold plate type liquid cooling heat dissipation main component indirectly transfers the heat of the heat-generating components to the cooling liquid enclosed in the circulation pipeline through the cold plate, and then conducts the heat out through the cooling liquid. The working fluid is used as an intermediate heat transfer medium to transfer the heat from the hot zone to the far end for cooling. The working liquid of the cold plate type liquid cooling heat dissipation main component is separated from the object being cooled, and the working liquid does not directly contact the electronic device. The working liquid and the electronic device are connected by the liquid cooling plate as a heat conduction component to transfer the heat of the object being cooled to the cooling liquid.

[0008] In one possible implementation, the DDR5 RDIMM memory socket is a standardized DDR5 R-DIMM, and the two standard memory slots corresponding to the DDR5 RDIMM memory socket support standard server memory RDIMM and ECC (Error Checking and Correcting).

[0009] In one possible implementation, the integrated storage computing chip is powered by a multi-phase power supply; The number of data channel lanes of the PCIe Gen5 RC in the integrated storage computing chip is a specified number, and the specified number of PCIe Gen5 RCs can be split in multiple ways.

[0010] Secondly, this application provides an AI computing method based on a RISC-V processor and a Smart NIC, applied to the AI ​​computing terminal described in the first aspect above; the method includes: The Smart NIC and RDMA interconnect chip are interconnected to obtain external interconnection data; AI calculations are performed on the external interconnection data through the integrated storage and computing chip based on RISC-V, NPU and UCIE memory to obtain AI calculation results; The AI ​​calculation results are cached through the DDR5 RDIMM memory module socket to be stored on a solid-state drive (SSD) or transmitted over a network to other devices besides the AI ​​computing terminal.

[0011] Thirdly, this application provides an AI computing system based on a RISC-V processor and a Smart NIC, applied to the AI ​​computing terminal described in the first aspect above; the system includes: The external interconnection module is used to interconnect with the RDMA interconnection chip through the Smart NIC to obtain external interconnection data; The computing module is used to perform AI calculations based on the external interconnection data through the integrated storage computing chip based on RISC-V, NPU and UCIE memory, and obtain AI calculation results. The memory module is used to cache the AI ​​calculation results through the DDR5 RDIMM memory module socket, so as to store them on a solid-state drive (SSD) or transmit them over a network to other devices other than the AI ​​computing terminal.

[0012] Fourthly, this application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the method described in the second aspect above.

[0013] Fifthly, this application also provides a computer-readable storage medium storing computer-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method described in the second aspect above.

[0014] This application brings the following beneficial effects: This application provides an AI computing terminal and method based on a RISC-V processor and a Smart NIC. The AI ​​computing terminal includes: an integrated storage computing chip based on RISC-V, NPU, and UCIE memory; a Smart NIC and RDMA interconnect chip; and a DDR5 RDIMM memory module socket. The integrated storage computing chip has two computing dies, each containing RISC-V and an NPU, and two UCIE memories with speeds exceeding a specified limit. The computing dies and UCIE memories are all integrated onto a single chip using a multi-die packaging method (MCM). The two computing dies are interconnected via a UCIE bus, and the computing dies and UCIE memories are interconnected via UCIE, providing memory bandwidth exceeding the specified speed. The integrated storage computing chip has a high-speed external bus of PCIe Gen5 RC and two standard DDR5 memory interfaces corresponding to the DDR5 RDIMM memory module socket. The Smart NIC and RDMA interconnect chip... The NIC dual-port interconnect interface supports RDMA. The maximum bandwidth of the external interconnect switch for AI computing terminals is a specified bandwidth. Multiple AI computing terminals are connected in series at half the specified bandwidth to enable multiple AI computing terminals to jointly run an AI model through collaborative work and memory sharing. In this solution, multiple computing cores (RISC-V + NPU) and high-speed computing memory are integrated into a single chip using multi-chip module (MCM) packaging technology. The chip also supports external expansion of computing memory capacity via standard DIMM memory modules. Efficient interconnection between multiple terminals is achieved using a PCIe bus-connected Smart NIC. This solution deeply integrates the RISC-V core and NPU to construct a hybrid memory hierarchy architecture where high-speed memory provides bandwidth and low-speed memory provides capacity, thus balancing high throughput and large-scale model loading requirements. It uses RISC-V + NPU as the core computing unit, Smart NIC as the interconnect channel interface, and a hybrid high-speed and low-speed memory solution. It integrates the NPU within the open-source RISC-V computing architecture and employs high-speed UCIe memory and standard DDR5. The hybrid memory solution of RDIMM server memory, along with the SmartNIC that supports RDMA and high speed, enables AI computing terminals to have high-speed and large-capacity memory configurations and supports multi-machine collaborative computing. Multiple AI computing terminals can be connected in series at half the maximum bandwidth to enable multiple AI computing terminals to work together and share memory to run larger AI models. This solves the technical problem that the current AI computing terminals have small computing memory capacity and low bandwidth, which makes it impossible to load and run larger AI models.

[0015] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a structural diagram illustrating the overall framework of the entire AI computing terminal provided in an embodiment of this application; Figure 2 A schematic diagram of the structure of an integrated storage computing chip based on RISC-V, NPU and UCIE memory provided in an embodiment of this application; Figure 3 A schematic diagram of the power supply structure provided in the embodiments of this application; Figure 4 This is a schematic diagram of a cold plate heat dissipation system provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The terms "comprising" and "having," and any variations thereof, used in the embodiments of this application, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0020] Currently, AI computing terminals based on x86 or Arm architectures have relatively small computing memory capacities. Solutions using MCM-packaged LPDDR5X memory chips are limited by package size, power consumption, and cost, typically integrating only 4-8 memory chips, resulting in a strictly limited total capacity that generally struggles to exceed 128GB. The technical impact is that when running large-scale AI inference models (such as LLMs with hundreds of billions of parameters) or complex computational tasks requiring high-frequency data exchange, insufficient memory capacity will prevent the model from loading completely, forcing the system to frequently swap data in and out, severely reducing computational efficiency. Furthermore, the upper limit of memory bandwidth restricts computational performance: while LPDDR5X has high bandwidth, the actual usable bandwidth is still significantly limited by the number of packaged chips and memory controller design, making it difficult to match the peak data throughput demands of multi-core NPUs or high-performance CPUs. The technical impact is that in bandwidth-sensitive tasks (such as real-time video analysis and high-frame-rate inference), insufficient memory bandwidth will create a data path bottleneck, causing computing cores to idle and restricting overall computing power. The overall solution also suffers from a low localization rate of core components.

[0021] Traditional x86 architecture CPUs in AI computing terminals separate compute memory from system memory, resulting in low resource utilization: the external DIMM memory of the CPU is only used by the system and CPU for computation and cannot be directly used as GPU compute memory; the GPU relies on its own video memory, and the two are physically isolated. The technical impact is that when GPU video memory is insufficient, data copying between CPU memory and GPU video memory must be performed via PCIe, introducing additional latency and bandwidth overhead; simultaneously, system memory cannot dynamically replenish GPU video memory capacity, causing resource redundancy and waste. Moreover, the limited video memory capacity of a single GPU makes it impossible to load large models: mainstream GPU video memory capacities are generally between 24GB and 80GB, making it difficult for a single GPU to independently load ultra-large-scale AI models of hundreds of GB or even TB levels. The technical impact is that to load large models, multi-card parallelism or model splitting strategies must be adopted, increasing system complexity, communication overhead, and power consumption; at the same time, video memory bandwidth becomes a new bottleneck in multi-card collaboration. Furthermore, the low system integration limits deployment scenarios: CPU, GPU, and Smart NIC all use standard PCIe slots for connection, requiring independent PCB space, power supply, and heat dissipation design. The technical impact is that the space occupation is large, making it difficult to meet the deployment requirements of space-constrained scenarios such as edge computing and embedded systems; the heat dissipation design is complex, as the heat dissipation of multiple independent components requires overall coordination of airflow and thermal management, increasing the difficulty and cost of system design; and power consumption control is difficult, as the power consumption of discrete components is superimposed, making it difficult to achieve fine-grained power consumption management and limiting the improvement of energy efficiency ratio.

[0022] Therefore, existing AI computing terminals are limited by both bandwidth and capacity. The small memory capacity and low bandwidth of current products prevent the loading and running of larger AI models. Furthermore, the lack of high RDMA interconnect bandwidth hinders the efficient distributed computing due to the absence of high-speed data interconnect channels during multi-terminal collaborative work. Moreover, the lack of domestic production of core components means that current solutions rely on non-domestic components for the computing core or interconnect chips, failing to meet the requirements for secure, reliable, and self-reliant industrial implementation.

[0023] Based on this, this application provides an AI computing terminal and method based on a RISCV processor and a Smart NIC. This method can solve the technical problem that the current AI computing terminals have small computing memory capacity and low bandwidth, which makes it impossible to load and run larger AI models.

[0024] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0025] This application provides a schematic diagram of the structure of an AI computing terminal based on a RISC-CV processor and a Smart NIC. For example... Figure 1 As shown, the AI ​​computing terminal includes: an integrated storage computing chip based on RISC-V, NPU, and UCIE memory; a Smart NIC and RDMA interconnect chip; and a DDR5 RDIMM memory socket. The integrated storage computing chip contains two computing dies, each containing RISC-V and an NPU, and two UCIE memories with speeds exceeding a specified limit. The computing dies and UCIE memories are integrated onto a single chip using a multi-die packaging method (MCM). The two computing dies are interconnected via a UCIE bus, and the computing dies and UCIE memories are interconnected via UCIE, providing memory bandwidth exceeding the specified speed. The integrated storage computing chip's external high-speed bus is PCIe Gen5 RC, and the DDR5 RDIMM memory socket corresponds to two standard DDR5 memory interfaces. The Smart NIC dual-port interconnect interface in the Smart NIC and RDMA interconnect chip supports RDMA. The maximum bandwidth of the AI ​​computing terminal's external interconnect switch is the specified bandwidth. Multiple AI computing terminals are connected in series at half the specified bandwidth, enabling multiple AI computing terminals to collaboratively run an AI model through collaborative work and memory sharing.

[0026] like Figure 2As shown, the RISC-V + NPU + UCIE Memory integrated computing chip (i.e., an integrated storage computing chip based on RISC-V, NPU, and UCIE memory) mainly consists of four dies: two computing dies (RISC-V + NPU) and two high-speed UCIE memory dies. The integrated computing chip's external high-speed bus is a 32-lane PCIe Gen5 RC (supporting multiple splitting methods) and two standard DDR5 memory interfaces. In other words, this integrated computing chip contains two computing dies and two high-speed UCIE memories, packaged together on a single chip using MCM multi-die packaging. The computing dies are interconnected via the UCIE bus, and the computing dies and high-speed UCIE memories are interconnected via UCIE, providing a total high-speed memory bandwidth of 1.6TB / s.

[0027] For example, the CPU is a 24-core Risc-V, 2GHz; the NPU is a 4-core NPU that can provide 80TOPS int8 fp8 computing power support; the PCIe is Gen5 32Gbps with a total of 32 lanes, and RC supports multiple specifications such as 2x16, 4x8, 1x16 + 2x8, 8x4, etc.; the DDR5 is 2-ch 6400Mbps, and supports up to 256G RDIMM; the UCIE memory has a total capacity of 40GB.

[0028] For Smart NIC dual-port interconnect interfaces, such as Figure 1 As shown, the interface is 2xQSFP, and the speed can support 2x200Gbps or 1x400Gbps; it supports RDMA, and the maximum bandwidth of the switch that can be interconnected by a single AI computing terminal is 400Gbps (i.e., the maximum bandwidth is the specified bandwidth). Multiple AI computing terminals can be connected in series at a speed of 200Gbps (i.e., half of the maximum bandwidth of 400Gbps) to achieve the purpose of multiple AI computing terminals working together. Memory sharing can be achieved, thereby running larger AI models.

[0029] Regarding the integrated computing capabilities of RISC-V and NPU, this application embodiment is based on the next-generation open-source RISC-V computing architecture. This AI computing terminal device innovatively integrates an NPU and adopts a hybrid memory solution combining high-speed UCIe Memory and standard DDR5 RDIMM server memory. It also features a Smart NIC supporting RDMA with speeds up to 400Gbps. This device boasts a revolutionary computing system, high-speed and large-capacity memory configuration, supports multi-machine collaborative computing, and all core components are domestically produced.

[0030] The solution provided in this application is the first to achieve deep integration of the RISC-V core and NPU, constructing a hybrid memory hierarchy architecture where high-speed memory provides bandwidth and low-speed memory provides capacity, thus balancing high throughput and large-scale model loading requirements. Furthermore, both the computing chip and the interconnect chip are domestically produced, filling the technological gaps in existing products regarding RISC-V computing core support, limited memory bandwidth and capacity, and inability to load large models. This application, using RISC-V+NPU as the core computing unit and Smart NIC as the interconnect interface, combined with a high-speed and low-speed hybrid memory solution, possesses complete domestic production capabilities and can be widely applied to secure and reliable AI inference computing and other related fields.

[0031] This application proposes a novel computing architecture for domestically produced AI computing terminals based on the RISC-V architecture. Addressing the current technological gap of lacking a comprehensive solution that combines high-bandwidth, large-capacity computing memory, high RDMA interconnect bandwidth, and collaborative computing capabilities between RISC-V and NPU, this solution fills the gap in the field. Based on multi-chip module (MCM) packaging technology, it integrates multiple computing cores (RISC-V + NPU) and high-speed computing memory onto a single chip. Simultaneously, the chip supports external expansion of computing memory capacity via standard DIMM memory modules and utilizes a PCIe bus to connect an external smart NIC, enabling efficient interconnection between multiple terminals.

[0032] In some embodiments, such as Figure 1 As shown, the AI ​​computing terminal also includes: a multi-phase power supply module, a reserved M.2 hard drive storage interface, and an external PCIe Switch device slot; the M.2 hard drive storage interface corresponds to an M.2 solid-state drive (SSD) with two M.2 slots. The M.2 SSD supports standard SSD specifications, NVMe, and PCIe Gen 5 x4, and is used for installing operating systems and storing file data.

[0033] For example, for M.2 solid-state drives (SSDs), there are a total of 2 M.2 slots, supporting standard 2230, 2280, and 22110 form factors for SSDs, supporting NVMe and PCIe Gen5 x4, used for installing operating systems and storing file data.

[0034] In some embodiments, such as Figure 1As shown, the Smart NIC dual-port interconnect interface in the Smart NIC and RDMA interconnect chip includes interface 2xQSFP; the aforementioned integrated storage computing chip, Smart NIC and RDMA interconnect chip, DDR5 RDIMM memory module socket, multi-phase power supply module, M.2 hard drive storage interface, and the corresponding components of the external PCIe Switch device slot are all mounted on a PCB by soldering.

[0035] In some embodiments, the AI ​​computing terminal further includes a heat dissipation component; the heat dissipation component includes a cold plate type liquid cooling main component and an air cooling auxiliary component; the cold plate type liquid cooling main component indirectly transfers the heat of the heat-generating components to the cooling liquid enclosed in the circulation pipeline through the cold plate, and the cooling liquid conducts the heat out, so as to use the working fluid as an intermediate heat transfer medium to transfer the heat from the hot zone to the far end for cooling; the working liquid corresponding to the cold plate type liquid cooling main component is separated from the object being cooled, and the working liquid does not directly contact the electronic device, and the working liquid and the electronic device are connected by the liquid cooling plate as a heat conduction component to transfer the heat of the object being cooled to the cooling liquid.

[0036] like Figure 4 As shown, the main heat dissipation components adopt cold plate liquid cooling. Cold plate liquid cooling is a mature solution with strong heat dissipation capacity, supporting rapid heat removal from the main chip and maintaining its stable operation. The entire system uses cold plate cooling as the primary method, supplemented by air cooling. The cold plate liquid cooling part indirectly transfers heat from the heat-generating components to the cooling liquid enclosed in a circulating pipeline through a cold plate (usually a closed cavity made of a high thermal conductivity metal such as copper or aluminum). The cooling liquid then carries away the heat. The technical principle of cold plate liquid cooling is to use the working fluid as an intermediate heat transfer medium, transferring heat from the hot zone to the far end for cooling. In this technology, the working fluid is separated from the object being cooled; the working fluid does not directly contact the electronic components, but rather transfers the heat from the object being cooled to the cooling liquid through efficient heat conduction components such as the liquid cooling plate. Therefore, cold plate liquid cooling technology is also known as indirect liquid cooling technology. By adopting a cold plate liquid cooling solution for the core components, the heat dissipation efficiency of the AI ​​computing terminal is improved, and the entire AI device is compact, small in size, and flexible in expansion.

[0037] In some embodiments, the DDR5 RDIMM memory socket is a standardized DDR5 R-DIMM, and the two standard memory slots corresponding to the DDR5 RDIMM memory socket support standard server memory RDIMMs and ECC. For example, the two standard memory slots can support standard server memory RDIMMs with a speed of 6400Mbps, a total maximum capacity of 256GB, a total bandwidth of 102.4 GB / s, and support ECC.

[0038] In some embodiments, the integrated storage computing chip is powered by a multi-phase power supply; the number of data channel lanes in the PCIe Gen5 RC of the integrated storage computing chip is a specified number, and the specified number of PCIe Gen5 RCs can be configured in multiple ways. For example, such as Figure 3 As shown, the core power supply adopts the standard industry multi-phase power supply, which can support the high power and high transient power demand of the main chip, that is, realize the high current and high transient demand of the main chip.

[0039] This application provides an AI computing method based on a RISC-CV processor and a Smart NIC. The method is applied to the aforementioned AI computing terminal based on a RISC-CV processor and a Smart NIC. The AI ​​computing method based on a RISC-CV processor and a Smart NIC includes: In step S110, external interconnection is performed through the Smart NIC and the RDMA interconnection chip to obtain external interconnection data.

[0040] Step S120: AI calculations are performed on the interconnected data using an integrated storage computing chip based on RISC-V, NPU and UCIE memory to obtain AI calculation results.

[0041] In step S130, the AI ​​calculation results are cached through the DDR5 RDIMM memory module socket to be stored in a solid-state drive (SSD) or transmitted over a network to other devices besides the AI ​​computing terminal.

[0042] The AI ​​computing method based on RISCV processor and Smart NIC provided in this application embodiment has the same technical features as the AI ​​computing terminal based on RISCV processor and Smart NIC provided in the above embodiments, so it can also solve the same technical problems and achieve the same technical effects.

[0043] This application also provides an AI computing system based on a RISCV processor and a Smart NIC. This system can be applied to the aforementioned AI computing terminal based on a RISCV processor and a Smart NIC. The AI ​​computing system based on a RISCV processor and a Smart NIC includes: The external interconnection module is used to interconnect with the RDMA interconnection chip through the Smart NIC to obtain external interconnection data; The computing module is used to perform AI calculations based on the external interconnection data through the integrated storage computing chip based on RISC-V, NPU and UCIE memory, and obtain AI calculation results. The memory module is used to cache the AI ​​calculation results through the DDR5 RDIMM memory module socket, so as to store them on a solid-state drive (SSD) or transmit them over a network to other devices other than the AI ​​computing terminal.

[0044] The AI ​​computing system based on RISCV processor and Smart NIC provided in this application embodiment has the same technical features as the AI ​​computing method based on RISCV processor and Smart NIC provided in the above embodiments, so it can also solve the same technical problems and achieve the same technical effects.

[0045] This application provides an electronic device, which includes a processor and a memory. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the method provided in the above embodiments.

[0046] Electronic devices also include: a bus and a communication interface, with the processor, communication interface and memory connected via the bus; the processor is used to execute executable modules stored in the memory, such as computer programs.

[0047] The memory may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0048] The bus can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be categorized as an address bus, data bus, control bus, etc. For ease of representation, a single bidirectional arrow can be used, but this does not imply that there is only one bus or one type of bus.

[0049] The memory is used to store programs. After receiving an execution instruction, the processor executes the program. The method executed by the apparatus defined by the process disclosed in any of the foregoing embodiments of this application can be applied to the processor or implemented by the processor.

[0050] The processor may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0051] Corresponding to the above-described AI computing method based on RISCV processor and Smart NIC, this application embodiment also provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are invoked and executed by the processor, the computer-executable instructions cause the processor to perform the steps of the above-described AI computing method based on RISCV processor and Smart NIC.

[0052] The AI ​​computing system based on a RISCV processor and Smart NIC provided in this application embodiment can be specific hardware on a device or software or firmware installed on the device. The device provided in this application embodiment has the same implementation principle and technical effects as the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0053] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0054] For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0055] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0056] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0057] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the AI ​​computing method based on the RISC-CV processor and SmartNIC described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0058] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0059] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. An AI computing terminal based on a RISC-V processor and a Smart NIC, characterized in that, The AI ​​computing terminal includes: an integrated storage computing chip based on RISC-V, NPU and UCIE memory, a Smart NIC and RDMA interconnect chip, and a DDR5 RDIMM memory module socket; The integrated storage computing chip is provided with two computing dies, each containing the RISC-V and the NPU, and two UCIE memories with a speed higher than the specified speed. The computing dies and the UCIE memories are all concentrated on one chip through MCM multi-die packaging. The two computing dies are interconnected through a UCIE bus, and the computing dies and the UCIE memories are interconnected through UCIE, which is used to provide memory bandwidth higher than the specified speed. The integrated storage computing chip has a high-speed external bus called PCIe Gen5 RC and two standard DDR5 memory interfaces corresponding to the DDR5 RDIMM memory module socket. The Smart NIC dual-port interconnection interface in the Smart NIC and RDMA interconnection chip supports RDMA. The maximum bandwidth of the external interconnection switch of the AI ​​computing terminal is a specified bandwidth. Multiple AI computing terminals are connected in series at a rate of half the specified bandwidth, so that multiple AI computing terminals can jointly run an AI model through collaborative work and memory sharing.

2. The AI ​​computing terminal according to claim 1, characterized in that, The AI ​​computing terminal also includes: Multi-phase power supply module, reserved M.2 hard drive storage interface, and external PCIe switch device slot; The M.2 hard drive storage interface corresponds to an M.2 solid-state drive (SSD) with two M.2 slots. The M.2 SSD supports standard SSD specifications, NVMe, and PCIe Gen5 x4, and is used for installing operating systems and storing file data.

3. The AI ​​computing terminal according to claim 2, characterized in that, The Smart NIC dual-port interconnect interface in the Smart NIC and RDMA interconnect chip includes an interface 2xQSFP; The integrated storage computing chip, the Smart NIC and RDMA interconnect chip, the DDR5 RDIMM memory module socket, the multi-phase power supply module, the M.2 hard drive storage interface, and the components corresponding to the external PCIe Switch device slot are all mounted on a single PCB by soldering.

4. The AI ​​computing terminal according to claim 1, characterized in that, The AI ​​computing terminal also includes a heat dissipation component; the heat dissipation component includes a cold plate type liquid cooling heat dissipation main component and an air cooling heat dissipation auxiliary component; The cold plate type liquid cooling heat dissipation main component indirectly transfers the heat of the heat-generating components to the cooling liquid enclosed in the circulation pipeline through the cold plate, and then conducts the heat out through the cooling liquid. The working fluid is used as an intermediate heat transfer medium to transfer the heat from the hot zone to the far end for cooling. The working liquid of the cold plate type liquid cooling heat dissipation main component is separated from the object being cooled, and the working liquid does not directly contact the electronic device. The working liquid and the electronic device are connected by the liquid cooling plate as a heat conduction component to transfer the heat of the object being cooled to the cooling liquid.

5. The AI ​​computing terminal according to claim 1, characterized in that, The DDR5 RDIMM memory socket is a standardized DDR5 R-DIMM, and the two standard memory slots corresponding to the DDR5 RDIMM memory socket support standard server memory RDIMM and ECC.

6. The AI ​​computing terminal according to claim 1, characterized in that, The integrated storage and computing chip is powered by a multi-phase power supply. The number of data channel lanes of the PCIe Gen5 RC in the integrated storage computing chip is a specified number, and the specified number of PCIe Gen5 RCs can be split in multiple ways.

7. An AI computing method based on a RISC-V processor and a Smart NIC, characterized in that, The method is applied to the AI ​​computing terminal described in any one of claims 1 to 6; the method includes: The Smart NIC and RDMA interconnect chip are interconnected to obtain external interconnection data; AI calculations are performed on the external interconnection data through the integrated storage and computing chip based on RISC-V, NPU and UCIE memory to obtain AI calculation results; The AI ​​calculation results are cached through the DDR5 RDIMM memory socket to be stored on a solid-state drive (SSD) or transmitted over a network to other devices besides the AI ​​computing terminal.

8. An AI computing system based on a RISC-V processor and a Smart NIC, characterized in that, The system is applied to the AI ​​computing terminal described in any one of claims 1 to 6; the system comprises: The external interconnection module is used to interconnect with the RDMA interconnection chip through the Smart NIC to obtain external interconnection data; The computing module is used to perform AI calculations based on the external interconnection data through the integrated storage computing chip based on RISC-V, NPU and UCIEmemory, and obtain AI calculation results. The memory module is used to cache the AI ​​calculation results through the DDR5 RDIMM memory module socket, so as to store them on a solid-state drive (SSD) or transmit them over a network to other devices other than the AI ​​computing terminal.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in claim 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method of claim 7.

Images