A kind of storage and network convergence server network card, traffic scheduling and processing method

Abstract

This invention relates to the field of servers, and provides a server network interface card (NIC) that integrates storage and networking, along with a traffic scheduling and processing method. The NIC includes: a network interface unit for processing network traffic; a storage protocol processing unit for processing storage traffic; a data exchange unit for forwarding and splitting network traffic and storage traffic according to traffic scheduling instructions; a GPU passthrough engine for establishing a direct data path between the data exchange unit and the GPU; a traffic scheduling unit for monitoring the NIC's operating status data and generating traffic scheduling instructions based on the monitoring results, which are then sent to the data exchange unit; and a global scheduling interface for receiving traffic scheduling policies and for in-band status broadcasting and traffic splitting signaling interaction with other node NICs. This invention solves the technical problems of low resource utilization, prominent I / O bottlenecks, and lack of dynamic traffic scheduling capabilities in existing servers due to the separation of storage and networking.

Description

Technical Field

[0001] This invention relates to the field of servers, and in particular to a server network interface card that integrates storage and networking, as well as a traffic scheduling and processing method. Background Technology

[0002] With the rapid development of AI large-scale model inference and distributed storage scenarios, server storage traffic and network traffic are increasing daily, and their interaction is becoming more frequent. For example, in a large-scale model inference cluster, the Prefill node needs to read a large amount of stored data, causing the network port carrying storage traffic to be nearly full (bandwidth usage reaches 100%), while the network port of the Decode node is idle for a long time. At the same time, the GPU utilization rate is less than 50% due to waiting for I / O data, resulting in a significant waste of hardware resources.

[0003] Currently, server storage and network functions are separated: storage traffic is typically transmitted via HBA cards, RAID cards, or the CPU's built-in controller, while network traffic is transmitted via a dedicated network interface card (NIC). The two cannot interact directly. If stored data needs to be sent to other nodes over the network, it must undergo protocol conversion and data copying by the host CPU, increasing CPU overhead and introducing additional latency. Existing NICs can only handle network traffic and cannot directly handle storage traffic. Specifically, current server NICs have the following drawbacks: 1) Storage and network are separated, resulting in low resource utilization: The storage and network functions of the server are hardware-independent, which leads to overload of storage-related ports of the Prefill node and idle network ports of the Decode node. The average utilization rate of network cards in the cluster is less than 50%, resulting in significant waste of hardware resources.

[0004] 2) Prominent I / O bottleneck: When storage traffic and network traffic need to interact (such as when storage data is forwarded over the network), the host CPU must perform protocol conversion and memory copying, which increases CPU overhead and data transmission latency. Consequently, the GPU utilization rate drops below 50% due to I / O waiting, which seriously affects the performance of scenarios with high real-time requirements, such as large model inference.

[0005] 3) Lack of dynamic traffic scheduling capability: Existing network cards can only process the network traffic they receive, and cannot sense the load status of other types of traffic, let alone achieve dynamic scheduling of cross-type traffic. When a node's network card is overloaded, it cannot migrate some traffic to other idle network cards within the same server or other nodes in the cluster, resulting in performance bottlenecks that cannot be alleviated.

[0006] 4) Poor compatibility and high integration cost: In existing servers, storage processing units (such as HBA and NVMe controllers) and network processing units (NICs) are hardware-independent, and the software stacks need to be adapted to different drivers, protocols, and management interfaces. If the two are to be integrated, significant changes to the upper-layer applications and operating systems are required, resulting in long adaptation cycles, high costs, and difficulty in flexibly deploying them in existing storage and network environments. Summary of the Invention

[0007] This invention provides a storage and network converged server network card, traffic scheduling and processing method to solve the technical problems in the background art.

[0008] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0009] According to a first aspect of the present invention, a storage and networking converged server network interface card (NIC) is provided, comprising: The network interface unit provides an external network interface for processing network traffic; The storage protocol processing unit is directly connected to the storage device and is used to process storage traffic; The data exchange unit is connected to the network interface unit, the storage protocol processing unit, and the traffic scheduling unit, respectively, and is used to forward and distribute network traffic and storage traffic according to the traffic scheduling instructions of the traffic scheduling unit; The GPU passthrough engine connects to the data exchange unit and is used to establish a direct data path between the data exchange unit and the GPU. The traffic scheduling unit, connected to the global scheduling interface, is used to monitor the working status data of the network card in real time, and generate traffic scheduling instructions based on the monitoring results and traffic scheduling strategy and send them to the data exchange unit. The global scheduling interface is used to receive traffic scheduling policies and to interact with other node network cards via in-band status broadcasting and traffic splitting signaling.

[0010] According to one embodiment of this application, the traffic scheduling unit includes: The monitoring unit is used to collect real-time network card operating status data, including bandwidth utilization, queue depth, traffic type, and data transmission rate. The overload judgment unit is used to determine whether the current network card is in an overload state based on the bandwidth utilization and traffic type, and to generate a traffic diversion trigger signal when overloaded. The scheduling unit is used to generate traffic scheduling instructions based on the traffic diversion trigger signal and traffic scheduling strategy, control the data exchange unit to divert the storage traffic of the current network card to the converged network card of the target node through the network interface unit, and send a redirection request message to the target node through the global scheduling interface.

[0011] According to one embodiment of this application, after the monitoring unit collects the working status data of the network card, it sends the data to the overload judgment unit for overload judgment and to the global scheduling interface for status synchronization.

[0012] According to one embodiment of this application, in the overload judgment unit, if the bandwidth occupancy rate of the storage traffic identified by the monitoring unit is greater than the overload threshold in the preset traffic scheduling strategy, the current network card is determined to be in an overload state, and a traffic diversion trigger signal is generated.

[0013] According to one embodiment of this application, the scheduling unit includes: The global status table maintenance unit is used to receive working status data broadcast by the network cards of all nodes in the cluster in real time through the global scheduling interface, and to maintain and update the global status table locally. The decision-making unit, in response to the traffic diversion trigger signal, determines the optimal target node for traffic diversion and the storage traffic that needs to be diverted based on the traffic scheduling strategy and the working status of each node's network card in the global status table, and then generates a traffic scheduling instruction and sends it to the data exchange unit.

[0014] According to one embodiment of this application, in the decision-making unit, the storage traffic that needs to be offloaded is determined to be in storage protocol requests as the smallest scheduling unit.

[0015] According to one embodiment of this application, the process of determining the storage traffic that needs to be offloaded in the decision-making unit includes: Obtain the priority of different storage traffic in the traffic scheduling strategy; Sort the pending storage protocol requests by priority; Calculate the diversion ratio based on the degree of overload; Storage protocol requests with the specified splitting ratio are selected as the storage traffic to be split, arranged from low to high priority.

[0016] According to one embodiment of this application, a software collaboration component running on the host side is also included, the software collaboration component comprising: Traffic monitoring software is used to obtain the working status data of the network cards of each node in the cluster and form a global view; The scheduling management software is used to provide users with traffic scheduling strategy configuration functions and traffic diversion forced intervention functions, and to send the configured traffic scheduling strategy and the generated forced diversion command to the traffic scheduling unit; the configured traffic scheduling strategy includes at least the optimal diversion target node selection strategy, overload threshold and priority of different storage traffic; Network interface card (NIC) drivers are used to enable interaction between the server's NIC and the host operating system.

[0017] According to a second aspect of the present invention, a traffic scheduling method for a server network interface card (NIC) based on the storage and network convergence of the first aspect is provided, comprising: The traffic scheduling unit collects the working status data of this network card in real time and broadcasts the working status data through the global scheduling interface. In response to the working status data exceeding the preset threshold, the traffic scheduling unit triggers a traffic diversion operation. Based on the traffic scheduling strategy and the working status of each node's network card in the global status table, it determines the optimal target node for diversion and the storage traffic that needs to be diverted, and generates a traffic scheduling instruction to be sent to the data exchange unit. The data exchange unit redirects the corresponding storage protocol request to the target node according to the traffic scheduling instruction, thus completing the traffic diversion.

[0018] According to a third aspect of the present invention, a traffic processing method for a server network interface card (NIC) based on the storage and network convergence described in the first aspect is provided, comprising: In response to a read command for stored data, the storage protocol processing unit reads and processes the stored data from the connected storage device. The traffic scheduling unit determines the destination address based on the read instruction and generates instructions to control the data exchange unit to send the processed storage data network interface unit to the target node or to control the GPU passthrough engine to directly write the processed storage data into the local GPU memory.

[0019] Compared with existing technologies, the beneficial effects of adopting the above technical solution are as follows: 1. Improved hardware resource utilization: By using global dynamic traffic scheduling, the storage traffic of overloaded nodes can be diverted to the converged network interface cards (NICs) of other idle nodes. This can alleviate the problem of uneven NIC load among nodes, effectively improve the average utilization of NICs within the cluster, and reduce the waste of hardware resources.

[0020] 2. Improved GPU utilization: By using a storage-network converged architecture and GPUDirect RDMA pass-through data path, the dependence on the host CPU is reduced when storage data interacts with network data, alleviating the bottleneck caused by GPU I / O wait and improving the utilization of GPU in scenarios such as large model inference.

[0021] 3. Reduced data transmission latency: Storage traffic and network traffic are processed and forwarded directly within the network card, without needing to be relayed through the host CPU, which can reduce data transmission latency; at the same time, the additional overhead of dynamic traffic splitting across nodes can be controlled at the microsecond level, meeting the needs of low-latency applications.

[0022] 4. Reduced CPU overhead: Storage protocol processing, traffic scheduling, and data pass-through are all handled by the network card hardware, without occupying host CPU resources, thus freeing up CPU for executing core business logic.

[0023] 5. Multi-scenario adaptation and scalability: NVMe compatible It supports multiple protocols such as oF, RoCE v2, and Ethernet, and supports bandwidth specifications such as 25G / 100G / 800G. It can flexibly adapt to different scenarios such as distributed storage, large model inference, and cloud virtualization, and can be expanded to ultra-large-scale AI clusters. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0025] Figure 1 This is a schematic diagram of a server network interface card (NIC) for storage and networking integration according to an embodiment of the present invention.

[0026] Figure 2 This is a flowchart illustrating the traffic scheduling process according to an embodiment of the present invention.

[0027] Figure 3 This is a schematic diagram of traffic processing according to an embodiment of the present invention. Detailed Implementation

[0028] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar modules or modules having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. Rather, embodiments of the present invention include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0029] To address the technical issues of low resource utilization, prominent I / O bottlenecks, and lack of dynamic traffic scheduling capabilities caused by the separation of storage and network in existing servers, specifically, storage traffic and network traffic need to be relayed and interacted through the host CPU, introducing additional latency and CPU overhead, causing GPU waiting; at the same time, the network card load is unbalanced between nodes (such as the network card of the Prefill node is overloaded and the Decode node is idle), with an overall utilization rate of less than 50%; existing network cards cannot detect and schedule another type of traffic, making it difficult to achieve dynamic traffic splitting across nodes or ports.

[0030] Based on this, "storage-network convergence" has become an important direction for server hardware architecture. This invention provides a server network interface card (NIC) that integrates storage and networking. By integrating a storage protocol processing unit and a traffic scheduling unit into the NIC, it achieves unified processing and dynamic scheduling of storage traffic and network traffic, improving NIC utilization, reducing GPU I / O latency, and adapting to the needs of scenarios such as large model inference.

[0031] Please refer to Figure 1 The storage and network converged server network card includes an integrated network interface unit, storage protocol processing unit, data exchange unit, GPU passthrough engine, traffic scheduling unit, and global scheduling interface.

[0032] Specifically, the network interface unit, acting as the network traffic inlet and outlet, provides an external network interface and is mainly used to process network traffic, including encryption / decryption functions, data packet forwarding, and RDMA data transmission. In one embodiment, the network interface unit includes at least an encryption / decryption module, a MAC function module, and a PCS function module. The encryption / decryption module is configured to encrypt and decrypt data; the MAC function module is configured to perform frame encapsulation / decapsulation, MAC address filtering, and priority-based flow control processing on transmitted and received data; the PCS function module is coupled to the MAC sublayer through a media-independent interface and is configured to perform physical layer encoding, scrambling, and multi-channel alignment processing on the data, and output the encoded data stream.

[0033] In some embodiments, in order to improve network transmission efficiency, the network interface unit also integrates functions such as packet spraying and multipath transmission.

[0034] Furthermore, the storage protocol processing unit is primarily used to process storage traffic. Storage traffic refers to the total amount of storage-related data processed by the network interface card (NIC), including storage protocol requests and their responses, data payloads, etc. Storage protocol requests are the specific carriers of storage traffic, referring to commands or transactions conforming to a specific storage protocol, such as NVMe-oF commands. In this embodiment, the storage protocol processing unit integrates an NVMe-oF / RoCE controller, supports the NVMe-oF protocol, and directly interfaces with distributed storage devices (such as NVMe SSDs) to exchange data directly without requiring intermediaries from the host CPU, effectively reducing data transmission latency. Furthermore, the storage protocol processing unit integrates at least data verification, format conversion, and SM4 encryption / decryption modules. The encryption / decryption modules support national standard SM4, IPsec, and other encryption protocols to ensure the security and integrity of data transmission.

[0035] The data exchange unit is connected to the network interface unit, the storage protocol processing unit, and the traffic scheduling unit, and is mainly used for data exchange and forwarding. In this embodiment, the data exchange unit forwards and distributes network traffic and storage traffic according to the traffic scheduling instructions of the traffic scheduling unit.

[0036] The GPU passthrough engine is connected to the data exchange unit. The GPU passthrough engine integrates GPUDirect RDMA functionality, providing a direct data path between the data exchange unit and the GPU. In this embodiment, according to instructions from the traffic scheduling unit, the data exchange unit can directly exchange data with the GPU through the GPU passthrough engine, effectively alleviating GPU I / O bottlenecks.

[0037] By integrating a network interface unit, a storage protocol processing unit, and a data exchange unit into the network card, a storage-network converged structure is formed. At the same time, a direct data path is established between the network interface unit and the storage protocol processing unit, enabling storage traffic and network traffic to be processed in parallel within the network card without the need for host CPU intervention for protocol conversion or data copying, thus breaking the limitations of the traditional architecture that separates storage and network.

[0038] The traffic scheduling unit, as the core unit of the network interface card (NIC) proposed in this invention, is responsible for real-time monitoring of the NIC's operating status data. Based on the monitoring results and traffic scheduling strategy, it generates traffic scheduling instructions and sends them to the data exchange unit to achieve dynamic traffic scheduling. The traffic scheduling unit is connected to the global scheduling interface. Specifically, the traffic scheduling unit mainly determines whether the NIC is currently overloaded based on the operating status data. If overloaded, it triggers a traffic splitting operation, thereby controlling the data exchange unit to complete the splitting of storage traffic.

[0039] The global scheduling interface is primarily used to receive traffic scheduling policies, which are then written into the traffic scheduling unit. It is also used for in-band status broadcasting and traffic splitting signaling interaction with other node network cards, enabling autonomous dynamic migration of cross-node traffic. In one embodiment, the global scheduling interface supports two types of communication channels: a management channel and a data channel. The management channel interfaces with the scheduling management software for configuring traffic scheduling policies (such as load balancing policies and priority rules), which are then converted into hardware parameters by the scheduling management software and written to the traffic scheduling units of each node. The data channel is implemented between the network cards of each node via an in-band broadcast protocol based on RoCE, directly synchronizing working status data and transmitting traffic splitting signaling such as redirection requests / responses, thus enabling autonomous dynamic migration of cross-node traffic.

[0040] In one embodiment, the global scheduling interface can also interface with the cluster scheduler to receive global traffic scheduling policies issued by the cluster scheduler through a management channel. That is, the cluster scheduler distributes traffic scheduling policies to all network interface cards (NICs) within the cluster. After receiving the policy configuration, each NIC converts it into hardware parameters by the scheduling management software and writes it to its respective node traffic scheduling unit. It should be noted that the cluster scheduler is a common component in cluster scenarios composed of multiple server nodes (such as Kubernetes), and existing solutions can be used to implement it. Its implementation is not the focus of this invention. In this invention, the cluster scheduler is only used as a means to uniformly distribute global scheduling policies.

[0041] In one embodiment, the traffic scheduling unit is implemented using an ASIC chip design and mainly includes a monitoring unit, an overload judgment unit, and a scheduling unit.

[0042] Specifically, the monitoring unit is responsible for collecting real-time network interface card (NIC) status data, including bandwidth utilization, queue depth, traffic type (including storage traffic and network traffic), and data transmission rate. In one embodiment, the monitoring unit can use hardware logic such as a hardware counter, queue depth sampler, and packet parsing logic to obtain the NIC's status data in real time. Specifically, the hardware counter counts the total number of bytes sent and received by the port, the total number of packets, and the number of bytes of storage traffic to calculate bandwidth utilization and transmission rate. The queue depth sampler collects the current queue depth of the data exchange unit. The traffic type is determined by independent packet parsing logic (hardware). When a data packet enters the NIC, it is identified by parsing the packet header fields, such as storage traffic (RoCEv2 (UDP port 4791), NVMe...). (oF (TCP port 4420), iSCSI (TCP port 3260), etc.) network traffic consists of other IP packets. The data transfer rate is triggered by the timer logic within the traffic scheduling unit, which reads the current value of the hardware counter, calculates the difference between the current value and the previous value, and then divides it by the time window length.

[0043] The data collected by the monitoring unit is sent directly to the overload judgment unit for decision comparison. A rapid load split is triggered by a preset threshold, and the entire process is completed in hardware, achieving a microsecond-level response. On the other hand, the collected data periodically broadcasts its own load status (bandwidth utilization, queue status, etc.) to other nodes in the cluster through a global scheduling interface, thereby synchronizing the working status data.

[0044] The overload judgment unit determines whether the current network card needs to trigger a traffic splitting operation based on a preset traffic scheduling algorithm. Specifically, the overload judgment unit identifies the storage traffic by monitoring the traffic type of the monitoring unit and determines whether the bandwidth utilization of the storage traffic is greater than the overload threshold in the preset traffic scheduling strategy (in this embodiment, it is set to 80%, which can be adjusted as needed). If it is greater than the threshold, the current network card is determined to be in an overload state, and a traffic splitting trigger signal is generated.

[0045] The scheduling unit generates traffic scheduling instructions based on the traffic diversion trigger signal generated by the overload judgment unit and the traffic scheduling strategy. It controls the data exchange unit to divert part of the storage traffic of the current network interface card to the converged network interface card of the target node, and sends a redirection request message to the target node through the global scheduling interface to complete the cross-node traffic diversion.

[0046] In one embodiment, the scheduling unit mainly includes a global state table maintenance unit and a decision-making unit. Specifically, the global state table maintenance unit is used to receive the working status data of all network cards of all nodes in the cluster in real time through the global scheduling interface, and maintain and update the global state table locally. Specifically, by parsing broadcast messages in the cluster, the working status data of other node network cards can be obtained, and the local global state table can be updated directly in sequence. This local global state table can be used to select the target node for optimal traffic allocation later. Furthermore, the global state table is stored in the hardware register / SRAM inside each network card traffic scheduling unit, and is updated and aged autonomously by hardware logic (if no update is received within timeout, the node is marked as invalid), without the need for host CPU participation.

[0047] The decision-making unit, in response to the traffic offloading trigger signal, determines the optimal target node for traffic offloading and the storage traffic that needs to be offloaded based on the traffic scheduling strategy and the working status data of each node's network interface card in the global status table, and then generates a traffic scheduling command and sends it to the data exchange unit. The traffic scheduling strategy includes at least the target node selection strategy, overload threshold, and priority information for different storage traffic types.

[0048] Specifically, when determining the optimal target node for traffic splitting, the decision unit queries the working status data of other node network cards in the maintained global status table to determine the load situation. Then, it determines the optimal target node for traffic splitting based on the target node selection strategy in the traffic scheduling policy. In one embodiment, the target node selection strategy includes: selecting idle nodes within the cluster whose network card bandwidth utilization is below 30%, and from these idle nodes, selecting nodes that are close to overloaded nodes and have low network latency as the optimal target node for traffic splitting.

[0049] In this embodiment, when determining the storage traffic that needs to be split, the storage protocol request is used as the smallest scheduling unit to determine the storage traffic that needs to be split, so as to avoid splitting the data stream of a single command and ensure transaction integrity.

[0050] Furthermore, this embodiment also provides a process for determining the storage traffic that needs to be offloaded: First, the priorities of different storage traffic in the traffic scheduling strategy are obtained. The configured traffic scheduling strategy includes the priorities of traffic that does not pass through storage, for example, 0~7, with 7 being the highest. By dividing the traffic into priorities, high-priority traffic (such as inference core requests and GPU passthrough data) can be reserved for local processing during subsequent traffic distribution. Low-priority traffic is then preferentially distributed to other nodes.

[0051] Next, the pending storage protocol requests are sorted by priority. Specifically, the scheduling unit maintains a queue of storage protocol requests, sorted by priority and arrival time.

[0052] Next, the traffic splitting ratio is calculated based on the overload level. For example, if the overload level is 30%, approximately 30% of the requests or data volume will be selected from the low-priority requests. The overload level is calculated by the traffic scheduling unit based on the difference between the current storage bandwidth utilization rate and the preset overload threshold. The calculation method is: Overload Level = (Current Storage Bandwidth Utilization Rate) / (Previous Storage Bandwidth Utilization Rate) Storage overload threshold) / storage overload threshold * 100%, where the current storage traffic bandwidth utilization rate is obtained in real time by the hardware counter of the monitoring unit (number of bytes of storage traffic / total bandwidth), and the storage overload threshold can be pre-configured by the scheduling management software and written into the traffic scheduling unit register.

[0053] Finally, storage protocol requests with the specified splitting ratio are selected as the storage traffic to be split, in order of priority from low to high.

[0054] When traffic splitting is triggered, a number of requests (note that the cumulative data length should not exceed a preset threshold) representing a certain proportion of the traffic splitting ratio are selected from the tail of the maintained queue (lowest priority area), and their target nodes are redirected to idle nodes selected from the global status table. In one embodiment, the selected requests can be locally marked as "split", and subsequent responses are handled by the target node.

[0055] By setting up a traffic scheduling unit in the network interface card (NIC) and defining a standardized global scheduling interface, each node's NIC can synchronize its own load status (such as bandwidth usage of storage ports / network ports, queue depth, etc.) in real time through this global scheduling interface. When an overloaded state is detected in a node, the traffic scheduling unit dynamically diverts some of the storage traffic processing requests to the NICs of other idle nodes according to the global status table, thereby achieving cross-boundary storage traffic load balancing.

[0056] In one embodiment, the traffic scheduling unit also provides data forwarding control functionality. Specifically, the traffic scheduling unit identifies the destination address of the computational task (such as tensors for large model inference) to which the received storage traffic belongs, determines whether it needs to be directly sent to GPU memory, and generates scheduling instructions. If necessary, the data exchange unit uses the integrated GPU passthrough engine to write the storage traffic directly from the network card's internal buffer to the GPU memory address, without going through the host memory. This process is executed in parallel with the protocol processing of the storage protocol processing unit and the data forwarding of the network interface unit, which can effectively reduce GPU I / O latency.

[0057] Building upon the aforementioned hardware design of the storage and networking converged server network interface card (NIC), one embodiment further includes a software coordinating component running in the host-side operating system, which works in conjunction with the hardware. Specifically, this software coordinating component mainly includes traffic monitoring software, scheduling management software, and the NIC driver.

[0058] The traffic monitoring software reads the working status data collected by the traffic scheduling unit of each node's network card through the network card driver, and aggregates the data of each node to form a global view, providing users with a display of the status of each node.

[0059] The scheduling management software primarily provides users with traffic scheduling policy configuration functions and traffic diversion forced intervention functions, and issues the configured traffic scheduling policies and generated forced diversion commands to the traffic scheduling unit. The configured traffic scheduling policies include at least the optimal diversion target node selection strategy, overload threshold, and priority of different storage traffic.

[0060] In one embodiment, the optimal traffic splitting target node selection strategy can be configured to select idle nodes within the cluster whose network interface card bandwidth utilization is less than 30%, and from these idle nodes, select nodes that are close to the overloaded nodes and have low network latency as the optimal traffic splitting target nodes. It should be noted that this selection strategy can be adjusted according to actual needs.

[0061] In one embodiment, the priority of different storage traffic can be set based on information such as task type, QoS level, and LBA address range.

[0062] It should be noted that the forced traffic diversion command issued by the scheduling management software must include at least the target node for diversion and the storage traffic that needs to be diverted.

[0063] The network interface card (NIC) driver is primarily responsible for the interaction between the NIC hardware and the host operating system. This includes receiving traffic scheduling policies from the host, enforcing traffic splitting commands, and feeding back the processed data from the NIC to the host, GPU, or storage device. In one embodiment, the NIC driver also supports software frameworks such as DPDK / VPP / OVS to improve the NIC's compatibility and scalability.

[0064] Please refer to Figure 2 This invention also proposes a traffic scheduling method for the aforementioned storage and network converged server network interface card, specifically including the following steps: S101. The traffic scheduling unit collects the working status data of this network card in real time and broadcasts the working status data through the global scheduling interface.

[0065] In this step, the traffic scheduling unit will collect real-time operational status data and broadcast its own operational status data, as well as obtain operational status data from other nodes, through the global scheduling interface. The traffic monitoring software can obtain the operational status data of the network cards of all nodes in the cluster through the network card driver. This operational status data includes bandwidth utilization, queue depth, traffic type data, and data transmission rate for each port.

[0066] S102. In response to the working status data exceeding the preset threshold, the traffic scheduling unit triggers a traffic diversion operation. Based on the traffic scheduling strategy and the working status of each node's network card in the global status table, it determines the optimal target node for diversion and the storage traffic that needs to be diverted, and generates a traffic scheduling instruction which is sent to the data exchange unit.

[0067] In this step, the traffic scheduling unit identifies the storage traffic by parsing the packet header and determines whether the bandwidth utilization of the storage traffic exceeds the overload threshold in the preset traffic scheduling strategy (set to 80% in this embodiment, which can be adjusted as needed). If it does, the unit determines that the current network card is in an overloaded state and generates a traffic diversion trigger signal. At this time, based on the traffic scheduling strategy and the working status of each node's network card in the global status table, the optimal target node for traffic diversion and the storage traffic that needs to be diverted are determined, and a traffic scheduling instruction is generated and sent to the data exchange unit.

[0068] S103. The data exchange unit forwards the corresponding storage traffic to the corresponding target node according to the traffic scheduling instruction. The network card of the target node processes the distributed storage traffic through the storage protocol processing unit, while maintaining the normal processing of its own network traffic, so as to realize the dynamic allocation of traffic and resource sharing.

[0069] Please refer to Figure 3 This invention also provides a traffic processing method for a server network interface card based on the aforementioned storage and network convergence, used to achieve coordinated processing of storage traffic and network traffic, specifically including the following steps: S201. In response to a read instruction for stored data, the storage protocol processing unit reads the stored data from the connected storage device and processes it, such as verification and format conversion (e.g., converting row storage format to GPU-friendly column storage format).

[0070] When the host or GPU needs to read stored data, it sends a read command to the storage protocol processing unit via the network card driver. At this time, the network card responds to the read command from the network card driver, and the storage protocol processing unit directly reads the data from the distributed storage device it interfaces with, without going through the CPU.

[0071] S202, the flow scheduling unit determines the destination address according to the read instruction and generates an instruction to control the data exchange unit to send the processed storage data network interface unit to the target node or to control the GPU passthrough engine to directly write the processed storage data into the local GPU memory.

[0072] In response to the information in the read instruction, the flow scheduling unit determines the target address for the storage data to be transmitted. If it needs to be transmitted to another node, it generates an instruction to control the data exchange unit to send the processed storage flow to the target node through the network interface unit. If it needs to be transmitted to the local GPU, it generates an instruction to control the data exchange unit to write the processed storage flow directly to the GPU memory through the GPU passthrough engine, thus alleviating the GPU I / O bottleneck.

[0073] The storage and network convergence server network card, traffic scheduling and processing method proposed in this embodiment of the invention has the following advantages: 1. Improved hardware resource utilization: Through global dynamic traffic scheduling, the storage traffic of overloaded nodes is diverted to the converged network cards of other idle nodes, which can alleviate the problem of uneven network card load among nodes, effectively improve the average utilization of network cards in the cluster, and reduce the waste of hardware resources.

[0074] 2. Improved GPU utilization: Through the storage-network converged architecture and GPUDirect RDMA pass-through data path, the dependence on the host CPU when interacting with storage data and network data is reduced, alleviating the bottleneck caused by GPU I / O waiting, and improving the utilization of GPU in scenarios such as large model inference.

[0075] 3. Reduced data transmission latency: Storage traffic and network traffic are processed and forwarded directly within the network card without being relayed through the host CPU, which reduces data transmission latency; at the same time, the additional overhead of dynamic traffic splitting across nodes can be controlled at the microsecond level, meeting the needs of low-latency applications.

[0076] 4. Reduced CPU overhead: Storage protocol processing, traffic scheduling, and data pass-through are all handled by the network card hardware, without occupying host CPU resources, thus freeing up CPU for executing core business logic.

[0077] 5. Multi-scenario adaptation and scalability: Compatible with NVMe It supports multiple protocols such as oF, RoCE v2, and Ethernet, and supports bandwidth specifications such as 25G / 100G / 800G. It can flexibly adapt to different scenarios such as distributed storage, large model inference, and cloud virtualization, and can be expanded to ultra-large-scale AI clusters.

[0078] It should be noted that, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "set" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances. The accompanying drawings in the embodiments are used to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0079] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A server network interface card (NIC) that integrates storage and networking, characterized in that, include: The network interface unit provides an external network interface for processing network traffic; The storage protocol processing unit is directly connected to the storage device and is used to process storage traffic; The data exchange unit is connected to the network interface unit, the storage protocol processing unit, and the traffic scheduling unit, respectively, and is used to forward and distribute network traffic and storage traffic according to the traffic scheduling instructions of the traffic scheduling unit; The GPU passthrough engine connects to the data exchange unit and is used to establish a direct data path between the data exchange unit and the GPU. The traffic scheduling unit, connected to the global scheduling interface, is used to monitor the working status data of the network interface card in real time, and generate traffic scheduling instructions based on the monitoring results and traffic scheduling policies and send them to the data exchange unit. as well as The global scheduling interface is used to receive traffic scheduling policies and to interact with other node network cards via in-band status broadcasting and traffic splitting signaling.

2. The server network interface card integrating storage and networking according to claim 1, characterized in that, The traffic scheduling unit includes: The monitoring unit is used to collect real-time network card operating status data, including bandwidth utilization, queue depth, traffic type, and data transmission rate. The overload judgment unit is used to determine whether the current network card is in an overload state based on the bandwidth utilization and traffic type, and to generate a traffic diversion trigger signal when overloaded. The scheduling unit is used to generate traffic scheduling instructions based on the traffic diversion trigger signal and traffic scheduling strategy, control the data exchange unit to divert the storage traffic of the current network card to the network card of the target node through the network interface unit, and send a redirection request message to the target node through the global scheduling interface.

3. The server network interface card integrating storage and networking according to claim 2, characterized in that, After the monitoring unit collects the network card's working status data, it sends the data to the overload judgment unit for overload judgment and to the global scheduling interface for status synchronization.

4. The server network interface card integrating storage and networking according to claim 2, characterized in that, In the overload judgment unit, if the bandwidth utilization rate of the storage traffic identified by the monitoring unit is greater than the overload threshold in the traffic scheduling strategy, the current network card is determined to be in an overload state, and a traffic diversion trigger signal is generated.

5. The server network interface card integrating storage and networking according to claim 2, characterized in that, The scheduling unit includes: The global status table maintenance unit is used to receive working status data broadcast by the network cards of all nodes in the cluster in real time through the global scheduling interface, and to maintain and update the global status table locally. The decision-making unit, in response to the traffic diversion trigger signal, determines the optimal target node for traffic diversion and the storage traffic that needs to be diverted based on the traffic scheduling strategy and the working status data of each node's network card in the global status table, and generates a traffic scheduling instruction to be sent to the data exchange unit.

6. The server network interface card integrating storage and networking according to claim 5, characterized in that, In the decision-making unit, the storage traffic that needs to be diverted is determined to be in the smallest scheduling unit, which is the storage protocol request.

7. The server network interface card (NIC) integrating storage and networking according to claim 6, characterized in that, The process of determining the storage traffic that needs to be diverted in the decision-making unit includes: Obtain the priority of different storage traffic in the traffic scheduling strategy; Sort the pending storage protocol requests by priority; Calculate the diversion ratio based on the degree of overload; Storage protocol requests with the specified splitting ratio are selected as the storage traffic to be split, arranged from low to high priority.

8. The server network interface card (NIC) integrating storage and networking according to claim 1, characterized in that, It also includes a software collaboration component running on the host side, the software collaboration component comprising: Traffic monitoring software is used to obtain the working status data of the network cards of each node in the cluster and form a global view; The scheduling management software is used to provide users with traffic scheduling strategy configuration functions and traffic diversion forced intervention functions, and to send the configured traffic scheduling strategy and the generated forced diversion command to the traffic scheduling unit; the configured traffic scheduling strategy includes at least the optimal diversion target node selection strategy, overload threshold and priority of different storage traffic; Network interface card (NIC) drivers are used to enable interaction between the server's NIC and the host operating system.

9. A traffic scheduling method for a server network interface card (NIC) based on the storage and network convergence described in any one of claims 1 to 8, characterized in that, include: The traffic scheduling unit collects the working status data of this network card in real time and broadcasts the working status data through the global scheduling interface. In response to the working status data exceeding the preset threshold, the traffic scheduling unit triggers a traffic diversion operation. Based on the traffic scheduling strategy and the working status of each node's network card in the global status table, it determines the optimal target node for diversion and the storage traffic that needs to be diverted, and generates a traffic scheduling instruction to be sent to the data exchange unit. The data exchange unit redirects the corresponding storage traffic to the target node according to the traffic scheduling instruction, thus completing the traffic diversion.

10. A method for processing traffic on a server network interface card (NIC) based on the storage and network convergence of any one of claims 1 to 8, characterized in that, include: In response to a read command for stored data, the storage protocol processing unit reads and processes the stored data from the connected storage device. The traffic scheduling unit determines the destination address based on the read instruction and generates instructions to control the data exchange unit to send the processed storage data network interface unit to the target node or to control the GPU passthrough engine to directly write the processed storage data into the local GPU memory.

Images