Network card data processing methods and network cards

Abstract

This application provides a data processing method for a network interface card (NIC). The method includes: the NIC acquiring a first I / O command, the first I / O command instructing the writing of data stored in a local server to at least one peer server; the first I / O command including address information and length information of the data stored on the local server; the NIC splitting the data according to the address information and length information to obtain multiple sets of address information and length information; the NIC acquiring data corresponding to the multiple sets of address information and length information from the local server according to the multiple sets of address information and length information, and sending the data to at least one peer server. In this technical solution, the NIC does not require a large amount of memory to store the data to be written, reducing program complexity and memory resource consumption.

Description

[0001] This application is a divisional application. The original application has the application number 202010999199.9 and the original application date is September 22, 2020. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of server technology, and more specifically, to a data processing method for a network interface card (NIC) and the NIC itself. Background Technology

[0003] With the rise of cloud computing, in order to reduce the utilization of host processors (e.g., central processing units, CPUs) and allow more host CPUs to be offered for sale, or to improve the user experience by freeing up user CPUs from the impact of numerous background host operations, services that originally ran on the host, such as management, storage, and network services, can be offloaded to the network interface card (NIC). The CPU on the NIC can then handle these services, thus freeing up virtually all of the host's CPU resources for the user.

[0004] Taking storage services as an example, in related technical solutions, for input / output (IO) write commands, the network interface card (NIC) needs to store the data to be written from the host into the NIC's memory. The NIC's processor then processes the data and sends it to the peer server. For IO read commands, the data is first read from the peer server into the NIC's memory, and then written into the host's memory.

[0005] In the aforementioned technical solutions, on the one hand, since data needs to be read from the host's memory into the network card's memory, this puts a lot of pressure on the network card's bus and greatly increases the overhead of the processor in the network card. On the other hand, the network card requires a large amount of memory to store this data, and the storage and management of this data greatly increases the complexity of the program and the consumption of memory resources. Summary of the Invention

[0006] This application provides a data processing method for a network card and a network card itself. The network card does not require a large amount of memory to store the data to be written, thus reducing the complexity of the program and the consumption of memory resources.

[0007] In a first aspect, a data processing method for a network interface card (NIC) is provided, comprising: the NIC acquiring a first input / output (IO) command, the first IO command instructing data stored in a local server to be written to at least one peer server, the first IO command including address information and length information of the data stored in the local server; the NIC splitting the data according to the address information and length information to obtain multiple sets of address information and length information; the NIC acquiring data corresponding to the multiple sets of address information and length information from the local server according to the multiple sets of address information and length information, and sending the data to the at least one peer server.

[0008] In the above technical solution, for I / O write commands, the data to be written does not need to be moved from the host's memory to the network card's memory. The network card processes storage services based on the storage address and data length of the data in the host's memory, rather than based on the actual data to be written stored in the host's memory. This reduces the load on the network card bus and the overhead of the network card processor, as data does not need to be read from the host's memory into the network card's memory. Furthermore, the network card does not require a large amount of memory to store the data to be written, reducing program complexity and memory resource consumption.

[0009] In one possible implementation, the network interface card (NIC) obtains data corresponding to the multiple sets of address information and length information from the first virtual machine (VM) that issued the first IO command in the local server.

[0010] In another possible implementation, the data is not stored in the network card's memory.

[0011] In another possible implementation, the method further includes: the network interface card (NIC) receiving multiple IO commands; the NIC selecting the first IO command from the multiple IO commands based on the storage address of the multiple IO commands stored in the peer server and / or the virtual machine (VM) that issues the multiple IO commands in the local server.

[0012] In another possible implementation, the network interface card (NIC) includes a hardware engine for processing the data plane. The hardware engine on the NIC selects the first IO command from the plurality of IO commands based on the storage address of the plurality of IO commands stored in the peer server and / or the virtual machine (VM) that issues the plurality of IO commands in the local server.

[0013] In another possible implementation, the method further includes: the network interface card (NIC) determining whether all of the plurality of data have been written to the at least one peer server based on the context information of the first IO command, wherein the context information of the first IO command includes the storage status of the plurality of data in the at least one peer server; when all of the plurality of data have been written to the at least one peer server respectively, the NIC sends an IO write complete message to the local server through an interface.

[0014] Secondly, a data processing method for a network interface card (NIC) is provided, comprising: the NIC acquiring a second I / O command, the second I / O command instructing the storage of data from at least one peer server to a local server, the second I / O command including address information and length information of the data to be written to the local server; the NIC splitting the second I / O command according to the address information and length information to obtain multiple I / O read messages, each of the multiple I / O read messages including information of each piece of data being written to the local server; and the NIC sending the I / O read messages to the at least one peer server respectively.

[0015] In the above technical solution, for IO reads, it is not necessary to first write the data read from the peer server into the network card's memory and then into the VM memory. This eliminates the need to send data to the network card's memory, reducing both the network card's memory resource consumption and bandwidth pressure, and also lowering the network card processor's overhead.

[0016] In one possible implementation, the information written to the local server for each piece of data includes any one of the following: address information and length information of each piece of data written to the local server; or address information and length information of each piece of data written to the local server and VM information of the local server that issues the second IO command; or a first identifier ID and offset information of the local server for writing the data, wherein the first ID is used to indicate the VM information of the local server that issues the second IO command and multiple sets of address information and length information of the VM for writing the data.

[0017] In another possible implementation, the method further includes: the network interface card (NIC) receiving a first IO read response message from a peer server, the first IO read response message including first data that the local server needs to obtain from the peer server; the NIC determining, based on the first IO read response message, the address information and length information of the first data to be written to the local server; and the NIC storing the first data in the local server based on the address information and length information.

[0018] In another possible implementation, the first IO read response message includes a first ID and offset information for writing the first data to the local server. The network interface card (NIC) determines multiple sets of address information and length information for writing data to the VM based on the first ID. The NIC determines the address information and length information for writing the first data to the local server from the multiple sets of address information and length information based on the offset information.

[0019] In another possible implementation, the first IO read response message includes the address information and length information of the first data written to the local server, as well as the VM information of the local server that issued the second IO command. The network interface card determines the address information and length information of the first data written to the local server based on the information included in the first IO read response message.

[0020] In another possible implementation, the length information is within the range of the permission table, which includes multiple sets of address information and length information for data written to the local server; the network card stores the first data in the local server according to the address information and length information included in the first IO read response message.

[0021] In another possible implementation, the method further includes: the network interface card (NIC) determining whether all of the plurality of data have been written to the local server based on the context information of the second IO command, wherein the context information of the second IO command includes the completion status of the plurality of data stored in the local server; when all of the plurality of data have been written to the local server, the NIC sends an IO read completion message to the local server through an interface.

[0022] Thirdly, a network interface card (NIC) is provided, including: an acquisition module, a splitting module, and a sending module. The acquisition module is used to acquire a first input / output (IO) command, the first IO command instructing the writing of data stored in the local server to at least one peer server, the first IO command including the address information and length information of the data stored in the local server; The splitting module is used to split the data according to the address information and length information to obtain multiple sets of address information and length information; The sending module is further configured to obtain data corresponding to the multiple sets of address information and length information from the local server based on the multiple sets of address information and length information, and send the data to the at least one peer server.

[0023] In one possible implementation, the acquisition module is specifically used to: acquire data corresponding to the multiple sets of address information and length information from the first VM based on the multiple sets of address information, length information, and the identifier of the first virtual machine VM that issued the first IO command in the local server.

[0024] In another possible implementation, the data is not stored in the network card's memory.

[0025] In another possible implementation, the network interface card (NIC) further includes a scheduling module. The acquisition module is also used to receive multiple IO commands; The scheduling module is used to select the first IO command from the plurality of IO commands based on the storage address of the plurality of IO commands stored in the peer server and / or the virtual machine (VM) that issues the plurality of IO commands in the local server.

[0026] In another possible implementation, the network interface card includes a hardware engine for processing the data plane. The hardware engine selects the first IO command from the plurality of IO commands based on the storage address of the plurality of IO commands stored in the peer server and / or the virtual machine (VM) that issues the plurality of IO commands in the local server.

[0027] In another possible implementation, the network interface card further includes: a determination module. The determining module is used to determine whether all of the plurality of data have been written to the at least one peer server based on the context information of the first IO command, wherein the context information of the first IO command includes the storage status of the plurality of data in the at least one peer server. The sending module is further configured to send an IO write complete message to the local server via an interface when all of the multiple data have been written to the at least one peer server.

[0028] The beneficial effects of the third aspect and any possible implementation of the third aspect correspond to the beneficial effects of the first aspect and any possible implementation of the first aspect, which will not be elaborated further.

[0029] Fourthly, a network interface card (NIC) is provided, including: an acquisition module, a splitting module, and a sending module. The acquisition module is used to acquire a second IO command, which instructs to store data from at least one peer server to the local server. The second IO command includes address information and length information of the data to be written to the local server. The splitting module is used to split the second IO command according to the address information and length information to obtain multiple IO read messages, and each of the multiple IO read messages includes information on each piece of data being written to the local server; The sending module is used to send the IO read message to the at least one peer server respectively.

[0030] In one possible implementation, the information written to the local server for each piece of data includes any one of the following: address information and length information of each piece of data written to the local server; or address information and length information of each piece of data written to the local server and VM information of the local server that issues the second IO command; or a first identifier ID and offset information of the local server for writing the data, wherein the first ID is used to indicate the VM information of the local server that issues the second IO command and multiple sets of address information and length information of the VM for writing the data.

[0031] In another possible implementation, the network interface card (NIC) further includes: a determination module and a storage module. The acquisition module is further configured to receive a first IO read response message from the peer server, wherein the first IO read response message includes first data that the local server needs to acquire from the peer server; The determining module is used to determine the address information and length information of the first data written to the local server based on the first IO read response message; The storage module is used to store the first data in the local server according to the address information and length information.

[0032] In another possible implementation, the first IO read response message includes a first ID and offset information for writing the first data to the local server. The determining module is specifically used to: determine multiple sets of address information and length information of the data written in the VM based on the first ID; The storage module is specifically used to: determine the address information and length information of the first data to be written to the local server from the multiple sets of address information and length information based on the offset information.

[0033] In another possible implementation, the first IO read response message includes the address information and length information of the first data written to the local server, as well as the VM information of the local server that issued the second IO command. The determining module is specifically used to: determine the address information and length information of the first data written to the local server based on the information included in the first IO read response message.

[0034] In another possible implementation, the determining module is specifically used to: determine that the address information and length information included in the first IO read response message are within the range of the permission table, wherein the permission table includes multiple sets of address information and length information of the data written in the local server; The storage module is specifically used to: store the first data in the local server according to the address information and length information included in the first IO read response message.

[0035] In another possible implementation, the determining module is further configured to: determine whether all of the plurality of data have been written to the local server based on the context information of the second IO command, wherein the context information of the second IO command includes the completion status of the plurality of data stored in the local server; The sending module is also used to: when all the data is written into the local server, send an IO read completion message to the local server through an interface.

[0036] The beneficial effects of the fourth aspect and any possible implementation of the fourth aspect correspond to the beneficial effects of the second aspect and any possible implementation of the second aspect, which will not be elaborated further.

[0037] Fifthly, a network interface card (NIC) is provided, comprising: a processor and a memory, wherein the processor executes instructions in the memory to cause the computing device to perform method steps as described in the first aspect or any possible implementation thereof.

[0038] In a sixth aspect, a network interface card (NIC) is provided, including a processor and a memory, wherein the processor executes instructions in the memory to cause the computing device to perform method steps as described in the second aspect or any possible implementation thereof.

[0039] In a seventh aspect, a server is provided, including a network interface card as described in the third aspect or any possible implementation thereof.

[0040] Eighthly, a server is provided, including a network interface card as described in the fourth aspect or any possible implementation thereof.

[0041] A ninth aspect provides a computer-readable storage medium including instructions; said instructions are configured to implement method steps performed as in the first aspect or any possible implementation thereof.

[0042] Alternatively, as one implementation method, the aforementioned storage medium can specifically be a non-volatile storage medium.

[0043] In a tenth aspect, a computer-readable storage medium is provided, including instructions for implementing method steps as performed in the second aspect or any possible implementation thereof.

[0044] Alternatively, as one implementation method, the aforementioned storage medium can specifically be a non-volatile storage medium.

[0045] Eleventhly, a chip is provided that acquires and executes instructions to implement the methods in the first aspect and any implementation thereof.

[0046] Optionally, as one implementation, the chip includes a processor and a data interface, through which the processor reads instructions stored in the memory and executes the methods in the first aspect and any implementation thereof.

[0047] Optionally, as one implementation, the chip may further include a memory storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to perform the method in the first aspect and any implementation thereof.

[0048] In a twelfth aspect, a chip is provided that acquires and executes instructions to implement the method in the second aspect or any possible implementation thereof.

[0049] Alternatively, as one implementation, the chip includes a processor and a data interface, through which the processor reads instructions stored in memory to implement the method in the second aspect or any possible implementation of the second aspect.

[0050] Alternatively, as one implementation, the chip may further include a memory storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to implement the method in the second aspect or any possible implementation of the second aspect. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of an architecture provided in an embodiment of this application.

[0052] Figure 2 This is a schematic diagram of an application scenario provided in an embodiment of this application.

[0053] Figure 3 This is a schematic flowchart illustrating a data processing method for a network interface card (NIC) provided in an embodiment of this application.

[0054] Figure 4This is a schematic diagram of the structure of a network card 400 provided in an embodiment of this application.

[0055] Figure 5 This is a schematic flowchart illustrating a method for a network interface card (NIC) to process I / O write commands, as provided in an embodiment of this application.

[0056] Figure 6 This is a schematic block diagram of an IO command scheduling method provided in an embodiment of this application.

[0057] Figure 7 This is a schematic block diagram illustrating a data splitting process provided in an embodiment of this application.

[0058] Figure 8 This is a schematic block diagram illustrating an IO command processing procedure provided in an embodiment of this application.

[0059] Figure 9 This is a schematic flowchart illustrating a method for a network interface card (NIC) to process I / O read commands, as provided in an embodiment of this application.

[0060] Figure 10 This is a schematic flowchart illustrating a method for a network interface card (NIC) to process I / O write response messages, as provided in an embodiment of this application.

[0061] Figure 11 This is a schematic flowchart illustrating another method for a network interface card (NIC) to process I / O write response messages, as provided in an embodiment of this application.

[0062] Figure 12 This is a schematic flowchart illustrating another method for a network interface card (NIC) to process I / O write response messages, as provided in an embodiment of this application.

[0063] Figure 13 This is a schematic block diagram of a network card 1300 provided in an embodiment of this application. Detailed Implementation

[0064] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0065] This application will present various aspects, embodiments, or features relating to systems comprising multiple devices, components, modules, etc. It should be understood and appreciated that individual systems may include additional devices, components, modules, etc., and / or may not include all devices, components, modules, etc. discussed in conjunction with the accompanying drawings. Furthermore, combinations of these approaches are also possible. Furthermore, in the embodiments of this application, the words "exemplary," "for example," etc., are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the term "exemplary" is intended to present the concept in a concrete manner. In the embodiments of this application, "corresponding" and "corresponding" can sometimes be used interchangeably. It should be noted that when the distinction is not emphasized, their intended meanings are consistent.

[0066] The network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0067] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0068] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0069] With the rise of cloud computing, in order to reduce the utilization of host processors (e.g., central processing units, CPUs), more host CPUs can be used for external sales, or to improve user experience by sparing users' CPUs from the impact of background host operations. Figure 1As shown, cloud computing vendors are gradually offloading management, storage, and network services that were originally running on the host to the network interface card (NIC), where the CPU on the NIC processes these services. In this way, the host's CPU can be provided entirely to the user.

[0070] The following is based on Figure 2 Taking the application scenario shown as an example, we will describe the storage service in detail.

[0071] like Figure 2 As shown, this scenario may include a host 210, a network interface card 220, and a destination storage device 230. Each device will be described in detail below.

[0072] 1. Host 210 The host 210 may include at least one processor 211 and a memory 212.

[0073] Optionally, the host 210 also includes a system bus 213. At least one processor 211 and a memory 212 are connected to the system bus 213.

[0074] Processor 211 can access memory 212 via system bus 213. For example, processor 211 can perform data read / write or code execution in memory 212 via system bus. System bus 213 may be, for example, a quick path interconnect (QPI) or ultra path interconnect (UPI). The system bus 213 is divided into address bus, data bus, control bus, etc. For ease of representation, Figure 2 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0075] In one possible implementation, the processor 211 primarily functions to interpret the instructions (or code) of a computer program and process data within the computer software. The instructions of the computer program and the data within the computer software can be stored in memory 212.

[0076] Processor 211 is the processing and control unit of host 210. Processor 211 may include multiple processor cores. All computations, command reception, command storage, and data processing in processor 211 are executed by the processor cores. Each processor core independently executes program instructions, utilizing parallel computing capabilities to accelerate program execution. A processor core has a fixed logical structure, including logical units such as L1 cache, L2 cache, execution units, instruction-level units, and bus interfaces.

[0077] Optionally, processor 211 may be an integrated circuit chip with signal processing capabilities. By way of example and not limitation, processor 110 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor, etc. For example, processor 110 may be a central processing unit (CPU).

[0078] Memory 212 is the main memory of host 210 (also referred to as host 210's RAM). Memory 212 is typically used to store various running software programs in the operating system, input / output (I / O) commands issued by upper-level applications, and information exchanged with external storage. To improve the access speed of processor 211, memory 212 needs to have a high access speed. In some computer system architectures, dynamic random access memory (DRAM) is used as memory 212. Processor 211 can access memory through the memory controller (… Figure 1 (Not shown in the image) High-speed access memory 212, to perform read and write operations on any memory cell in memory 212.

[0079] It should also be understood that the memory 212 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0080] 2. Network card 220 Network interface card 220 is used to enable host 210 to communicate with other servers in the network (e.g., Figure 1 Communication between the target storage device 230 (shown) and the network card 220. The network card 220 can be built into the host 210, or it can also be used as an external device of the host 210 and connected to the host 210 through an interface. This interface can be, for example, a bus and interface standard (PCIe) interface.

[0081] For ease of description, Figure 2 The network card 220 is described as an external device connected to the host 210.

[0082] The network interface card 220 may also include a memory 222 and a processor 221. Optionally, the network interface card 220 may also include a bus 223. The internal structure of the network interface card 220 may be similar to that of the host 210. For details, please refer to the description of the various parts of the host 210 above, which will not be repeated here.

[0083] 3. Destination storage device 230 The destination storage device 230 may also include a memory, a processor, and a network interface card (NIC). Optionally, the destination storage device 230 may also include a bus. The internal structure of the destination storage device 230 may be similar to that of the host 210; please refer to the description of the various parts of the host 210 above for details, which will not be repeated here.

[0084] In the data processing and communication scenario, host 210 is the local end, and destination storage device 230 is the communication peer. The virtual machine (VM) on host 210 issues input / output (I / O) commands, and the processor 211 on host 210 can send these I / O commands to memory 212 via bus 213. Network interface card 220 retrieves the I / O commands from memory 212 and processes them.

[0085] Taking read and write commands as examples of I / O commands, a read command is a command issued by an application (e.g., a VM) running on the host to instruct the reading of data from a remote device (e.g., a destination storage device), and a write command is a command issued by an application (e.g., a VM) running on the host to instruct the writing of data to a remote device. The host's processor can receive I / O commands and store them in memory, so that the I / O commands wait to be processed by the network interface card 220 before being sent to the destination storage device 230, which is the communication peer.

[0086] In the relevant technical solutions, taking the IO command as an IO write command as an example, the data to be written is stored in memory 212. Network interface card (NIC) 220 first needs to move the data to be written from memory 212 of host 210 to memory 222 of NIC 220. Then, processor 221 of NIC 220 retrieves the data to be written from memory 222 of NIC 220, completes the storage service processing for this data, generates a data message packet, and sends the data to be written to the destination storage device 230 at the communication peer through the generated message packet.

[0087] In the relevant technical solutions, taking the IO command as an IO read command as an example, the processor 221 of the network card 220 obtains data from the destination storage device 230 of the communication peer and stores the data in the memory 222 of the network card 220. Then, it retrieves data from the memory 222 of the network card 220, assembles the data into complete IO data, and writes the IO data into the memory 212 of the host 210.

[0088] In the aforementioned technical solutions, taking an I / O write command as an example, the data to be written needs to be moved from the host 210's memory 212 to the network card 220's memory 222, and then the network card 220's processor 221 processes the storage service for this data. On the one hand, since data needs to be read from the host 210's memory 212 into the network card 220's memory 222, this puts a lot of pressure on the network card 220's bus 223 and greatly increases the overhead of the network card 220's processor 221. On the other hand, the network card 220 requires a large amount of memory 222 to store this data, and the storage and management of this data greatly increases the complexity of the program and the consumption of memory 222 resources.

[0089] In view of this, this application proposes a data storage method in which the data to be written does not need to be moved from the memory 212 of the host 210 to the memory 222 of the network card 220. On the one hand, this can reduce the resource consumption and bandwidth pressure of the memory 222; on the other hand, it can also reduce the overhead of the processor 221 in the network card 220.

[0090] The following is combined with Figure 3 This application provides a detailed description of a data storage method according to an embodiment.

[0091] Figure 3 This is a schematic flowchart illustrating a data processing method for a network interface card (NIC) provided in an embodiment of this application. Figure 3 As shown, the method may include steps 310-330, which will be described in detail below.

[0092] Step 310: The network card obtains the first IO command, which includes the address information and length information of the data stored on the local server.

[0093] In this embodiment of the application, the first IO command can be used to instruct data stored in the local server to be written to at least one peer server. As an example, the data can be stored in the memory of the local server.

[0094] As an example, the aforementioned local server could be Figure 2 Host 210 in the example. The aforementioned peer server can be a storage device or a computing device with computing capabilities. As an example, the peer server is... Figure 2 The destination storage device 230.

[0095] The network interface card (NIC) can be used as an external device for the local server and connects to the server via an interface. This interface can be a bus or peripheral component interface express (PCIe) interface. Please refer to [reference needed] for specific connection details. Figure 2 The description in the text will not be repeated here.

[0096] Step 320: The network card splits the data according to the address information and length information to obtain multiple sets of address information and length information.

[0097] The network interface card (NIC) can split the data based on the address and length information carried in the first I / O command to obtain multiple sets of address and length information. It should be understood that the address and length information of the data can be used to describe a data block stored in the local server's memory.

[0098] In this embodiment, the network interface card's (NIC) memory does not need to store data that needs to be written from the local server to at least one peer server. Therefore, the NIC can split the data not based on the actual data, but on virtual address information (e.g., address and length information of the data in the local server's memory). This reduces the load on the NIC bus and the overhead of the NIC processor, as the data does not need to be read from the local server's memory. Furthermore, the NIC does not require a large amount of memory to store the data to be written, reducing program complexity and memory resource consumption.

[0099] Step 330: The network card obtains the data corresponding to the multiple sets of address information and length information from the local server based on the multiple sets of address information and length information, and sends the data to at least one peer server.

[0100] It should be understood that in step 330, when the network card needs to send data to at least one peer server, it directly obtains the data corresponding to the multiple sets of address information and length information from the local server based on the multiple sets of address information and length information, and sends it to at least one peer server.

[0101] It should be noted that the data obtained by the network card is not stored in the network card's memory, but is obtained from the memory of the local server and then sent directly to at least one peer server.

[0102] In one possible implementation, the network interface card (NIC) can obtain data corresponding to multiple sets of address and length information from the local server based on the multiple sets of address and length information obtained after splitting. Optionally, in some embodiments, the NIC can also obtain data corresponding to the multiple sets of address and length information from the first virtual machine (VM) on the local server, based on the multiple sets of address and length information and the identifier corresponding to the first VM that issued the first IO command. In this way, the NIC can accurately obtain data from the memory of a specific VM.

[0103] In the above technical solution, the network interface card (NIC) can serve as an external device for the local server and connect to it via an interface. For I / O write commands, the data to be written does not need to be moved from the local server's memory to the NIC's memory. The NIC processes storage services based on the storage address and length of the data in the local server's memory, rather than based on the actual data to be written stored in the local server's memory. This reduces the load on the NIC and memory bus, as it eliminates the need to read data from the local server's memory into the NIC's memory, thus reducing the overhead of the NIC processor. Furthermore, the NIC does not require a large amount of memory to store the data to be written, reducing program complexity and memory resource consumption.

[0104] The following is combined with Figure 4 This application provides a detailed description of the structure of a possible network interface card (NIC) according to an embodiment.

[0105] Figure 4 This is a schematic diagram of the structure of a network card 400 provided in an embodiment of this application. Figure 4 As shown, the network card 400 is connected to the host 210 via the PCIe interface.

[0106] The network card 400 may include: a data plane offloading engine 410, a memory controller 420, a CPU core 430, memory 440, and a system bus 450.

[0107] 1. Data plane unloading engine 410 As an example, the data plane offloading engine 410 can consist of hardware logic or engines from dedicated processing architectures that specifically support data plane processing, such as field-programmable gate arrays (FPGAs) or network processors (NPs). Its characteristic is its ability to perform fast packet forwarding, but it is not suitable for handling complex protocols.

[0108] It should be understood that the data plane offloading engine 410 is more suitable for network forwarding processing, so it can usually handle the data plane for network forwarding. However, due to the complexity of storage services, the data plane is difficult to handle entirely by it and usually requires a CPU core 430 to complete.

[0109] The data plane offloading engine 410 can be connected to the host 210 via a PCIe interface to retrieve data stored in the host 210's memory 212. The data plane offloading engine 410 can also be connected to the CPU core 430 and the memory controller 420 via the system bus 450, enabling high-speed communication between the various modules within the data plane offloading engine 410 and the CPU core 430 and the memory controller 420.

[0110] The data plane offloading engine 410 may include: a device simulation module 411, an IO command processing module 412, a data scheduling module 430, and a network protocol processing engine 460.

[0111] The functions of each module included in the data plane unloading engine 410 are described in detail below.

[0112] The device emulation module 411 is mainly used to emulate devices, enabling the network card 400 to present the required devices, such as storage and network devices, to the host or virtual machines (VMs) running on the host. As an example, the device emulation module 411 is used to present non-volatile memory express (NVMe) storage devices, small computer system interface (SCSI) devices, virtual small computer system interface (virtio-SCSI) devices, virtio-BLK devices, and other types of virtio devices.

[0113] It should be understood that device simulation can also be accomplished by both hardware logic and CPU cores. Typically, strong data interaction is handled by the data plane offloading engine, while configuration-related tasks are handled by the CPU cores.

[0114] The IO command processing module 412 is used to perform simple analysis and processing of IO commands. As an example, it can be used to determine whether an IO command is a read or write command. As another example, IO read and write commands typically require scheduling to ensure the device's IO quality of service (QoS).

[0115] The data scheduling module 430 is used to schedule multiple I / O commands. As an example, this scheduling may include QoS processing of data. For instance, to avoid unreasonable load on a VM or volume, it is necessary to limit the bandwidth and packet rate of each device. For storage I / O data, it is also typically necessary to schedule I / O commands based on the volume's input / output operations per second (IOPS), bandwidth, and the overall VM's IOPS and bandwidth management.

[0116] The network protocol processing engine 460 is mainly used to process network protocols and ensure that data is reliably transmitted to the destination node. The aforementioned network protocol can be of various types, and this application does not impose any specific limitations on it. For example, the network protocol can be a remote direct data access (RDMA) protocol, or it can be a transmission control protocol (TCP) protocol, or it can be a custom reliable transmission protocol.

[0117] 2. Memory controller 420 The memory controller 420 is used to control the data interaction between the memory 440 and the CPU core 430. Specifically, the memory controller 420 receives memory access requests from the CPU core 430 and controls access to the memory 440 based on the memory access requests.

[0118] In one implementation example, each memory controller 420 addresses memory 440 via system bus 450. An arbitrator (not shown) is configured on the system bus to handle and coordinate contention for access by multiple CPU cores 430.

[0119] 3. CPU core 430 The CPU core 430 is the most important component of the CPU, also known as the CPU processing unit. It is manufactured from single-crystal silicon using a specific production process. All CPU calculations, command reception, command storage, and data processing are performed by the CPU core 430. Each CPU core 430 independently executes program instructions, utilizing parallel computing capabilities to accelerate program execution. Various CPU cores 430 have a fixed logical structure; for example, a CPU core 430 includes logical units such as L1 cache, L2 cache, execution unit, instruction set unit, and bus interface.

[0120] CPU core 430 is generally used for complex business processing and usually runs an operating system (OS), which can run complex business processing software, various management software, control plane software, etc.

[0121] 4. Memory 440 Memory 440 can be used as external memory for CPU core 430 to store some entries of the CPU core 430's program execution and data plane unloading engine 410.

[0122] This application does not specifically limit the external memory 440. As an example, the external memory 440 is double data rate (DDR) memory.

[0123] It should be understood that because the on-chip memory integrated within the data plane offloading engine 410 is a cache, it is fast but expensive, and can only be integrated in small quantities. Large-scale entries need to be stored in external memory 440. For example, external memory 440 stores large forwarding flow tables required for network forwarding, network connection send / receive queues, network connection context entries, I / O context entries, etc.

[0124] It should also be understood that when we say data is sent to the CPU, we are actually referring to data being sent to the external memory 440 of CPU core 430. From this article onwards, the terms "data sent to the CPU" and "data sent to CPU memory" refer to data being sent to the external memory 440 of CPU core 430.

[0125] 5. System bus 450 The system bus 450 can connect the data plane offloading engine 410 with the CPU core 430 and the memory controller 420, enabling communication between the data plane offloading engine 410, the CPU core 430, and the memory controller 420 via the system bus 450.

[0126] The following is based on Figure 4 Taking the hardware architecture shown as an example, combined with Figure 5 The example below details a specific implementation of how a network interface card (NIC) processes I / O write commands. It should be understood that... Figure 5 The examples are merely to help those skilled in the art understand the embodiments of this application, and are not intended to limit the embodiments of the application to... Figure 5 The specific numerical values ​​or specific scenarios will be considered. Those skilled in the art will obviously be able to make various equivalent modifications or changes based on the examples given, and such modifications and changes also fall within the scope of the embodiments of this application.

[0127] Figure 5 This is a schematic flowchart illustrating a method for a network interface card (NIC) to process I / O write commands, as provided in an embodiment of this application. Figure 5 As shown, the method may include steps 510-560, which will be described in detail below.

[0128] Step 510: The VM in host 210 issues an IO write command.

[0129] A VM on the host can initiate storage I / O write commands based on a storage I / O device interface (such as NVMe, SCSI, Virtio-BLK, Virtio-SCSI, etc.).

[0130] It should be noted that, Figure 5 The IO commands or pending IO commands mentioned in the illustrated embodiments can be understood as pending IO write commands.

[0131] It should be understood that a virtualized device can be identified by an identity (ID) in the embodiments of this application. The storage I / O devices described above are typically mapped to a function on a PCIe network. When single-root I / O virtualization (SR-IOV) is enabled, the storage I / O device can be either a physical function (PF) or a virtual function (VF). When scalable I / O virtualization (scalable-IOV) is enabled, the storage I / O device can also be an Analog Devices (ADI).

[0132] Since there are usually more Virtual Functions (VFs) in general, in VM application scenarios, VMs are primarily assigned VFs. For simplicity, the following descriptions will use VFs, but unless specifically specified as PFs, all descriptions using VFs are equally applicable to PFs. In other words, as an example, a VM running on host 210 can be identified using a VF.

[0133] Step 520: The processor 211 in the host 210 sends a notification message to the network card 400.

[0134] After the processor 211 sends the pending I / O write command to the memory 212, it can send a notification message to the network card 400. This notification message is used to notify the data plane unloading engine 410 that there is an I / O command to be processed.

[0135] As an example, the notification message could be a door bell (DB).

[0136] Step 530: The IO command processing module 412 in the network card 400 processes the IO write command to be processed.

[0137] For IO commands such as IO read and IO write commands, QoS control is usually required. For example, for the logical unit number (LUN) information (namespace in NVMe) in IO commands, the VM's LUN ID and the entry VF information can be converted into a global LUN ID, which can be the address information of the destination storage device. Alternatively, the VF + VM LUN ID can be used directly.

[0138] For ease of description, the LUN ID mentioned below can be understood as the global LUN ID.

[0139] It should be understood that VF information (e.g., VF ID) is not in the IO command, just like port attributes when a packet enters the network card; this information is not in the packet itself.

[0140] Step 540: The data scheduling module 430 in the network card 400 schedules the IO commands to be processed.

[0141] It should be understood that step 540 is optional. That is, if the above-mentioned IO commands to be processed are multiple IO commands, in order to avoid unreasonable load on a VM or a volume, it is necessary to limit the bandwidth and packet rate of each device. Therefore, it is necessary to schedule the above-mentioned multiple IO commands and select the IO commands that meet the conditions from the multiple IO commands. The data scheduling module 430 can store the IO commands that meet the conditions in the IO message receiving queue in the memory 440 of the network card 400.

[0142] like Figure 6 As shown, the data scheduling module 430 in the data plane offloading engine 410 schedules the multiple IO commands and stores the scheduled IO commands in the IO message receiving queue.

[0143] In one possible implementation, the data scheduling module 430 can perform primary scheduling of the multiple IO commands, selecting IO commands that meet the specified conditions. For example, the data scheduling module 430 can schedule the IO commands to be processed based on the storage address of the destination storage device where the IO commands to be processed are stored, selecting IO commands that meet the specified conditions. Alternatively, the data scheduling module 430 can also schedule the IO commands to be processed based on the virtual machine (VM) that issues the multiple IO commands from the host 210, selecting IO commands that meet the specified conditions.

[0144] In another possible implementation, the data scheduling module 430 can also perform multi-level scheduling on the multiple IO commands and select the IO commands that meet the conditions. Taking two-level scheduling as an example, the first level is based on LUNID to schedule IO commands, and the second level is based on VM to schedule IO commands. Specifically, the data scheduling module 430 can schedule the IO commands to be processed according to the storage address of the IO commands to be processed in the destination storage device and the virtual machine (VM) in the host 210 that issued the multiple IO commands, and select the IO commands that meet the conditions.

[0145] In this embodiment, if the size of the IO message receiving queue is small, it can be stored using the internal cache of the data plane offloading engine 410. However, if the size of the IO message receiving queue is large and the internal cache of the chip cannot hold it, these queues can be placed in the external memory 440 of the network card 400.

[0146] The eligible IO commands mentioned above can be those that measure the volume of data read or written by the IO command. It should be understood that these measurements may include, but are not limited to, measuring IOPS, the allowed number of IO operations per second, or IO bandwidth.

[0147] In this embodiment, a token bucket algorithm can be used for traffic measurement. The token bucket algorithm involves placing a certain number of tokens in a bucket, where one token allows the transmission of a specified unit of data (e.g., 1 byte). After transmitting 1 byte of data, a token is removed from the bucket. When there are no remaining tokens in the bucket, transmitting any amount of data is considered to exceed the rated bandwidth. Data transmission can only continue when there are remaining tokens in the bucket.

[0148] For example, two token buckets can be used to achieve more committed performance (guaranteed performance) and allowed peak performance (maximum allowed performance when processing power is available) for more precise control.

[0149] The token bucket algorithm is like water in a pond; it can both flow in and out. Tokens can be not only removed from the bucket but also added to it. To ensure data can be sent at any time, tokens can be added to the bucket. Therefore, the rate at which tokens are added to the bucket determines the data transmission rate. For example, if a user's bandwidth is set to 1GBps and each token represents 1Mbit, then it's sufficient to add 1000 tokens to the bucket every second.

[0150] Therefore, in order to achieve a more flexible QoS scheduling effect in this embodiment, the token bucket provides a token adjustment interface, and the CPU core 430 replenishes tokens based on the characteristics of the packets. For example, to avoid the VM continuously generating small I / O commands, the CPU core 430 can perform on-the-spot merging and appropriately replenish some tokens. In this way, insufficient bandwidth can be avoided, which would affect the user experience.

[0151] In the above technical solutions, QoS scheduling performed by the data plane can achieve more accurate and stable QoS guarantees and better performance. Supporting token replenishment and adjustment by CPU cores can further provide greater QoS flexibility.

[0152] Step 550: CPU core 430 processes the I / O commands.

[0153] After the IO commands are stored in the external memory 440 of the network card 400, the software on the CPU core 430 analyzes the IO commands. As an example, IO commands typically include information such as: LUN, logical block address (LBA), data size, and a list of scatter gathers (SG) pointing to the data block.

[0154] LUN represents the ID of the storage unit to be accessed, for example, Figure 2 The ID of the destination storage device 230 is shown. LBA indicates the block containing the data in the storage unit to be accessed, which is the starting logical address of the data stored in the LUN. size indicates the size of the data to be read after this address. CPU core 430 can address the destination storage device based on LUN, LBA, and size.

[0155] An SG list is a list of multiple addresses and lengths. An address plus a length constitutes a memory block, and multiple address blocks constitute the total memory size required to store I / O data in the host. In other words, an SG list describes a scattered data block using a sequence of starting address and length, with multiple data blocks combined into a logically contiguous data block.

[0156] It should be understood that an address plus a length in the SG list can correspond to the address and length information of the data mentioned above.

[0157] For I / O writes, the SG list indicates which parts of the host's memory (RAM) are read from and written to the destination storage device.

[0158] It should be noted that the SG itself only contains address and length information. In virtualization applications, the VF ID information must also be added in order to accurately map to the memory address of the corresponding VM.

[0159] In this embodiment, the CPU core 430 can perform storage service processing based on the address and length of the data to be processed in the host memory. This storage service processing can involve various steps, such as data splitting, data deduplication, data encryption, data compression, and data snapshots.

[0160] The following section uses data splitting as an example to describe in detail the specific implementation process of CPU core 430 for storage business processing based on the address and length of the data to be processed in the host memory.

[0161] It should be understood that in distributed storage systems commonly used in cloud computing, data is distributed across hundreds or thousands of destination storage devices. This allows for high-speed concurrent access to the data, achieving extremely high performance, as well as better security and scalability. This process is called data splitting. That is, splitting the data into fixed-size blocks, such as 4K, 8K, 16K, 32K, etc., and sending different blocks to different destination storage devices.

[0162] For I / O writes, the data to be written needs to be split into smaller blocks and sent to different destination storage devices. For I / O reads, different data blocks need to be retrieved from multiple destination storage devices and combined into a complete I / O data. Reading is the reverse process of writing.

[0163] Optionally, in some embodiments, to improve the efficiency of sending data to multiple destination storage devices, aggregation can be performed appropriately, combining multiple data blocks destined for the same destination storage device into a larger data packet, such as a 128KB packet, which is then sent out all at once. This avoids frequent calls to the protocol stack for data transmission, which could overload the CPU core 430.

[0164] The typical process of splitting data is as follows: Figure 7 As shown, IO data is horizontally split, dividing a single IO data point into multiple data blocks. Vertically, this generates individual data packets, each uniquely corresponding to a distributed remote destination storage device. Each data packet may include one data block, or it may include multiple data blocks destined for the same storage device.

[0165] In this embodiment, since the real data written by IO is stored in the memory 212 of the host 210 and not in the memory 440 of the network card 400, the process of the CPU core 430 partitioning the data is not based on the real data written by IO, but on the virtual data pointer of SG in the IO command (the address information and length information of the data in the memory 212 of the host 210), which points to the data block with the address and length of the corresponding host VF.

[0166] In other words, the embodiments of this application do not split the actual data written by IO, but split it based on the memory address and data length of the IO data of a certain VM stored in memory 212, thereby obtaining the split VF+SG information.

[0167] Step 560: CPU core 430 instructs network protocol processing engine 460 to send IO write data to at least one destination storage device at a remote location.

[0168] After processing the I / O command, CPU core 430 can generate a message packet, which includes a message header. The message header can carry information about the corresponding destination storage device, which is usually the network connection's send queue. CPU core 430 can put the split M data blocks into M message queues. Each data block does not store the actual data to be written, but rather VF+SG information.

[0169] CPU core 430 can also send storage data processing instructions to network protocol processing engine 460. These instructions are used to instruct network protocol processing engine 460 to obtain the actual data written by IO, encapsulate the actual data written by IO, and send the encapsulated message to the corresponding remote destination storage device, thereby realizing the IO write process.

[0170] As an example, the network protocol processing engine 460 reads the VF+SG information stored in each data block from M message queues, and initiates DMA to the VM corresponding to the VF, thereby obtaining the actual data of IO write corresponding to each data block based on the VF+SG information.

[0171] like Figure 8 As shown, for example, the network protocol processing engine 460 can receive message 1 sent by the CPU core 430, which contains: VF1+SG1, VF2+SG1, VF3+SG1, ..., VFm+SGn. The network protocol processing engine 460 can obtain or read the data corresponding to VF1+SG1, VF2+SG1, VF3+SG1, ..., VFm+SGn from the memory 212 of the host 210 through the PCIe interface.

[0172] For example, the network protocol processing engine 460 can receive messages 2 sent by the CPU core 430, which contain: VF1+SG2, VF2+SG2, VF3+SG2, ..., VFm+SGm. The network protocol processing engine 460 can obtain or read the data corresponding to VF1+SG2, VF2+SG2, VF3+SG2, ..., VFm+SGm from the memory 212 of the host 210 through the PCIe interface.

[0173] The network protocol processing engine 4600 can encapsulate the data obtained from the memory 212, and then combine it into a message that actually contains data before sending it to the remote destination storage device, thereby realizing the writing of the data written by IO to the destination storage device.

[0174] Optionally, the network interface card 400 can also determine whether all the data has been written to at least one destination storage device based on the context information of the IO command. When all the data has been written to the at least one destination storage device, the network interface card 400 sends an IO write complete message to the host 210 through the PCIe interface. It should be understood that the context information of the IO command may include the storage status of the data in the at least one destination storage device.

[0175] The following is based on Figure 4 Taking the hardware architecture shown as an example, combined with Figure 9 The example below details a specific implementation of how a network interface card (NIC) processes I / O read commands. It should be understood that... Figure 9 The examples are merely to help those skilled in the art understand the embodiments of this application, and are not intended to limit the embodiments of the application to... Figure 9 The specific numerical values ​​or specific scenarios will be considered. Those skilled in the art will obviously be able to make various equivalent modifications or changes based on the examples given, and such modifications and changes also fall within the scope of the embodiments of this application.

[0176] Figure 9 This is a schematic flowchart illustrating a method for a network interface card (NIC) to process I / O read commands, as provided in an embodiment of this application. Figure 9 As shown, the method may include steps 910-950, which will be described in detail below.

[0177] Step 910: The VM in host 210 issues an IO read command.

[0178] An IO read command is used to instruct the retrieval of data from the destination storage device and store that data in host 210. As an example, this data can be stored in the memory of the VM that issued the IO read command on host 210.

[0179] An IO read command can also be called an IO read message.

[0180] Step 915: The processor 211 in the host 210 sends a notification message to the network card 400.

[0181] Step 917: The data plane offloading engine 410 in the network card 400 forwards the IO read command to the external memory 440 of the network card 400.

[0182] Step 920: CPU core 430 on network card 400 processes IO read commands.

[0183] Software analysis of IO read commands on CPU core 430. IO read commands typically include information about the destination storage device, which may include, but is not limited to: LUN, LBA, data size, and the VF+SG list of the host 210's local memory for storing data.

[0184] For IO reads, the VF+SG information indicates where the data to be read from the destination storage device will be written to in the host 210's memory.

[0185] In this embodiment, the CPU core 430 can split the IO read command into multiple IO read messages based on the address information and length information (SG list) of the data read from the destination storage device and written to the memory of the host 210. Each of the multiple IO read messages includes information on each piece of data read from the destination storage device and written to the memory of the host 210.

[0186] It should be understood that the information for each piece of data written to the memory of the host 210 includes any one of the following: address information and length information (SG list) for each piece of data written to the memory of the host 210; or address information and length information for each piece of data written to the memory of the host 210 and VM information (VF+SG list) for the host 210 that issues the IO read command; or a first identifier ID and offset information for writing the data in the memory of the host 210, wherein the first ID is used to indicate the VM information for the host 210 that issues the IO read command and multiple sets of address information and length information (SG list) for writing the data in the VM.

[0187] Step 925: CPU core 430 instructs network protocol processing engine 460 to send message packets to the remote destination storage device.

[0188] After the CPU core 430 processes the IO read command, it can instruct the network protocol processing engine 460 to send the multiple IO read messages to multiple remote destination storage devices. The message may include a message header, which may carry the storage address information of the corresponding destination storage device; this information is typically the transmission queue of the network connection.

[0189] CPU core 430 can also notify network protocol processing engine 4600 to initiate network protocol sending processing, and network protocol processing engine 4600 sends IO read message packets to the corresponding destination storage device.

[0190] Step 930: The data plane offloading engine 410 receives the returned IO read response message from the destination storage device.

[0191] The destination storage device can read the corresponding storage data based on the storage address information in the IO read message packet, and feed the storage data back to the data plane offloading engine 410 through the IO read response message.

[0192] An IO read response message can be understood as a response message from the destination storage device to the received IO read command.

[0193] There are various formats for the IO read response message returned by the destination storage device. This application does not make any specific limitation on this format. It will be described in detail below in conjunction with step 940. It will not be described in detail here.

[0194] Step 940: The data plane offloading engine 410 writes the data in the IO read response message directly into the VM's memory.

[0195] In this embodiment, after receiving the data in the IO read response message, the data plane offloading engine 410 can directly write the data in the IO read response message directly to the VM memory that issued the IO read command, instead of first writing the data to the memory 440 of the network card 400 and then writing it from the memory 440 to the VM memory that issued the IO read command. That is, the memory 440 of the network card 400 does not store the data in the IO read response message.

[0196] The data plane offloading engine 410 needs to obtain the address and length information of each data block stored in the memory 212 of the host 210 based on the IO read message. This enables the data to be directly written into the VM memory of the host 210 without first writing it into the memory 440 of the network card 400 and then into the VM memory. In this way, the data does not need to be sent to the memory of the network card. On the one hand, this can reduce the resource consumption and bandwidth pressure of the network card memory. On the other hand, it can also reduce the overhead of the network card processor.

[0197] Specifically, the data plane offloading engine 410 can determine the VF+SG information stored in the memory 212 of the host 210 for each data block. There are several ways to obtain the VF+SG information, and the different implementation methods are described in detail below.

[0198] In one possible implementation, see [link to relevant documentation]. Figure 10 The IO read response message can include: the IO ID, offset information, and data for each data block. The data plane offloading engine 410 can query the corresponding IO context based on the IO ID of each data block in the IO read response message to obtain VF+SG information.

[0199] It should be understood that in this embodiment, after an IO read command is sent to the destination storage device, the CPU core 430 of the data plane offloading engine 410 can request IO context information. One IO context can correspond to one ID. This IO context information can be used to record the IO ID corresponding to each block, the VF to which the IO is processed, the local LUN and SG list of the VM, and the address and length of the data to be read in the destination storage device.

[0200] It should be noted that an IO read command can consist of multiple such blocks, which can be different blocks of different IOs.

[0201] For example, if a data block in an IO read message has an ID of ID1 and an offset of 2, then see... Figure 10 When performing an address lookup, the data plane offloading engine 410 can determine that the information stored in the memory 212 of the host 210 for the data block is VF1+SG2 based on the ID being ID1 and the offset information being 2. After obtaining the VF1+SG2 information, the data plane offloading engine 410 can directly write the data into the memory 212 of the host 210 using direct memory access (DMA) technology.

[0202] In another possible implementation, see Figure 11 The IO read response message may include a message ID (messageID, MSG ID) and data. The data plane offloading engine 410 can retrieve the VF+SG information from the MSDID table based on the MSG ID in the IO read response message.

[0203] It should be understood that in this embodiment, an ID and an MSG ID table can be created based on each IO read message. The MSG ID table records the VF+SG information of the data block corresponding to each data fragment. The sent IO read message can carry the MSG ID, and of course, it must also contain the length information of each fragment. The fragment order is consistent with the ID order in the MSG ID table.

[0204] When the IO read response message returns, the data plane offloading engine 410 retrieves the SG information from the MSDID table based on the MSG ID of the returned message. It then determines the specific address where the data needs to be placed based on the data block number. Finally, it writes the data to that address using DMA technology, thus enabling direct writing of data into the VM's memory.

[0205] In another possible implementation, see Figure 12An IO read response message can carry one or more VF+SGs along with the data. It should be understood that the one or more VF+SGs carried in the IO read response message represent the address information of the read data stored in the host's memory.

[0206] The data plane offloading engine 410 can directly obtain the VF+SG information of the corresponding data block through the IO read response message, and based on the VF+SG information, write the data directly into the memory 212 of the host 210 through direct memory access (DMA) technology.

[0207] Optionally, in some embodiments, to verify the legality and validity of the VF+SG information carried in the IO read response message, the data plane offloading engine 410 can compare the SG address in the IO read response message with the SG information in the corresponding SG permission table based on VF lookup to see if it is within the address range of the permission table. If it is within the address range of the permission table, the data plane offloading engine 410 directly writes the data to the host address corresponding to the VF+SG using DMA technology, thereby realizing the direct writing of data into the VM's memory. Otherwise, the write of this data block is discarded, and error handling is notified.

[0208] It should also be understood that, in this embodiment of the application, an SG permission table can be established for each VF for each IO read message. The SG permission table is created when the IO is initiated and deleted when the IO is completed. The SG permission table includes the VF+SG corresponding to writing data to VM memory.

[0209] Step 950: The data plane offloading engine 410 sends an IO read complete message to the host 210.

[0210] Since the returned IO read response message consists of M data packet return messages, an IO read is considered complete only after all M data packet return messages have arrived. Because this process is relatively simple, in this embodiment, it can be handled by the data plane offloading engine 410. This way, the returned data no longer needs to be processed by the CPU core 430 in the network card 400, improving processing performance.

[0211] As an example, the CPU core 430 in the network card 400 determines that all data reception is complete, and then sends an IO read completion message to the device emulation module 411 of the data plane offloading engine 410. There are several possible implementation methods, which are described in detail below.

[0212] In one possible implementation, based on the previously allocated IO context, each ID's corresponding IO context records not only the necessary information for IO processing but also the completion status of the message ID. The data plane offloading engine 410 obtains the IO context based on the IO ID carried in the returned message and sets the status of the corresponding message to the completion status according to the message ID. When the status of all messages is set to completion, the CPU core 430 in the network card 400 directly sends an IO read completion message to the device simulation module 411.

[0213] It should be understood that once the I / O read is complete, the corresponding I / O context becomes invalid, and the CPU can reassign that context.

[0214] In another possible implementation, based on the previously allocated IO context, each ID's corresponding IO context records not only the necessary information for IO processing but also the data block offset number. The returned message can carry the IO ID and the data block offset number. The data plane offloading engine 410 obtains the IO ID, acquires the IO context, and sets the corresponding data block completion flag based on the data block offset number. When all data is complete, the CPU core 430 in the network card 400 directly sends an IO read completion message to the device emulation module 411.

[0215] The above text combined Figures 1 to 12 The data processing method of the network card in the embodiments of this application is described in detail below. Figure 13 The following describes in detail the device embodiments of this application.

[0216] Figure 13 This is a schematic block diagram of a network interface card (NIC) 1300 provided in an embodiment of this application. The NIC 1300 is capable of performing... Figure 2 , Figure 5 , Figure 9 The steps of the method shown will not be detailed here to avoid repetition. The network interface card 1300 includes: an acquisition module 1310, a splitting module 1320, and a sending module 1330. The acquisition module 1310 is used to acquire a first input / output (IO) command, the first IO command instructing the writing of data stored in the local server to at least one peer server, the first IO command including the address information and length information of the data stored in the local server; The splitting module 1320 is used to split the data according to the address information and length information to obtain multiple sets of address information and length information; The sending module 1330 is further configured to obtain data corresponding to the multiple sets of address information and length information from the local server based on the multiple sets of address information and length information, and send the data to the at least one peer server.

[0217] Optionally, the acquisition module 1310 is specifically used to: acquire data corresponding to the multiple sets of address information and length information from the first VM based on the multiple sets of address information, length information, and the identifier of the first virtual machine VM that issued the first IO command in the local server.

[0218] Optionally, the network card's memory does not store the data.

[0219] Optionally, the network interface card 1300 further includes a scheduling module 1340. The acquisition module 1310 is also used to receive multiple IO commands; The scheduling module 1340 is used to select the first IO command from the plurality of IO commands based on the storage address of the plurality of IO commands stored in the peer server and / or the virtual machine (VM) that issues the plurality of IO commands in the local server.

[0220] Optionally, the network interface card 1300 includes a hardware engine for processing the data plane. The hardware engine selects the first IO command from the plurality of IO commands based on the storage address of the plurality of IO commands stored in the peer server and / or the virtual machine VM that issues the plurality of IO commands in the local server.

[0221] Optionally, the network interface card 1300 further includes: a determination module 1350. The determining module 1350 is used to determine whether all of the plurality of data have been written to the at least one peer server based on the context information of the first IO command, wherein the context information of the first IO command includes the storage status of the plurality of data in the at least one peer server. The sending module 1330 is further configured to send an IO write complete message to the local server via an interface when all of the multiple data have been written to the at least one peer server.

[0222] Optionally, the acquisition module 1310 is further configured to acquire a second IO command, the second IO command instructing the storage of at least one peer server to the local server, the second IO command including address information and length information of the data to be written to the local server; The splitting module 1320 is further configured to split the second IO command according to the address information and length information to obtain multiple IO read messages, each of the multiple IO read messages including information on each piece of data being written to the local server; The sending module 1330 is also used to send the IO read message to the at least one peer server respectively.

[0223] In one possible implementation, the information written to the local server for each piece of data includes any one of the following: address information and length information of each piece of data written to the local server; or address information and length information of each piece of data written to the local server and VM information of the local server that issues the second IO command; or a first identifier ID and offset information of the local server for writing the data, wherein the first ID is used to indicate the VM information of the local server that issues the second IO command and multiple sets of address information and length information of the VM for writing the data.

[0224] In another possible implementation, the network interface card 1300 further includes a storage module 1360. The acquisition module 1300 is further configured to receive a first IO read response message from the peer server, wherein the first IO read response message includes first data that the local server needs to obtain from the peer server; The determining module 1350 is further configured to determine the address information and length information of the first data written to the local server based on the first IO read response message; The storage module 1360 is used to store the first data in the local server according to the address information and length information.

[0225] In another possible implementation, the first IO read response message includes a first ID and offset information for writing the first data to the local server. The determining module 1350 is specifically used to: determine multiple sets of address information and length information of the data written in the VM based on the first ID; The storage module 1360 is specifically used to: determine the address information and length information of the first data to be written to the local server from the multiple sets of address information and length information according to the offset information.

[0226] In another possible implementation, the first IO read response message includes the address information and length information of the first data written to the local server, as well as the VM information of the local server that issued the second IO command. The determining module 1350 is specifically used to: determine the address information and length information of the first data written to the local server based on the information included in the first IO read response message.

[0227] In another possible implementation, the determining module 1350 is specifically used to: determine that the address information and length information included in the first IO read response message are within the range of the permission table, wherein the permission table includes multiple sets of address information and length information of the data written in the local server; The storage module 1360 is specifically used to: store the first data in the local server according to the address information and length information included in the first IO read response message.

[0228] In another possible implementation, the determining module 1350 is further configured to: determine whether all of the plurality of data have been written to the local server based on the context information of the second IO command, wherein the context information of the second IO command includes the completion status of the plurality of data stored in the local server; The sending module 1330 is also used to: when all the multiple data are written into the local server, send an IO read completion message to the local server through an interface.

[0229] The modules of the examples described above can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0230] The network card 1300 here can be represented as a functional module. The term "module" here can be implemented in software and / or hardware, without specific limitations.

[0231] For example, a "module" can be a software program, a hardware circuit, or a combination of both that implements the above functions. The hardware circuit may include an application-specific integrated circuit (ASIC), electronic circuitry, a processor (e.g., a shared processor, a proprietary processor, or a group processor) and memory for executing one or more software or firmware programs, integrated logic circuitry, and / or other suitable components that support the described functions.

[0232] This application also provides a chip that acquires and executes instructions to implement the above-described method.

[0233] Alternatively, as one implementation, the chip includes a processor and a data interface, through which the processor reads instructions stored in memory and executes the above method.

[0234] Alternatively, as one implementation, the chip may also include a memory storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to perform the methods described above.

[0235] This application also provides a computer-readable storage medium storing instructions for use in the methods described in the above-described method embodiments.

[0236] This application also provides a computer program product containing instructions for implementing the methods described in the above method embodiments.

[0237] In one implementation example, the processor can be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0238] In one implementation example, the memory can be volatile memory or non-volatile memory, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0239] In this article, the term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. A and B can be singular or plural. Additionally, the character " / " generally indicates an "or" relationship between the preceding and following objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0240] In this application, "multiple" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0241] In the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0242] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0243] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computing device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0244] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method includes: The network card obtains a first write command, which is used to instruct the first data to be written to the peer server. The first write command includes first address information and first length information. The first data is stored in the memory of the local server. The processor core in the network card splits the first data according to the first address information and the first length information to obtain multiple sets of second address information and second length information, wherein one set of second address information and second length information is used to describe a data block of the first data; The data plane offloading engine in the network interface card obtains the corresponding second data based on the second address information and the second length information, and The second data is sent to the peer server; The first data acquired by the network card is not stored in the network card's memory.

2. The method according to claim 1, characterized in that, The network interface card is simulated as a storage device of the local server.

3. The method according to claim 2, characterized in that, The network interface card is emulated as one of the following types of storage devices: NVMe storage device, virtual small computer system interface virtio-SCS device, or virtio-BLK device.

4. The method according to any one of claims 1-3, characterized in that, The first write command is of any of the following types: NVMe, SCSI, Virtio-BLK, or Virtio-SCSI write command.

5. The method according to any one of claims 1-3, characterized in that, The method includes: The network card encrypts the first data.

6. The method according to any one of claims 1-3, characterized in that, The first write command comes from the first virtual machine deployed in the local server.

7. The method according to claim 1, characterized in that, The method further includes: After sending all the second data contained in the first data to the peer server, the network card sends a write command completion message to the local server.

8. The method according to claim 1, characterized in that, The first write command is stored in the memory of the local server. The network interface card (NIC) acquires the first write command by: The network card retrieves the first write command from the memory via the bus.

9. A network interface card (NIC), characterized in that, The network interface card (NIC) is located in the server, and the NIC includes a processor core and a data plane offload engine. The processor core is used to split the first data according to the first address information and the first length information to obtain multiple sets of second address information and second length information. Each set of second address information and second length information is used to describe a data block of the first data. The first address information and the first length information are obtained from a first write command stored in the memory of the server. The first write command is used to instruct the first data to be written to the peer server. The data plane offloading engine is used to obtain the corresponding second data according to the second address information and the second length information, and send the second data to the peer server, wherein the first data is not stored in the memory of the network card.

10. The network interface card according to claim 9, characterized in that, The network interface card is simulated as a storage device for the local server.

11. The network interface card according to claim 9, characterized in that, The network interface card is emulated as one of the following types of storage devices: NVMe storage device, virtual small computer system interface virtio-SCS device, or virtio-BLK device.

12. The network interface card according to claim 9, characterized in that, The first write command is of any of the following types: NVMe, SCSI, Virtio-BLK, or Virtio-SCSI write command.

13. The network interface card according to any one of claims 9-12, characterized in that, The first write command comes from the first virtual machine deployed on the local server.

14. A server, characterized in that, The server includes a CPU and a network interface card as described in any one of claims 9-13. The network card receives a write command sent from the CPU and, based on the write command, sends the data stored in the server's memory to the peer server.

Images