Network card and method for processing data by the network card
By introducing a hardware processing engine into the network card, data is obtained and processed directly from the host memory, and the processed data is sent directly to the destination device, the bus pressure and memory consumption problems during network card storage service processing is solved, and the resource Efficient use.
Patent Information
- Application Number
- CN202011038826.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-28
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2040-09-28
AI Technical Summary
In the prior art, when performing storage service processing, network cards need to frequently obtain data from memory, resulting in high bus pressure and a large amount of memory is needed to store processed data, which increases program complexity and resource consumption.
A network card is designed to obtain data directly from the host's memory through a hardware processing engine for processing, and the processed data is directly sent to the destination storage device without storing it in the memory of the network card, reducing bus pressure and memory consumption.
It effectively reduces the bus pressure and memory resource consumption of network cards during storage service processing, and reduces program complexity and cost.
Smart Images

Figure CN113778320B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of servers, and more specifically, to a network card and a method for processing data by the network card. Background Art
[0002] With the rise of cloud computing, in order to reduce the occupancy rate of the host processor (e.g., central processing unit (CPU)), so that more host CPUs can be used for external sales, or to provide a better user experience, and to protect the user's CPU from the impact of more host background services, the services originally running on the host, such as management services, storage services, network services, etc., can be offloaded to the network card, and the CPU on the network card will process the above services. In this way, the host's CPU can basically be fully provided to the user.
[0003] Taking storage services as an example, in the relevant technical solutions, for input and output (IO) write commands, the network card needs to store the data to be written in the host into the memory of the network card, and the processor of the network card then obtains the data from the memory of the network card, processes the data for storage services, and writes the processed data to the destination storage device. For IO read commands, the network card needs to store the data obtained from the destination storage device into the memory of the network card first, and the processor of the network card then obtains the data from the memory of the network card, processes the data for storage services, and writes the processed data to the host.
[0004] In the above-mentioned related technical solutions, on the one hand, since the network card needs to repeatedly obtain data from the memory of the network card through the bus when performing data storage business processing, this puts a lot of pressure on the bus of the network card. On the other hand, the network card needs a large amount of memory to store these processed data, and the storage and management of these data greatly increases the complexity of the program and the consumption of network card memory resources. Summary of the invention
[0005] The present application provides a network card that can reduce the pressure on the network card bus and the consumption of the network card memory when performing storage business processing on data.
[0006] In a first aspect, a network card is provided, comprising: a processor and a hardware processing engine,
[0007] The processor is configured to obtain a first input / output (IO) command, where the first IO command indicates that first data stored in the local server is written into at least one opposite server;
[0008] The processor is further used to send first indication information to the hardware processing engine, where the first indication information is used to instruct the hardware processing engine to obtain the first data from the local server, and the first indication information includes address information and length information of the first data stored in the local server;
[0009] The hardware processing engine is used to directly process the first data and send the processed first data to the at least one opposite server during the process of obtaining the first data from the local server according to the address information and length information. The first data is not stored in the memory of the network card.
[0010] The network card directly processes the data for storage services when it obtains data from the host's memory, and directly sends the processed data to the destination storage device at the other end. The data obtained from the host's memory is not stored in the network card's memory, and the processed data is not stored in the network card's memory, but is directly sent to the destination storage device. In this way, on the one hand, since the network card does not need to repeatedly obtain data from the network card's memory through the bus when performing storage service processing on the data, the pressure on the network card's bus is reduced. On the other hand, the network card does not need a large amount of memory to store the processed data, which can reduce the complexity of the program and the consumption of memory resources.
[0011] In a possible implementation, the first indication information is further used to instruct the hardware processing engine to perform any one or more combinations of the following processing on the first data: data fingerprint calculation, data deduplication processing, data encryption, data compression, data redundancy error correction coding calculation,
[0012] The hardware processing engine is specifically used for: in the process of obtaining the first data from the local server according to the address information and the length information, directly processing the first data according to the first indication information.
[0013] In another possible implementation, the data redundancy error correction coding calculation may be, for example, an erasure code (EC) calculation.
[0014] In another possible implementation, the first indication information instructs the hardware processing engine to perform fingerprint calculation on the first data, and the hardware processing engine is specifically used to: perform the fingerprint calculation on the first data to obtain fingerprint information corresponding to the first data;
[0015] The hardware processing engine is further used to send fingerprint information corresponding to the first data to the processor;
[0016] The processor is further configured to asynchronously perform deduplication processing according to fingerprint information corresponding to the first data.
[0017] In another possible implementation, the hardware processing engine is specifically used to: after the first data is processed, put the processed first data into a sending queue of the hardware processing engine, and send it to the at least one peer server;
[0018] The processor is further configured to, when the first data includes duplicate data, send second indication information to a peer server storing the duplicate data through the hardware processing engine, wherein the second indication information is used to delete the duplicate data.
[0019] In another possible implementation, the hardware processing engine is specifically used to: after processing the first data, send a notification message to the processor, where the notification message is used to notify the processor to send the first data;
[0020] The processor is further configured to, in the process of placing the processed first data into the sending queue of the hardware processing engine, if it is determined that the first data includes duplicate data, no longer send the duplicate data to the sending queue, or
[0021] It is also used to send second indication information to the opposite server storing the duplicate data through the hardware processing engine after placing the processed first data into the sending queue of the hardware processing engine, if it is determined that the first data includes duplicate data, wherein the second indication information is used to delete the duplicate data.
[0022] In another possible implementation, the first indication information includes multiple groups of address information and length information, and the processor is further used to split the first data according to the address information and length information of the first data stored in the local server to obtain the multiple groups of address information and length information.
[0023] In another possible implementation, the processed first data includes the processed first data and redundant error correction data corresponding to the processed first data, the redundant error correction data is stored in the memory of the network card, and the processed first data is not stored in the memory of the network card.
[0024] The hardware processing engine is used to obtain the redundant error correction data from the memory of the network card and re-acquire the first data after the processor from the local server;
[0025] The hardware processing engine is used to send the processed first data and the redundant error correction data to the at least one peer server.
[0026] It should be understood that the redundant error correction data may be obtained by performing data redundant error correction coding calculation (eg, EC calculation) on the processed first data.
[0027] In another possible implementation, the processed first data includes the processed first data and redundant error correction data corresponding to the processed first data, the redundant error correction data is stored in the memory of the network card, and the processed first data is stored locally in the hardware processing engine.
[0028] The hardware processing engine is used to obtain the redundant error correction data from the memory of the network card and obtain the processed first data from the local device;
[0029] The hardware processing engine is used to send the processed first data and the redundant error correction data to the at least one peer server.
[0030] In another possible implementation, the processor is further configured to receive a first IO read response message from the peer server, where the first IO read response message includes second data that the local server needs to obtain from the peer server;
[0031] The hardware processing engine is further used to process the second data and directly write the processed second data into the local server. The second data is not stored in the memory of the network card.
[0032] In another possible implementation, the hardware processing engine is further used to determine to perform any one or more combinations of the following processing on the second data: data decryption, data decompression.
[0033] In another possible implementation, the first IO read response message carries decryption information, and the hardware processing engine is specifically configured to: determine and decrypt the second data according to the decryption information.
[0034] In a second aspect, a network card is provided, comprising: a processor and a hardware processing engine,
[0035] The processor is configured to receive a first IO read response message from a peer server, wherein the first IO read response message includes second data that the local server needs to obtain from the peer server;
[0036] The hardware processing engine is used to process the second data and directly write the processed second data into the local server. The second data is not stored in the memory of the network card.
[0037] In a possible implementation, the hardware processing engine is further used to determine to perform any one or more combinations of the following processing on the second data: data decryption, data decompression.
[0038] In another possible implementation, the first IO read response message carries decryption information, and the hardware processing engine is specifically configured to: determine and decrypt the second data according to the decryption information.
[0039] In a third aspect, a method for processing data by a network card is provided, wherein the network card includes a processor and a hardware processing engine, and the method includes:
[0040] The processor of the network card obtains a first input / output (IO) command, where the first IO command instructs writing first data stored in the local server into at least one opposite server;
[0041] The processor of the network card sends first indication information to the hardware processing engine, where the first indication information is used to instruct the hardware processing engine to obtain the first data from the local server, and the first indication information includes address information and length information;
[0042] In the process of obtaining the first data from the local server according to the address information and length information, the hardware processing engine of the network card directly processes the first data and sends the processed first data to the at least one opposite server. The first data is not stored in the memory of the network card.
[0043] In a possible implementation, the first indication information is also used to instruct the hardware processing engine to perform any one or more combinations of the following processing on the first data: data fingerprint calculation, data deduplication processing, data encryption, data compression, and data redundant error correction coding calculation. In the process of obtaining the first data from the local server according to the address information and length information, the hardware processing engine of the network card directly processes the first data according to the first indication information.
[0044] In another possible implementation, the data redundancy error correction coding calculation may be, for example, an erasure code (EC) calculation.
[0045] In another possible implementation, the first indication information instructs the hardware processing engine to perform fingerprint calculation on the first data, and the hardware processing engine of the network card performs the fingerprint calculation on the first data to obtain fingerprint information corresponding to the first data; the hardware processing engine of the network card sends the fingerprint information corresponding to the first data to the processor; and the processor of the network card performs deduplication processing based on the fingerprint information corresponding to the first data.
[0046] In another possible implementation, the method further includes: after the hardware processing engine of the network card processes the first data, placing the processed first data into a sending queue of the hardware processing engine and sending it to the at least one peer server; when the processor of the network card includes duplicate data in the first data, the processor sends second indication information to the peer server storing the duplicate data through the hardware processing engine, and the second indication information is used to delete the duplicate data.
[0047] In another possible implementation, the method further includes: after the hardware processing engine of the network card completes processing of the first data, the hardware processing engine of the network card sends a notification message to the processor, wherein the notification message is used to notify the processor to send the first data; when the processor of the network card puts the processed first data into the sending queue of the hardware processing engine, if it is determined that the first data includes duplicate data, the processor no longer sends the duplicate data to the sending queue; or after the processor of the network card puts the processed first data into the sending queue of the hardware processing engine, if it is determined that the first data includes duplicate data, the processor sends a second indication message to the peer server storing the duplicate data through the hardware processing engine, wherein the second indication message is used to delete the duplicate data.
[0048] In another possible implementation, the first indication information includes multiple groups of address information and length information, and the method further includes: the processor of the network card splits the first data according to the address information and length information of the first data stored on the local server to obtain the multiple groups of address information and length information.
[0049] In another possible implementation, the processed first data includes the processed first data and redundant error correction data corresponding to the processed first data, the redundant error correction data is stored in the memory of the network card, and the processed first data is not stored in the memory of the network card, and the method also includes: the hardware processing engine of the network card obtains the redundant error correction data from the memory of the network card, and re-obtains the first data after the processor from the local server; the hardware processing engine of the network card sends the processed first data and the redundant error correction data to the at least one opposite server.
[0050] In another possible implementation, the processed first data includes the processed first data and redundant error correction data corresponding to the processed first data, the redundant error correction data is stored in the memory of the network card, and the processed first data is stored locally in the hardware processing engine, and the method also includes: the hardware processing engine of the network card obtains the redundant error correction data from the memory of the network card, and obtains the processed first data from the local; the hardware processing engine of the network card sends the processed first data and the redundant error correction data to the at least one peer server.
[0051] In another possible implementation, the method further includes: the processor of the network card receives a first IO read response message from the opposite server, the first IO read response message including second data that the local server needs to obtain from the opposite server; the hardware processing engine of the network card processes the second data and directly writes the processed second data to the local server, and the second data is not stored in the memory of the network card.
[0052] In another possible implementation, the method further includes: the hardware processing engine of the network card determines to perform any one or more combinations of the following processing on the second data: data fingerprint calculation, data deduplication processing, data decryption, and data decompression.
[0053] In another possible implementation manner, the method further includes: a hardware processing engine of the network card determines and decrypts the second data according to the decryption information.
[0054] The beneficial effects of the third aspect and any possible implementation of the third aspect correspond to the beneficial effects of the first aspect and any possible implementation of the first aspect, and will not be elaborated on here.
[0055] In a fourth aspect, a method for processing data by a network card is provided, the network card comprising a processor and a hardware processing engine, the method comprising: the processor of the network card receives a first IO read response message from a peer server, the first IO read response message comprising second data that a local server needs to obtain from the peer server; the hardware processing engine of the network card processes the second data, and directly writes the processed second data to the local server, and the second data is not stored in the memory of the network card.
[0056] In a possible implementation, the method further includes: the hardware processing engine of the network card determines to perform any one or more combinations of the following processing on the second data: data decryption, data decompression.
[0057] In another possible implementation, the first IO read response message carries decryption information, and the method further includes: a hardware processing engine of the network card determines and decrypts the second data according to the decryption information.
[0058] In a fifth aspect, a network card is provided, comprising: a processor and a memory, wherein the processor runs instructions in the memory so that the computing device executes the method steps executed in the above-mentioned first aspect or any possible implementation manner of the first aspect.
[0059] In a sixth aspect, a network card is provided, comprising a processor and a memory, wherein the processor runs instructions in the memory so that the computing device executes the method steps executed in the second aspect or any possible implementation of the second aspect.
[0060] In a seventh aspect, a server is provided, comprising a network card as in the first aspect or any possible implementation of the first aspect or a network card as in the fifth aspect or any possible implementation of the fifth aspect.
[0061] In an eighth aspect, a server is provided, comprising a network card in the second aspect or any possible implementation of the second aspect or a network card in the sixth aspect or any possible implementation of the sixth aspect.
[0062] In a ninth aspect, a computer-readable storage medium is provided, comprising instructions; the instructions are used to implement the method steps performed in the third aspect or any possible implementation manner of the third aspect.
[0063] Optionally, as an implementation manner, the above-mentioned storage medium may specifically be a non-volatile storage medium.
[0064] In a tenth aspect, a computer-readable storage medium is provided, comprising instructions; the instructions are used to implement the method steps performed in the fourth aspect or any possible implementation manner of the fourth aspect.
[0065] Optionally, as an implementation manner, the above-mentioned storage medium may specifically be a non-volatile storage medium.
[0066] In an eleventh aspect, a chip is provided, which obtains instructions and executes the instructions to implement the method in the third aspect and any one of the implementation methods of the third aspect.
[0067] Optionally, as an implementation manner, the chip includes a processor and a data interface, and the processor reads instructions stored in the memory through the data interface to execute the method in the above-mentioned third aspect and any implementation manner of the third aspect.
[0068] Optionally, as an implementation method, the chip may also include a memory, in which instructions are stored, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the method in the third aspect and any one of the implementation methods of the third aspect.
[0069] In a twelfth aspect, a chip is provided, which obtains instructions and executes the instructions to implement the method in the above-mentioned fourth aspect or any possible implementation manner of the fourth aspect.
[0070] Optionally, as an implementation method, the chip includes a processor and a data interface, and the processor reads instructions stored in the memory through the data interface to implement the method in the above-mentioned fourth aspect or any possible implementation method of the fourth aspect.
[0071] Optionally, as an implementation method, the chip may also include a memory, in which instructions are stored, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to implement the method in the fourth aspect or any possible implementation method of the fourth aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0072] Figure 1 It is a schematic diagram of an architecture provided in an embodiment of the present application.
[0073] Figure 2 It is a schematic diagram of an application scenario provided in an embodiment of the present application.
[0074] Figure 3 It is a schematic flow chart of a method for processing data by a network card provided in an embodiment of the present application.
[0075] Figure 4 It is a structural diagram of a network card 400 provided in an embodiment of the present application.
[0076] Figure 5 It is a schematic flowchart of a method for a network card to process an IO write command provided in an embodiment of the present application.
[0077] Figure 6 It is a schematic block diagram of a data splitting process provided in an embodiment of the present application.
[0078] Figure 7 It is a schematic flowchart of a method for performing storage service processing on data by a service acceleration engine 450 provided in an embodiment of the present application.
[0079] Figure 8 It is a schematic flowchart of a method for a network card to process an IO read command provided in an embodiment of the present application. DETAILED DESCRIPTION
[0080] The technical solution in this application will be described below in conjunction with the accompanying drawings.
[0081] The present application will present various aspects, embodiments or features around a system including multiple devices, components, modules, etc. It should be understood and appreciated that each system may include additional devices, components, modules, etc., and / or may not include all devices, components, modules, etc. discussed in conjunction with the figures. In addition, combinations of these schemes may also be used.
[0082] In addition, in the embodiments of the present application, words such as "exemplary" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present concepts in a concrete way.
[0083] In the embodiments of the present application, "corresponding (corresponding, relevant)" and "corresponding (corresponding)" can sometimes be used interchangeably. It should be pointed out that when the distinction between them is not emphasized, the meanings they intend to express are consistent.
[0084] The network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are equally applicable to similar technical problems.
[0085] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0086] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.
[0087] With the rise of cloud computing, in order to reduce the occupancy rate of host processors (e.g., central processing unit (CPU)), more host CPUs can be used for external sales, or to provide a better user experience and protect the user's CPU from the impact of more host background business. Figure 1 As shown, today's cloud computing vendors are gradually offloading management services, storage services, network services, etc. that were originally running on the host to the network card, and the CPU on the network card processes the above services. In this way, the host's CPU can basically be fully provided to users.
[0088] Below Figure 2 The storage service is described in detail by taking the application scenario shown in the figure as an example.
[0089] like Figure 2 As shown, the scenario may include a host 210, a network card 220, and a destination storage device 230. Each device is described in detail below.
[0090] 1. Host 210
[0091] The host 210 may include at least one processor 211 and a memory 212 .
[0092] Optionally, the host 210 further includes a system bus 213 , wherein at least one processor 211 and a memory 212 are connected to the system bus 213 , respectively.
[0093] The processor 211 can access the memory 212 through the system bus 213. For example, the processor 211 can read and write data or execute code in the memory 212 through the system bus. The system bus 213 can be, for example, a quick path interconnect (QPI) or an ultra path interconnect (UPI). The system bus 213 is divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0094] In a possible implementation, the function of the processor 211 is mainly to interpret the instructions (or codes) of the computer program and process the data in the computer software. The instructions of the computer program and the data in the computer software can be stored in the memory 212.
[0095] The processor 211 is the computing core and control unit of the host 210. The processor 211 may include multiple processor cores. All calculations, command reception, command storage, and data processing in the processor 211 are performed by the processor core. The processor cores run program instructions independently and use the ability of parallel computing to speed up the program. The processor core has a fixed logical structure, for example, including logical units such as a first-level cache, a second-level cache, an execution unit, an instruction-level unit, and a bus interface.
[0096] Optionally, the processor 211 may be an integrated circuit chip and has the ability to process signals. As an example and not a limitation, the processor 110 is a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Among them, the general-purpose processor is a microprocessor, etc. For example, the processor 110 is a central processing unit (CPU).
[0097] The memory 212 is the main memory of the host 210 (also referred to as the memory of the host 210). The memory 212 is usually used to store various running software programs in the operating system, input and output (IO) commands issued by upper-layer applications, and information exchanged with external memory. In order to improve the access speed of the processor 211, the memory 212 needs to have the advantage of fast access speed. In some computer system architectures, dynamic random access memory (DRAM) is used as the memory 212. The processor 211 can access the memory through the memory controller ( Figure 1 (not shown) high-speed access to the memory 212, and performing read and write operations on any storage unit in the memory 212.
[0098] It should also be understood that the memory 212 in the embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).
[0099] 2. Network card 220
[0100] The network card 220 is used to realize the communication between the host 210 and other servers in the communication network (for example, Figure 1The network card 220 may be built into the host 210, or may be an external device of the host 210 and connected to the host 210 through an interface. The interface may be, for example, a peripheral component interface express (PCIE) interface.
[0101] For ease of description, Figure 2 In the description, the network card 220 is taken as an external device of the host 210.
[0102] The network card 220 may also include a memory 222 and a processor 221. Optionally, the network card 220 may also include a bus 223. The internal structure of the network card 220 may be similar to that of the host 210. For details, please refer to the above description of each part of the host 210, which will not be repeated here.
[0103] 3. Destination storage device 230
[0104] The destination storage device 230 may also include a memory, a processor, and a network card. Optionally, the destination storage device 230 may also include a bus. The internal structure of the destination storage device 230 may be similar to that of the host 210. For details, please refer to the above description of each part of the host 210, which will not be repeated here.
[0105] In the scenario of data processing and communication, the host 210 is used as the local end, and the destination storage device 230 is used as the other end of the communication. The virtual machine (VM) on the host 210 issues an input output (IO) command, and the processor 211 on the host 210 can send the IO command to the memory 212 through the bus 213. The network card 220 obtains the IO command from the memory 212 and processes the IO command.
[0106] Taking the IO commands as read commands and write commands as examples, the read command refers to a command issued by an application (e.g., VM) running on the host to instruct to read data from a remote device (e.g., a destination storage device), and the write command refers to a command issued by an application (e.g., VM) running on the host to instruct to write data to a remote device. The processor of the host can receive the IO command and store it in the memory, so that the IO command waits to be processed by the network card 220 and then sent to the destination storage device 230 as the communication peer.
[0107] Taking the IO write command as an example, before the network card 220 sends the IO write data stored in the memory 212 of the host 210 to the destination storage device 230, it is also necessary to process the IO write data for storage services, and then send the processed data to the destination storage device 230. The embodiment of the present application does not specifically limit the processing of storage services in the IO write process, which may include but is not limited to: data deduplication processing, data encryption processing, data compression processing, data redundancy error correction coding calculation (for example, EC calculation), etc.
[0108] Taking the IO read command as an example, after the network card 220 obtains the data to be read from the destination storage device 230, it can process the IO read data as a storage service, and then store the processed data in the memory 212. The embodiment of the present application does not specifically limit the processing of the storage service during the IO read process, which may include but is not limited to: data decryption processing, data decompression processing, etc.
[0109] In the related technical solution, taking the IO command as an IO write command as an example, the network card 220 needs to move the IO from the memory 212 of the host 210 to the memory 222 of the network card 220, and the processor 221 of the network card 220 performs storage service processing on the IO write data in the memory 222. Then, the processed data is stored in the memory 222 of the network card 220, so that the processor 221 of the network card 220 performs the next step of processing on the data in the memory 222.
[0110] Taking the IO command as an IO read command, after the network card 220 obtains the data to be read from the destination storage device 230, it needs to store the data in the memory 222 of the network card 220 first, and the processor 221 of the network card 220 performs storage service processing on the IO write data in the memory 222. Then, the processed data is stored in the memory 222 of the network card 220, so that the processor 221 of the network card 220 writes the data in the memory 222 into the memory 212 of the host 210.
[0111] In the above-mentioned related technical solutions, on the one hand, since the network card 220 needs to repeatedly obtain data from the memory 222 of the network card 220 through the bus 223 when performing storage business processing on the data, this brings great pressure to the bus 223 of the network card 220. On the other hand, the network card 220 needs a large amount of memory 222 to store these processed data, and the storage and management of these data greatly increases the complexity of the program and the consumption of memory 222 resources.
[0112] In view of this, the present application proposes a network card, which directly processes the data for storage services in the process of obtaining data from the host's memory, and directly sends the processed data to the destination storage device at the other end. The data obtained from the host's memory is not stored in the network card's memory, and the processed data is not stored in the network card's memory, but is directly sent to the destination storage device. In this way, on the one hand, since the network card does not need to repeatedly obtain data from the network card's memory through the bus when performing storage business processing on the data, the pressure on the network card's bus is reduced. On the other hand, the network card does not need a large amount of memory to store the processed data, which can reduce the complexity of the program and the consumption of memory resources.
[0113] Combine the following Figure 3 , a method for processing data by a network card provided in an embodiment of the present application is introduced in detail.
[0114] Figure 3 FIG. 1 is a schematic flow chart of a method for processing data by a network card provided in an embodiment of the present application. Figure 3 As shown, the method may include steps 310-330, and steps 310-330 are described in detail below.
[0115] Step 310: The processor of the network card obtains a first input / output (IO) command, where the first IO command instructs writing first data stored in the local server to at least one opposite server.
[0116] The first IO command may also include address information and length information of the data stored in the local server.
[0117] In the embodiment of the present application, the first IO command may be used to instruct to write data stored in the local server to at least one opposite server. As an example, the data may be stored in the memory of the local server.
[0118] As an example, the local server can be Figure 2 The host 210 in the above-mentioned peer server can be a storage device or a computing device with computing capabilities. In one example, the peer server is Figure 2 The destination storage device 230 in.
[0119] The network card can be used as an external device of the local server and connected to the local server through an interface. The interface can be a bus and interface standard (peripheral component interface express, PCIE) interface. For specific connection relationships, please refer to Figure 2 The description in will not be repeated here.
[0120] Step 320: The processor of the network card sends first indication information to the hardware processing engine of the network card, where the first indication information is used to instruct the hardware processing engine to obtain the first data from the local server, and the first indication information includes address information and length information where the first data is stored on the local server.
[0121] After processing the first IO command, the processor of the network card may send the first indication information to the hardware processing engine of the network card. As an example, the processor of the network card may split the data based on the address information and length information of the data stored in the first IO command on the local server to obtain multiple groups of address information and length information.
[0122] It should be understood that the address information and length information of the data can be used to describe a data block stored in the memory of the local server.
[0123] Step 330: In the process of obtaining the first data from the local server according to the address information and length information, the hardware processing engine of the network card directly processes the first data and sends the processed first data to the at least one opposite server. The first data is not stored in the memory of the network card.
[0124] After receiving the first indication information sent by the processor of the network card, the hardware processing engine of the network card can obtain the first data from the local server based on the address information and length information of the first data carried in the first indication information. In addition, in the process of obtaining the first data, the hardware processing engine of the network card directly processes the first data for storage services, and sends the processed first data to the at least one opposite server.
[0125] In the embodiment of the present application, the memory of the network card does not need to store the first data. Thus, on the one hand, since the data does not need to be read from the memory of the local server to the memory of the network card, the pressure on the network card bus can be reduced, and the overhead of the network card processor can also be reduced. On the other hand, the network card does not need a large amount of memory to store the data to be written, which reduces the complexity of the program and the consumption of memory resources.
[0126] In the above technical solution, the network card directly processes the data for storage services when acquiring data from the host's memory, and directly sends the processed data to the destination storage device at the other end. The data acquired from the host's memory is not stored in the network card's memory, and the processed data is not stored in the network card's memory, but is directly sent to the destination storage device. In this way, on the one hand, since the network card does not need to repeatedly acquire data from the network card's memory through the network card's bus when performing storage service processing on the data, the bus pressure of the network card is reduced. On the other hand, the network card does not need a large amount of memory to store the processed data, which can reduce the complexity of the program and the consumption of the network card's memory resources.
[0127] Combine the following Figure 4 , a possible structure of a network card provided in an embodiment of the present application is described in detail.
[0128] Figure 4 4 is a schematic diagram of the structure of a network card 400 provided in an embodiment of the present application. Figure 4 As shown, the network card 400 is connected to the host 210 via a PCIE interface.
[0129] The network card 400 may include: a data plane offload engine 410 , a memory controller 420 , a CPU core 430 , a memory 470 , and a system bus 460 .
[0130] 1. Data plane offload engine 410
[0131] The data plane offload engine 410 may also refer to the hardware processing engine mentioned above. As an example, the data plane offload engine 410 may be composed of hardware logic or an engine with a dedicated processing architecture that specifically supports data plane processing, such as a field programmable gate array (FPGA) or a network processor (NP). Its characteristic is that it can quickly forward messages, but it is not suitable for complex protocol processing.
[0132] It should be understood that the data plane offload engine 410 is more suitable for processing network forwarding, so it can usually handle the data plane of network forwarding. However, due to the complexity of the business, the data plane of the storage business is difficult to be completed directly by it, and usually requires the CPU core 430 to complete it.
[0133] The data plane offload engine 410 may be connected to the host 210 through a PCIE interface to obtain data stored in the memory 212 of the host 210. The data plane offload engine 410 may also be connected to the CPU core 430 and the memory controller 420 through a system bus 460, so as to achieve high-speed communication between each module inside the data plane offload engine 410 and the CPU core 430 and the memory controller 420.
[0134] The data plane offload engine 410 may include: a device simulation module 411 , an IO command processing module 412 , a data scheduling module 413 , a network protocol processing engine 440 , and a service acceleration engine 450 .
[0135] The functions of each module included in the data plane offload engine 410 are described in detail below.
[0136] The device simulation module 411 is mainly used to implement device simulation so that the network card 400 can present the required devices to the host or the virtual machine VM running on the host, such as the simulation of storage, network and other devices. As an example, the device simulation module 411 is used to implement the presentation of non-volatile memory express (NVMe) storage devices, small computer system interface (SCSI) devices, virtual small computer system interface (virtio-SCSI) devices, virtio-BLK devices, and other types of virtio devices.
[0137] It should be understood that device simulation can also be completed by hardware logic and CPU core. Usually, strong data interaction is completed by the data plane offload engine, and configuration-related functions are completed by the CPU core.
[0138] The IO command processing module 412 is used to perform simple analysis and processing on the IO command. As an example, it can be used to determine whether the IO command is a read command or a write command. As another example, for the IO read and write commands, scheduling is usually required to ensure the quality of service (QoS) of the device IO.
[0139] The data scheduling module 413 is used to schedule multiple IO commands. As an example, the scheduling may include QoS processing for data. For example, in order to avoid unreasonable load on a VM or a volume, it is necessary to limit the bandwidth and packet rate of each device. For storing IO data, it is usually necessary to schedule IO commands based on the input output operation per second (IOPS) of the volume, bandwidth, and IOPS and bandwidth management of the entire VM.
[0140] The service acceleration engine 450 is mainly used to perform storage service processing on data. The present application embodiment does not specifically limit the storage service processing process, which may include any one or any combination of the following: fingerprint calculation, compression, decompression, encryption, decryption, EC calculation, etc.
[0141] As an example, the service acceleration engine 450 may include: a fingerprint calculation engine 451 , a compression engine 452 , an encryption engine 453 , an EC calculation engine 454 , a decryption engine 455 , and a decompression engine 456 .
[0142] The service acceleration engine 450 can communicate with the CPU core 430 at high speed via the bus 460. In the vertical direction, the service acceleration engine 450 can also process the data from the host 210 in a pipeline-like manner by sequentially using any one or any combination of the following engines: fingerprint calculation engine 451, compression engine 452, encryption engine 453, and EC calculation engine 454.
[0143] In the embodiment of the present application, each engine in the service acceleration engine 450 can be configured whether to be transparent, stopped, or enabled. Among them, transparent means that the data directly skips the processing of this engine and directly enters the next level; stop means that the data stops entering the next level and is directly forwarded to the CPU core 430 for processing; enable means that the data enters the next level engine after normal processing.
[0144] Optionally, as a candidate, a selection module ( Figure 4 (not shown in the figure), the selection module selects which unloading engine or network card CPU is the next level to process the data. As an example, the order selected by the selection module can be determined by configuration.
[0145] For ease of description, the functions of each engine included in the business acceleration engine 450 are described in detail below.
[0146] The fingerprint calculation engine 451 is used to perform fingerprint calculation on the data obtained from the memory 212 of the host 210. As an example, the fingerprint calculation can be performed using a hash algorithm, for example, the data can be fingerprinted using the secure hash algorithm 256 (SHA256). The result of the fingerprint calculation can be used for data deduplication processing. For specific data deduplication processing, please refer to the description below, which will not be described in detail here.
[0147] The compression engine 452 is used to compress the data to obtain compressed data. Since the actual data block of the compressed data becomes smaller, the compression status can be recorded in the message header. In this way, the message can be assembled according to the actual compressed data when it is sent, saving network bandwidth.
[0148] Optionally, as a simplified solution, since the stored data is stored in blocks, the compressed data can be recorded in units of stored data blocks, and the free space is filled with 0.
[0149] The decompression engine 456 is used to decompress the data to obtain the data before compression.
[0150] The encryption engine 453 is used to encrypt data to obtain encrypted data.
[0151] The decryption engine 455 is used to decrypt the encrypted data to obtain the data before encryption.
[0152] EC calculation engine 454 is used to perform redundant protection on data. As an example, the EC method is to perform EC calculation on M data blocks to generate N redundant blocks, which are usually stored on different storage devices. If no more than N of the M data blocks are damaged at the same time, the data can be accurately restored, which is usually called M+N EC calculation. The larger the M number, the higher the data storage efficiency, and the larger the N number, the higher the reliability. In this way, as long as the number of nodes that fail at the same time does not exceed N, the original data can be restored. It should be understood that when M and N are 1, it is a typical 1+1 backup solution.
[0153] The network protocol processing engine 440 is mainly used to complete the processing of the network protocol to ensure that the data is reliably transmitted to the destination node. There can be multiple network protocols, which are not specifically limited in this application. For example, the network protocol can be a remote direct data access (RDMA) protocol, or a transmission control protocol (TCP) protocol, or a custom reliable transmission protocol.
[0154] 2. Memory controller 420
[0155] The memory controller 420 is used to control data interaction between the memory 470 and the CPU core 430. Specifically, the memory controller 420 receives a memory access request from the CPU core 430, and controls access to the memory 470 based on the memory access request.
[0156] In one implementation example, each memory controller 420 addresses the memory 470 via the system bus 460 . An arbiter (not shown in the figure) is configured in the system bus, and the arbiter is responsible for processing and coordinating the competing accesses of the multiple CPU cores 430 .
[0157] 3. CPU core 430
[0158] The CPU core 430 is the most important component of the CPU and can also be called the CPU processing unit. It is made of single crystal silicon using a certain production process. All CPU calculations, command acceptance, command storage, and data processing are performed by the CPU core 430. The CPU cores 430 run program instructions independently and use the ability of parallel computing to speed up the program. Various CPU cores 430 have a fixed logical structure. For example, the CPU core 430 includes logical units such as a primary cache, a secondary cache, an execution unit, an instruction level unit, and a bus interface.
[0159] The CPU core 430 is generally used for complex business processing and usually runs an operating system (OS), so that complex business processing software, various management software, control plane software, etc. can be run thereon.
[0160] 4. Memory 470
[0161] The memory 470 may be used as an external memory of the CPU core 430 , and is used to store program execution of the CPU core 430 and some table entries of the data plane offload engine 410 .
[0162] The embodiment of the present application does not specifically limit the external memory 470. As an example, the external memory 470 is a double data rate (DDR) memory.
[0163] It should be understood that since the on-chip memory integrated inside the data plane offload engine 410 is a high-speed cache, it is fast and costly, so it can only be integrated in small quantities. Large-scale table entries need to be stored in the external memory 470. For example, the external memory 470 stores a large forwarding flow table required for network forwarding, a network connection receiving and sending queue, a network connection context table entry, an IO context table entry, and the like.
[0164] It should also be understood that the data we often say is sent to the CPU, in fact, it is sent to the external memory 470 of the CPU core 430. The data sent to the CPU and sent to the CPU memory mentioned later in this article all refer to the data sent to the external memory 470 of the CPU core 430.
[0165] 5. System bus 460
[0166] The system bus 460 may connect the data plane offload engine 410 with the CPU core 430 and the memory controller 420 , so that the data plane offload engine 410 may communicate with the CPU core 430 and the memory controller 420 through the system bus 460 .
[0167] Below Figure 4 As an example, the hardware architecture shown in Figure 5 The example in the figure describes in detail a specific implementation method of the network card processing the IO write command. It should be understood that Figure 5 The examples are only intended to help those skilled in the art understand the embodiments of the present application, and are not intended to limit the embodiments of the present application to Figure 5 Those skilled in the art can obviously make various equivalent modifications or changes based on the examples given, and such modifications and changes also fall within the scope of the embodiments of the present application.
[0168] Figure 5 1 is a schematic flow chart of a method for a network card to process an IO write command provided in an embodiment of the present application. Figure 5 As shown, the method may include steps 510-570, and steps 510-570 are described in detail below.
[0169] Step 510: The VM in the host 210 issues an IO write command.
[0170] A VM of the host can initiate an IO write command for a storage service based on a storage IO device interface (such as NVMe, SCSI, Virtio-BLK, Virtio-SCSI, etc.).
[0171] It should be understood that a virtualized device can be identified by an identification (identity, ID). The above storage IO devices are usually mapped to a function (function) on PCIE. When single-root I / O virtualization (SR-IOV) is enabled, the above storage IO devices can be either physical functions (PF) or virtual functions (VF). Since there are usually more virtual functions (VF), in VM application scenarios, VMs are mainly assigned VFs. For the sake of simplicity, the following description is described in terms of VFs, but as long as it is not specifically emphasized that it can only be PF, all VF descriptions are also applicable to PF. That is to say, as an example, a VM running in the host 210 can be identified by VF.
[0172] Step 520 : The processor 211 in the host 210 sends a prompt message to the network card 400 .
[0173] After the processor 211 sends the IO write command to be processed to the memory 212, it may send a prompt message to the network card 400. The prompt message is used to notify the data plane offload engine 410 that there is an IO command to be processed.
[0174] As an example, the prompt message may be a door bell (DB).
[0175] Step 530: The IO command processing module 412 in the network card 400 processes the pending IO write command.
[0176] For IO commands such as IO read and IO write commands, QoS control is usually required. For example, for the logical unit number (LUN) information (namespace in NVMe) in the IO command, the VM's LUN ID and the entry's VF information can be converted into a global LUN ID, which can be the address information of the destination storage device.
[0177] For the convenience of description, the LUN ID that appears in the following text can be understood as a global LUN ID.
[0178] It should be understood that VF information (eg, VF ID) is not in the IO command, just like the port attributes of a message entering the network card, this information is not in the message.
[0179] Step 540: The data scheduling module 413 in the network card 400 schedules the IO commands to be processed.
[0180] It should be understood that step 540 is optional. That is, if the above-mentioned IO commands to be processed are multiple IO commands, in order to avoid unreasonable load of a VM or a volume, it is necessary to limit the bandwidth and packet rate of each device. Therefore, the data scheduling module 413 is required to schedule the above-mentioned multiple IO commands and select the IO commands that meet the conditions from the multiple IO commands. The data scheduling module 413 can store the IO commands that meet the conditions in the IO message receiving queue in the memory 470 of the network card 400.
[0181] The data scheduling module 413 in the data plane offload engine 410 schedules the multiple IO commands and stores the scheduled IO commands in an IO message receiving queue.
[0182] Step 550: The CPU core 430 splits the data based on the address information and length information in the IO command.
[0183] After the IO command is stored in the external memory 470 of the network card 400, the software on the CPU core 430 analyzes the IO command. As an example, the IO command may generally include information such as LUN, logical block address (LBA), data size, and a scatter gather (SG) list pointing to the data block.
[0184] LUN represents the ID of the storage unit to be accessed, for example, Figure 2 The ID of the destination storage device 230 shown. LBA indicates the block where the data on the storage unit to be accessed is located, that is, the logical address starting address of the data to be accessed stored in the LUN. Size indicates the size of the data after the address. The CPU core 430 can address the destination storage device based on LUN, LBA and size.
[0185] The SG list is a list of multiple addresses and lengths. An address + length constitutes a memory block, and multiple address blocks constitute the total memory size required to store IO data in the host. In other words, the SG list refers to a sequence of starting address + length to describe a dispersed data block, and multiple data blocks are combined into a logically continuous data block.
[0186] It should be understood that one address+length in the SG list may correspond to the address information and length information of the data mentioned above.
[0187] For IO write, the SG list indicates where the data is read from the host's storage (memory) and written to the destination storage device.
[0188] It should be noted that SG itself only has address and length information. When used in virtualization applications, VF ID information must be added to accurately map it to the memory address of the corresponding VM.
[0189] The specific implementation process of the CPU core 430 splitting the data based on the address of the data to be processed in the host memory and the data length is described in detail below.
[0190] It should be understood that for the distributed storage system commonly used in cloud computing, data will be stored in hundreds or thousands of destination storage devices. In this way, high-speed concurrent access to data can be achieved at the same time, extremely high performance can be obtained, and better security and scalability can be obtained. This process is the process of splitting the data. That is, the data is split into fixed-size data blocks, such as 4K, 8K, 16K, 32K, etc., and different data blocks are sent to different destination storage devices.
[0191] For IO writing, the data to be written needs to be split, and the different data blocks obtained after the splitting are sent to different destination storage devices.
[0192] Optionally, in some embodiments, in order to improve the efficiency of sending to multiple destination storage devices, it is usually possible to appropriately aggregate multiple data blocks sent to the same destination storage device into a larger data packet, such as a 128KB large packet, and send it out at one time. In this way, frequent calls to the protocol stack for data transmission can be avoided, which may cause CPU core 430 overload.
[0193] The process of CPU core 430 splitting data is as follows: Figure 6 As shown. The CPU core 430 splits the IO data horizontally, splitting one IO data into multiple data blocks. What is generated vertically is a data packet, each data packet uniquely corresponds to a distributed remote destination storage device. Each data packet may include a data block, or may also include multiple data blocks sent to the same destination storage device.
[0194] Since the real data written by IO is stored in the memory 212 in the host 210, but not in the memory 470 in the network card 400, the process of partitioning the data by the CPU core 430 is not based on the real data written by IO, but is based on the virtual data pointer of SG in the IO command (the address information and length information of the data in the memory 212 of the host 210), which points to the data block with the address and length of the corresponding host VF.
[0195] Step 560: The CPU core 430 instructs the data plane offload engine 410 to obtain the real data written by the IO.
[0196] After the CPU core 430 processes the IO command, it can generate a message packet, which includes a message header. The message header can carry the information of the corresponding destination storage device, which can usually be a sending queue of a network connection. The CPU core 430 can put the split M data blocks into M message queues, where each data block does not store the real data to be written, but the VF+SG information corresponding to each data block obtained by splitting. The VF+SG information is the same as the address information and length information indicating the real data written by the IO in the memory 212 of the host 210.
[0197] The CPU core 430 may also initiate a storage data processing instruction to the data plane offload engine 410, the instruction including the VF+SG information corresponding to each data block obtained by the above splitting, and instructing the data plane offload engine 410 to obtain the real data of the IO write from the memory 212 of the host 210 according to the VF+SG information corresponding to each data block. After the data plane offload engine 410 obtains the real data of the IO write, it may perform message encapsulation on the real data of the IO write, and send the encapsulated message to the corresponding remote destination storage device, thereby realizing the IO write process.
[0198] Optionally, the instruction is also used to instruct the data plane offload engine 410 to perform storage service processing on the acquired data, so that the data plane offload engine 410 can perform corresponding storage service processing on the acquired data according to the instruction.
[0199] The embodiments of the present application perform any one or more combinations of the following processing on the acquired data: data fingerprint calculation, data deduplication processing, data encryption, data compression, and data redundancy error correction coding calculation.
[0200] As an example, the data redundancy error correction coding calculation may be an erasure code (EC) calculation. The data generated by the data redundancy error correction coding calculation may be referred to as redundant error correction data.
[0201] Step 570: The data plane offload engine 410 processes the data while obtaining the real data written by the IO from the memory 212 in the host 210, and directly sends the processed data to at least one remote destination storage device.
[0202] As an example, the data plane offload engine 410 reads the VF+SG information corresponding to each data block from the M message queues respectively, and initiates DMA to the VM corresponding to the VF respectively, so as to obtain the real data written by the IO corresponding to each data block according to the VF+SG information.
[0203] In the embodiment of the present application, the data plane offload engine 410 can also directly process the data after reading the data. Specifically, after the data plane offload engine 410 obtains the real data written by IO from the memory 212, the data enters the service acceleration engine 450. After the service acceleration engine 450 processes the data for storage services, the processed data is directly sent to the network protocol processing engine 440, and the network protocol processing engine 440 directly sends the processed data to at least one remote destination storage device.
[0204] Combine the following Figure 7 , the specific implementation process of step 570 is described in detail.
[0205] Figure 7 FIG. 4 is a schematic flow chart of a method for performing storage service processing on data by a service acceleration engine 450 provided in an embodiment of the present application. Figure 7 As shown, the method includes steps 710-750, and steps 710-750 are described in detail below.
[0206] It should be understood that the processing process of the storage service by the service acceleration engine 450 in the embodiment of the present application is not specifically limited, and may include any one or any combination of the following: fingerprint calculation, compression, encryption, EC calculation, etc.
[0207] For ease of description, Figure 7 The China-Israel business acceleration engine 450 performs fingerprint calculation, compression, encryption, and EC calculation on the data and then sends it to the corresponding destination storage device.
[0208] Step 710 : the fingerprint calculation engine 451 in the service acceleration engine 450 performs fingerprint calculation on the data obtained from the memory 212 of the host 210 .
[0209] As an example, the fingerprint calculation can be performed using a hash algorithm. For example, the data can be fingerprinted using the SHA256 algorithm to obtain a hash value for each data block, that is, a data fingerprint for each data block. The data fingerprint of the data block can be used for data deduplication processing.
[0210] In the embodiment of the present application, the deduplication processing of data can be performed asynchronously. Specifically, the CPU core 430 can assist in the deduplication processing of data in an asynchronous manner.
[0211] For ease of understanding, the specific implementation process of data deduplication processing is described in detail below.
[0212] In a possible implementation, after the data plane offloading engine 410 calculates the data fingerprint of each data block, the data plane offloading engine 410 queries the deduplication database according to the data fingerprint of each data block. If the data fingerprint of the data block has a corresponding data address, the query result is notified to the CPU core 430, and the CPU core 430 performs subsequent deduplication processing. If the deduplication database does not have the data address corresponding to the data fingerprint of the data block, it means that there is no duplicate new data, and the data fingerprint and data address of the new data are inserted into the deduplication database.
[0213] In another possible implementation, after the data plane offload engine 410 calculates the data fingerprint of each data block, it directly sends the data fingerprint of each data block to the CPU event notification queue, and the CPU core 430 performs fingerprint comparison and deduplication processing.
[0214] It should be understood that the data address is the storage address of the data, usually LUN+LBA.
[0215] Step 720 : The compression engine 452 in the business acceleration engine 450 compresses the data in step 710 .
[0216] If data compression is enabled, the compression engine 452 performs compression processing to obtain compressed data. Since the actual data block of compressed data becomes smaller, the compression status needs to be recorded in the message header so that the message can be assembled according to the actual compressed data when it is sent, which can save network bandwidth.
[0217] As an example, since the storage data is stored in blocks (eg, 512, 4K), the compressed data can be recorded in units of storage data blocks, and the free space is filled with 0.
[0218] Step 730 : The encryption engine 453 in the business acceleration engine 450 encrypts the data compressed in step 720 .
[0219] If encryption is required, the encryption engine 453 obtains the corresponding encryption information and encrypts the data based on the encryption information. In one example, the encryption engine 453 can obtain the encryption information from the queue.
[0220] It should be understood that the encryption information is prepared in advance and placed in the queue by the CPU core 430 during stripe splitting and message encapsulation in step 560. For example, the encryption information may include an encryption key and an information value (IV), wherein the encryption key may be obtained by the CPU core 430 from a key server, and the IV may be generated by the CPU core 430 according to the LBA address of the data block.
[0221] Step 740: The EC calculation engine 454 in the business acceleration engine 450 performs data redundancy error correction coding calculation (eg, EC calculation) on the data.
[0222] After the encryption engine 453 completes the encryption processing on the data, the EC calculation engine 454 may perform EC calculation on the data. For example, the encryption engine 453 encrypts M data blocks to obtain M encrypted data blocks, and the EC calculation engine 454 performs EC calculation on the M encrypted data blocks to obtain N redundant data blocks.
[0223] It should be understood that the N redundant data blocks are calculated by the EC calculation engine 454 based on the encrypted M data blocks, so the N redundant data blocks are also encrypted data.
[0224] Step 750: Notify the network protocol processing engine 440 to start sending messages.
[0225] Since the network protocol processing engine 440 usually needs to establish a reliable data connection with each destination storage node based on the connection, an independent sending queue is usually required. There are many implementation methods here, and different implementation methods are described in detail below.
[0226] In a possible implementation, after each engine in the service acceleration engine 450 completes data processing, it can notify the CPU core 430, and the CPU core 430 then puts the message into the sending queue of the network protocol processing engine 440. This implementation requires the CPU core 430 to process, so the delay is relatively large and the performance is poor.
[0227] During this period of time, the CPU core 430 can perform deduplication processing synchronously or asynchronously. Synchronous deduplication processing means that if a data block with the same fingerprint is found, a direct data comparison is performed. If the data is found to be the same, the duplicate data block will not be sent down. Asynchronous deduplication processing means that data sending is not affected, and the deduplication check is performed asynchronously. If the data block is detected to be a duplicate block (the same fingerprint does not guarantee that the data is truly the same, so it is usually necessary to perform a true comparison of the data blocks), the deletion of the corresponding data block is initiated. Asynchronous deduplication can avoid the delay caused by increasing deduplication during the reading and writing process.
[0228] In another possible implementation, the data plane offload engine 410 directly puts the message into the sending queue of the network protocol processing engine 440, and then notifies the network protocol processing engine 440 to process. In this implementation, the CPU core 430 is not required to process, the latency is lower, and the performance is better.
[0229] At this time, for the deduplication processing, the data plane offload engine 410 needs to notify the CPU core 430, and the CPU core 430 performs deduplication processing asynchronously. If it is a new block, the fingerprint data and data address are inserted into the fingerprint library; if the data block is detected to be a duplicate block, after the address reference relationship is established, the CPU core 430 initiates a deletion command, and through the data block deletion process, the data plane offload engine 410 notifies the storage device of the duplicate block to delete the duplicate block. The advantage of this method is that it avoids the increase in read and write delays caused by the introduction of deduplication, thereby improving the user experience.
[0230] When the network protocol processing engine 440 is sending messages, the same batch of M+N data may not be sent immediately due to the different depths of the sending queues. In order not to store the data in the memory 222 of the network card 220, several possible implementations are described below.
[0231] In a possible implementation, after the EC calculation engine 454 calculates N redundant data blocks based on the M data blocks, the N redundant data blocks are saved in the memory 222 of the network card 220, and the remaining M data blocks are directly discarded. When the message is sent, the data plane offload engine 410 independently obtains the M data blocks from the memory 212 of the host 210, and performs the above-mentioned related processing. In this implementation, the sending of each message is independent of each other and is processed independently, which is simple to implement.
[0232] In another possible implementation, a buffer temporary storage area can be established in the memory 222 of the network card 220, and a cache area can be established in the chip of the data plane offload engine 410 (which can also be understood as the local area of the data plane offload engine 410). The N redundant data blocks calculated by the EC are placed in the cache area, and only when there is a cache conflict will the data in the cache be stored in the buffer temporary storage area of the memory 222. When the network protocol processing engine 440 sends a message, it reads data from the buffer. If there is data in the cache, it reads directly from the cache, otherwise it reads from the external memory. After the message is sent successfully, the corresponding cache data is cleared and the buffer is recycled.
[0233] In another possible implementation, the N redundant data blocks calculated by the EC are stored in the memory 222 of the network card 220, and the remaining M data blocks are directly put into the delayed deletion linked list in the data plane offload engine 410 instead of being directly deleted.
[0234] Optionally, in order for the network protocol processing engine 440 to quickly search for stored data, a hash index based on VF+address can be established for the linked list. A simple implementation is to create a linked list of result areas in the order of addition based on a hash table based on VF+address as the key.
[0235] Optionally, in some embodiments, it is not necessary to create an index table and directly query the linked list.
[0236] When the internal buffer of the temporary data plane unloading engine is insufficient, the data at the end of the delayed deletion list is directly discarded. If there is an index table, the corresponding index table record is deleted.
[0237] When the network protocol processing engine 440 sends a message, it first uses the VF+ address to query whether the data is in the delayed discard list. If so, it is read directly from the delayed deletion list. Otherwise, M data blocks are obtained again from the memory 212 of the host 210, and the previous data compression and encryption process needs to be repeated.
[0238] An IO may be composed of multiple stripes, and a stripe is an M+N. Only when all the data blocks of an M+N are sent to the destination storage device, the stripe is considered complete. Similarly, only when all the multiple stripes are sent to the destination storage device, the IO is considered complete, and the network card 400 replies to the host that the IO is successful.
[0239] Therefore, the shorter the time interval between the sending of M+N messages is, the better the effect will be. Furthermore, if the waiting time of these messages in the queue is shorter, the probability that the data in the above other implementations are stored in the chip will be higher. If the network protocol sending queue is empty, the message will be sent out immediately by the network protocol processing engine 440. The CPU core 430 can initiate EC calculation and sending based on the depth of the corresponding network protocol processing engine 440 sending queue, giving priority to the shallow queue. When the queue depth reaches a threshold value, the EC calculation API can be suspended to reduce the occupation of the chip's internal buffer.
[0240] After the network protocol processing engine 440 completes the message encapsulation, it sends it to the destination storage device to complete the IO data writing to the destination storage device. When receiving the completion message that all the stripe data and the corresponding EC redundant data of the current IO writing are written to the storage device, it returns to the host IO writing completion.
[0241] Below Figure 4 As an example, the hardware architecture shown in Figure 8 The example in the figure describes in detail a specific implementation method of the network card processing the IO read command. It should be understood that Figure 8 The examples are only intended to help those skilled in the art understand the embodiments of the present application, and are not intended to limit the embodiments of the present application to Figure 8 Those skilled in the art can obviously make various equivalent modifications or changes based on the examples given, and such modifications and changes also fall within the scope of the embodiments of the present application.
[0242] Figure 8 FIG. 1 is a schematic flow chart of a method for processing an IO read command by a network card provided in an embodiment of the present application. Figure 8 As shown, the method may include steps 810-850, and steps 810-850 are described in detail below.
[0243] Step 810: The VM in the host 210 issues an IO read command.
[0244] The IO read command is used to instruct to obtain the data to be read from the destination storage device and store the data in the host 210. As an example, the data can be stored in the memory corresponding to the VM that issues the IO read command in the host 210.
[0245] Step 820 : The processor 211 in the host 210 sends a prompt message to the network card 400 .
[0246] Step 830 : the data plane offload engine 410 in the network card 400 forwards the IO read command to the external memory 470 of the network card 400 .
[0247] Step 840: The CPU core 430 on the network card 400 processes the IO read command.
[0248] The software on the CPU core 430 analyzes the IO read command. The IO read command may generally include the destination storage device information to be read, and the storage device information includes but is not limited to: LUN, LBA, data size, and VF+SG list of the memory where the host 210 stores data locally.
[0249] For IO reads, the VF+SG information indicates where the data read from the destination storage device is written to the storage (memory) of the host 210 .
[0250] In the embodiment of the present application, the CPU core 430 can split the IO read command according to the address information and length information (SG list) of the data read from the destination storage device and written to the memory of the host 210 to obtain multiple IO read messages. Each of the multiple IO read messages includes information that each data in the data read from the destination storage device is written to the memory of the host 210.
[0251] It should be understood that the information of each data written to the memory of the host 210 includes any one of the following: the address information and length information (SG list) of each data written to the memory of the host 210; or the address information and length information of each data written to the memory of the host 210 and the VM information (VF+SG list) of the host 210 that issues the IO read command; or the first identification ID and the offset information of the data written in the memory of the host 210, the first ID is used to indicate the VM information of the host 210 that issues the IO read command and multiple sets of address information and length information (SG list) of the data written in the VM.
[0252] Step 825: The CPU core 430 instructs the network protocol processing engine 440 to send a message packet to the remote destination storage device.
[0253] After the CPU core 430 processes the IO read command, it can instruct the network protocol processing engine 440 to send the multiple IO read messages to multiple remote destination storage devices. The message may include a message header, which may carry storage address information of the corresponding destination storage device, which may generally be a sending queue of a network connection.
[0254] The CPU core 430 may also notify the network protocol processing engine 440 to initiate a network protocol sending process, and the network protocol processing engine 440 sends the IO read message packet to the corresponding destination storage device.
[0255] Step 850: The data plane offload engine 410 receives a returned IO read response message from the destination storage device.
[0256] The destination storage device can read the corresponding storage data based on the storage address information in the IO read message packet, and feed the storage data back to the data plane offload engine 410 through an IO read response message. The IO read response message can include: data written in the memory 212 of the host 210, decryption information, decompression information, etc.
[0257] The IO read response message can be understood as a response message fed back by the destination storage device to the received IO read command.
[0258] If the data in the IO read response message is encrypted data, the decryption engine 455 in the data plane offload engine 410 can decrypt the data in the IO read response message based on the decryption information to obtain the data before encryption.
[0259] If the data in the IO read response message is compressed data, the decompression engine 456 in the data plane offload engine 410 may further decompress the data in the IO read response message based on the decompression information to obtain the data before compression.
[0260] It should be noted that for IO reading, stripe data needs to be obtained from multiple destination storage devices and combined into complete IO data. Reading is the reverse process of writing. The difference is that if the data is correct, EC calculation processing is not required, because only when the data is wrong, it is necessary to restore normal data from redundant data.
[0261] After performing the above processing on the data in the read response message, the data plane offload engine 410 may directly write the data into the host / VM memory specified by the VF+SG.
[0262] In the above technical solution, all storage business processing is completed by reading the data once, avoiding the bandwidth and latency caused by repeated calls to the acceleration module and repeated CPU processing in the usual way. Data does not need to be sent to the network card memory, which greatly reduces the bandwidth requirements between the data plane and the network card CPU memory, reduces costs, and improves performance.
[0263] An embodiment of the present application also provides a chip, which obtains instructions and executes the instructions to implement the above method.
[0264] Optionally, as an implementation manner, the chip includes a processor and a data interface, and the processor reads instructions stored in the memory through the data interface to execute the above method.
[0265] Optionally, as an implementation manner, the chip may further include a memory, in which instructions are stored, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the above method.
[0266] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions, and the instructions are used in the method in the above method embodiment.
[0267] The embodiment of the present application also provides a computer program product including instructions, wherein the instructions are used to implement the method in the above method embodiment.
[0268] In one implementation example, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0269] In one implementation example, the memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).
[0270] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.
[0271] In this application, "multiple" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.
[0272] In various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0273] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0274] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computing device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0275] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A network card, characterized in that: The network card includes: a processor and a hardware processing engine, The processor is used to obtain a first input / output (IO) command, where the first IO command indicates that first data stored in the local server is written into at least one opposite server; The processor is further used to send first indication information to the hardware processing engine, where the first indication information is used to instruct the hardware processing engine to obtain the first data from the local server, and the first indication information includes address information and length information; The hardware processing engine is used to directly process the first data and send the processed first data to the at least one opposite server during the process of obtaining the first data from the local server according to the address information and length information. The first data is not stored in the memory of the network card.
2. The network card according to claim 1, characterized in that: The first indication information is further used to instruct the hardware processing engine to perform any one or more combinations of the following processing on the first data: data fingerprint calculation, data deduplication processing, data encryption, data compression, data redundancy error correction coding calculation, The hardware processing engine is specifically used for: in the process of obtaining the first data from the local server according to the address information and the length information, directly processing the first data according to the first indication information.
3. The network card according to claim 2, characterized in that: The first indication information instructs the hardware processing engine to perform fingerprint calculation on the first data, The hardware processing engine is specifically used to perform the fingerprint calculation on the first data to obtain fingerprint information corresponding to the first data; The hardware processing engine is further used to send fingerprint information corresponding to the first data to the processor; The processor is further configured to perform deduplication processing according to fingerprint information corresponding to the first data.
4. The network card according to claim 1, characterized in that: The hardware processing engine is specifically used to: after processing the first data, put the processed first data into a sending queue of the hardware processing engine, and send it to the at least one peer server; The processor is further configured to, when the first data includes duplicate data, send second indication information to a peer server storing the duplicate data through the hardware processing engine, wherein the second indication information is used to delete the duplicate data.
5. The network card according to claim 1, characterized in that: The hardware processing engine is specifically used to: after processing the first data, send a notification message to the processor, where the notification message is used to notify the processor to send the first data; The processor is further configured to, in the process of placing the processed first data into the sending queue of the hardware processing engine, if it is determined that the first data includes duplicate data, no longer send the duplicate data to the sending queue; or It is also used to send second indication information to the opposite server storing the duplicate data through the hardware processing engine after placing the processed first data into the sending queue of the hardware processing engine, if it is determined that the first data includes duplicate data, wherein the second indication information is used to delete the duplicate data.
6. The network card according to claim 1, characterized in that: The first indication information includes multiple sets of address information and length information. The processor is further configured to split the first data according to the address information and length information of the first data stored in the local server to obtain the multiple groups of address information and length information.
7. The network card according to claim 1, characterized in that: The processed first data includes the processed first data and redundant error correction data corresponding to the processed first data, the redundant error correction data is stored in the memory of the network card, and the processed first data is not stored in the memory of the network card, The hardware processing engine is used to obtain the redundant error correction data from the memory of the network card and re-acquire the first data after the processor from the local server; The hardware processing engine is used to send the processed first data and the redundant error correction data to the at least one peer server.
8. The network card according to claim 1, characterized in that: The processed first data includes the processed first data and redundant error correction data corresponding to the processed first data, the redundant error correction data is stored in the memory of the network card, and the processed first data is stored locally in the hardware processing engine. The hardware processing engine is used to obtain the redundant error correction data from the memory of the network card and obtain the processed first data from the local device; The hardware processing engine is used to send the processed first data and the redundant error correction data to the at least one peer server.
9. The network card according to any one of claims 1 to 8, characterized in that: The processor is further configured to receive a first IO read response message from the opposite server, wherein the first IO read response message includes second data that the local server needs to obtain from the opposite server; The hardware processing engine is further used to process the second data and directly write the processed second data into the local server. The second data is not stored in the memory of the network card.
10. The network card according to claim 9, characterized in that: The hardware processing engine is further used to determine to perform any one or more combinations of the following processing on the second data: data fingerprint calculation, data deduplication processing, data decryption, and data decompression.
11. The network card according to claim 10, characterized in that: The first IO read response message carries decryption information, The hardware processing engine is specifically used to: determine and decrypt the second data according to the decryption information.
12. A method for processing data by a network card, characterized in that: The network card includes a processor and a hardware processing engine, and the method includes: The processor of the network card obtains a first input / output (IO) command, where the first IO command instructs writing first data stored in the local server into at least one opposite server; The processor of the network card sends first indication information to the hardware processing engine, where the first indication information is used to instruct the hardware processing engine to obtain the first data from the local server, and the first indication information includes address information and length information; In the process of obtaining the first data from the local server according to the address information and length information, the hardware processing engine of the network card directly processes the first data and sends the processed first data to the at least one opposite server. The first data is not stored in the memory of the network card.
13. The method according to claim 12, characterized in that The first indication information is further used to instruct the hardware processing engine to perform any one or more combinations of the following processing on the first data: data fingerprint calculation, data deduplication processing, data encryption, data compression, data redundancy error correction coding calculation, The hardware processing engine of the network card directly processes the first data in the process of acquiring the first data from the local server according to the address information and the length information, including: In the process of acquiring the first data from the local server according to the address information and the length information, the hardware processing engine of the network card directly processes the first data according to the first indication information.
14. The method according to claim 13, characterized in that The first indication information instructs the hardware processing engine to perform fingerprint calculation on the first data, The hardware processing engine of the network card directly processes the first data in the process of acquiring the first data from the local server according to the address information and the length information, including: The hardware processing engine of the network card performs the fingerprint calculation on the first data to obtain fingerprint information corresponding to the first data; The hardware processing engine of the network card sends fingerprint information corresponding to the first data to the processor; The processor of the network card performs deduplication processing according to the fingerprint information corresponding to the first data.
15. The method according to claim 12, characterized in that The method further comprises: After the hardware processing engine of the network card processes the first data, the hardware processing engine puts the processed first data into a sending queue of the hardware processing engine and sends the processed first data to the at least one peer server; When the first data includes duplicate data, the processor of the network card sends second indication information to a peer server storing the duplicate data through the hardware processing engine, where the second indication information is used to delete the duplicate data.
16. The method according to claim 12, characterized in that The method further comprises: After the hardware processing engine of the network card has processed the first data, the hardware processing engine of the network card sends a notification message to the processor, where the notification message is used to notify the processor to send the first data; When the processor of the network card places the processed first data into the sending queue of the hardware processing engine, if it is determined that the first data includes duplicate data, the processor of the network card no longer sends the duplicate data to the sending queue; or After the processor of the network card puts the processed first data into the sending queue of the hardware processing engine, if it is determined that the first data includes duplicate data, it sends second indication information to the opposite server storing the duplicate data through the hardware processing engine, and the second indication information is used to delete the duplicate data.
17. The method according to claim 12, characterized in that The first indication information includes multiple groups of address information and length information, and the method further includes: The processor of the network card splits the first data according to the address information and length information of the first data stored in the local server to obtain the multiple groups of address information and length information.
18. The method according to claim 12, characterized in that The processed first data includes the processed first data and redundant error correction data corresponding to the processed first data, the redundant error correction data is stored in the memory of the network card, and the processed first data is not stored in the memory of the network card, and the method further includes: The hardware processing engine of the network card obtains the redundant error correction data from the memory of the network card, and re-obtains the first data after the processor from the local server; The hardware processing engine of the network card sends the processed first data and the redundant error correction data to the at least one opposite server.
19. The method according to claim 12, characterized in that The processed first data includes the processed first data and redundant error correction data corresponding to the processed first data, the redundant error correction data is stored in the memory of the network card, and the processed first data is stored locally in the hardware processing engine, and the method further includes: The hardware processing engine of the network card obtains the redundant error correction data from the memory of the network card, and obtains the processed first data from the local; The hardware processing engine of the network card sends the processed first data and the redundant error correction data to the at least one opposite server.
20. The method according to any one of claims 12 to 19, characterized in that The method further comprises: The processor of the network card receives a first IO read response message from the opposite server, where the first IO read response message includes second data that the local server needs to obtain from the opposite server; The hardware processing engine of the network card processes the second data and directly writes the processed second data into the local server. The second data is not stored in the memory of the network card.
21. The method according to claim 20, characterized in that The method further comprises: The hardware processing engine of the network card determines to perform any one or more combinations of the following processing on the second data: data fingerprint calculation, data deduplication processing, data decryption, and data decompression.
22. The method according to claim 21, characterized in that The method further comprises: The hardware processing engine of the network card determines and decrypts the second data according to the decryption information.
Citation Information
Patent Citations
High-speed and safe data exchange method based on intelligent network card
CN109218308A
