IO request processing method, intelligent network card, server and server system

Abstract

The invention relates to the technical field of cloud computing and intelligent network cards, and provides an IO request processing method, an intelligent network card, a server and a server system.The method comprises the steps that hardware logic firmware receives an IO request sent by the server through a block device according to a preset PCI address; the hardware logic firmware sends the IO request to a queue corresponding to a preset performance level of the PCI address according to the preset performance level of the PCI address and a mapping relation, the mapping relation represents a corresponding relation between the performance level and the queue, and different performance levels correspond to queues with different configurations; one queue is pre-bound with a processing core of a CPU (Central Processing Unit) for processing the IO request in the queue; and the CPU sends the IO request in the queue corresponding to the performance level preset by the PCI address to the storage device. According to the embodiment, the IO request processing efficiency of the intelligent network card can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud computing and smart network cards, and in particular to an IO request processing method, a smart network card, a server and a server system. Background Art

[0002] The IT infrastructure of cloud data centers, such as computing servers and storage devices, realizes cloud computing, cloud storage and cloud services by providing customers with cloud computing power and storage resources. As the service network rate of data centers is upgraded from 10GE to 25GE and 100GE, the network data processing brought by high bandwidth will consume a large amount of server computing resources. In order to improve the effective computing power and service performance of the server, smart network cards can be used for network offloading, storage offloading and other processing to release the server's CPU resources.

[0003] A mainstream architecture of SNC (Smart NIC) is the hardware logic firmware + multi-core CPU architecture, which has both high performance and flexibility and is more widely adopted. For the storage offload function, the hardware logic firmware completes the storage virtualization function and the logic of the virtio_blk backend driver; the multi-core CPU completes the communication with the backend storage device. In this way, it can achieve high performance of hardware forwarding and support access to different types of backend storage devices, such as IPSAN (IP Storage Area Network), NVME (Nonvolatile Memory Express), CEPH (a distributed storage system) and other types of backend storage devices.

[0004] With the development of storage technology, storage vendors have launched physical storage with higher storage performance. How to provide faster storage access on Smart NICs, fully utilize the potential of physical storage, and avoid it becoming a bottleneck for storage access has become a technical problem that needs to be solved on Smart NICs. Summary of the invention

[0005] The object of the present invention is to provide an IO request processing method, an intelligent network card, a server and a server system, which can improve the efficiency of the intelligent network card in processing IO requests.

[0006] The embodiments of the present invention can be implemented as follows:

[0007] In a first aspect, the present invention provides an IO request processing method, which is applied to a smart network card, wherein the smart network card includes a hardware logic firmware and a central processing unit CPU, wherein the hardware logic firmware is connected to a server through a peripheral device interconnection PCI bus, wherein the server is loaded with a block device, and wherein the CPU is connected to a storage device through a network, wherein the method includes:

[0008] The hardware logic firmware receives the IO request sent by the server through the block device according to a preset PCI address, where the PCI address is an address used by the block device for addressing using a PCI bus;

[0009] The hardware logic firmware sends the IO request to a queue corresponding to the performance level preset for the PCI address according to the performance level and mapping relationship preset for the PCI address, wherein the mapping relationship represents the corresponding relationship between the performance level and the queue, and the block devices of the PCI addresses corresponding to different performance levels correspond to storage hard disks with different performances, and a queue is pre-bound to a CPU processing core for processing the IO request in the queue;

[0010] The CPU sends the IO request in the queue corresponding to the performance level preset at the PCI address to the storage device.

[0011] In an optional implementation manner, the step of sending the IO request to a queue corresponding to the performance level preset for the PCI address according to the performance level preset for the PCI address and the mapping relationship includes:

[0012] The hardware logic firmware encapsulates the PCI address and the IO request into an IO message;

[0013] The hardware logic firmware sends the IO message to a queue corresponding to a performance level of the PCI address.

[0014] In an optional implementation, there are multiple IO requests, each of which is encapsulated into an IO message, a message header of each IO message carries the PCI address, there are multiple queues corresponding to the performance level of the PCI address, and the step of the hardware logic firmware sending the IO message to the queue corresponding to the performance level of the PCI address includes:

[0015] The hardware logic firmware sends multiple IO messages in parallel to multiple queues corresponding to the performance levels of the PCI addresses.

[0016] In an optional implementation manner, the CPU pre-creates a sub-block device, the sub-block device communicates with the storage device, and the step of the CPU sending the IO request in the queue corresponding to the performance level preset for the PCI address to the storage device includes:

[0017] The CPU uses a processing core of the CPU bound to a queue corresponding to the performance level of the PCI address to take out the IO message from the queue corresponding to the performance level of the PCI address and parse the PCI address and the IO request from the IO message;

[0018] The CPU determines the sub-block device bound to the parsed PCI address according to the parsed IO request, the PCI address, and the binding relationship between the sub-block device and the PCI address by using the CPU processing core bound to the queue corresponding to the performance level of the PCI address;

[0019] The CPU uses a processing core of the CPU bound to a queue corresponding to a performance level of the PCI address to send the parsed IO request to the storage device through a sub-block device bound to the parsed PCI address.

[0020] In an optional embodiment, the performance level includes a first level and a second level, the performance of the first level is greater than the performance of the second level, the number of queues corresponding to the first level is greater than the number of queues corresponding to the second level and / or the depth of the queues corresponding to the first level is greater than the depth of the queues corresponding to the second level.

[0021] In an optional implementation manner, the CPU pre-creates a sub-block device, and there is a binding relationship between the sub-block device and a processing core of the CPU, and the method further includes:

[0022] The CPU receives a response request from the storage device in response to the IO request by using a pre-created sub-block device, wherein the sub-block device receiving the response request is the same as the sub-block device sending the IO request;

[0023] The CPU uses the processing core of the CPU bound to the sub-block device that responds to the request to send the response request to the queue bound to the processing core of the CPU bound to the sub-block device that responds to the request;

[0024] The hardware logic firmware takes out the response request from the queue bound to the processing core of the CPU bound to the sub-block device of the response request, and sends it to the block device through the PCI bus.

[0025] In an optional implementation, the sub-block device is bound to a preset PCI address of the block device, and the CPU uses the processing core of the CPU bound to the sub-block device that responds to the request to send the response request to a queue bound to the processing core of the CPU bound to the sub-block device that responds to the request, comprising:

[0026] The CPU uses a processing core of the CPU bound to the sub-block device that responds to the request to encapsulate the PCI address of the block device bound to the sub-block device that responds to the request and the response request into a response message;

[0027] The CPU uses the processing core of the CPU bound to the sub-block device that responds to the request to send the response message to the queue bound to the processing core of the CPU bound to the sub-block device that responds to the request;

[0028] The step of the hardware logic firmware taking out the response request from the queue bound to the processing core of the CPU bound to the sub-block device of the response request, and sending it to the block device through the PCI bus includes:

[0029] The hardware logic firmware takes out the response message from the queue bound to the processing core of the CPU bound to the sub-block device that responds to the request, and parses the PCI address and the response request from the response message;

[0030] The hardware logic firmware sends the parsed response request to the block device through the PCI bus according to the parsed PCI address.

[0031] In a second aspect, the present invention provides a smart network card, the smart network card includes a hardware logic firmware and a central processing unit CPU, the hardware logic firmware is connected to a server through a peripheral device interconnection PCI bus, the server is loaded with a block device, and the CPU is connected to a storage device through a network;

[0032] The hardware logic firmware is used to receive an IO request sent by the server through the block device according to a preset PCI address, where the PCI address is an address used by the block device for addressing using a PCI bus;

[0033] The hardware logic firmware is further used to send the IO request to a queue corresponding to the performance level preset for the PCI address according to the performance level and mapping relationship preset for the PCI address, wherein the mapping relationship represents the corresponding relationship between the performance level and the queue, and different performance levels correspond to queues with different configurations, and one queue is pre-bound to a CPU processing core for processing the IO request in the queue;

[0034] The CPU is used to send the IO request in the queue corresponding to the performance level preset at the PCI address to the storage device.

[0035] In a third aspect, the present invention provides a server, comprising a peripheral device interconnection PCI bus, wherein the server establishes a connection with the smart network card as described in the above embodiment through the PCI bus.

[0036] In a fourth aspect, the present invention provides a server system, comprising the server as described in the aforementioned embodiment and the smart network card as described in the aforementioned embodiment.

[0037] Compared with the prior art, the IO request processing method, smart network card, server and server system provided in this embodiment pre-set performance levels for block devices and set queues with different configurations for different performance levels. One queue is bound to a processing core of a CPU for processing IO requests in the queue. As a result, IO requests sent by block devices of different performance levels are sent to corresponding configured queues and processed by processing cores of different CPUs, thereby avoiding mutual influence of IO requests sent by block devices of different performance levels, thereby affecting the efficiency of the smart network card in processing IO requests. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 This is an example diagram of an application scenario provided by this embodiment.

[0040] Figure 2 This is a block diagram of an example of a smart network card provided in this embodiment.

[0041] Figure 3 This is an example flowchart of the IO request processing method provided in this embodiment.

[0042] Figure 4 This is an example diagram of the IO request processing process provided in this embodiment.

[0043] Icons: 10-Smart NIC; 11-CPU; 12-Hardware logic firmware; 13-PCI bus; 20-Server; 30-Storage device. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0045] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0047] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, they are based on the orientation or position relationship shown in the accompanying drawings, or are the orientation or position relationship in which the product of the invention is usually placed when used. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0048] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0049] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0050] Please refer to Figure 1 , Figure 1 This is an example diagram of an application scenario provided in this embodiment. Figure 1 In the embodiment, the smart network card 10 is connected to the server 20 through a PCI (Peripheral Component Interconnect) bus. One communication connection method can be that the smart network card 10 is inserted into the server 20, and the smart network card 10 communicates with various types of remote storage devices 30 through the network.

[0051] The smart network card 10 is a network card with an independent computing unit that can complete specific infrastructure functional operations. It can not only realize the connection function of the network card, but also usually offload some CPU processing tasks, such as network acceleration, security acceleration, storage offload, storage acceleration, cloud platform management and other tasks. In the storage offload scenario, the smart network card 10 simulates the storage block device through the PCI (Peripheral Component Interconnect Express, a high-speed serial computer bus standard) bus, implements the back end of the virtual block device in the smart network card 10, and converts the communication protocol and storage protocol through software, communicates with various types of remote storage devices 30, and provides cloud storage access functions to the server 20. The storage offload function of the smart network card 10 enables the server 20 to use the storage resources on the remote storage device 30 without perceiving the specific type of the remote storage device 30 and the physical type of the storage components of the storage device 30.

[0052] The server 20 may be an independent computer device or a physical server. The server 20 is also called a HOST or a bare metal server.

[0053] The smart network card 10 and the server 20 can also form a server system to provide storage services to the outside world. Users can interact with the server 20 through a browser, and the server 20 sends the user's IO request to the storage device 30 through the smart network card.

[0054] The storage device 30 may be an IPSAN (IP Storage Area Network), NVME, CEPH or other storage device.

[0055] based on Figure 1 This embodiment also provides a block diagram of the smart network card 10, please refer to Figure 2 , Figure 2 This is a block diagram of an example of a smart network card provided in this embodiment. The smart network card 10 includes a CPU 11 and a hardware logic firmware 12. In some scenarios, the smart network card 10 also includes a memory for storing a computer program executed by the CPU 11.

[0056] The hardware logic firmware 12 is connected to the server 20 through a PCI bus, the CPU 11 and the hardware logic firmware 12 can be electrically connected or connected through a PCI bus, and the CPU 11 is connected to the storage device 30 through a network.

[0057] The hardware logic firmware 12 may be a FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), a DPU (Data Processing Unit), or the like.

[0058] In this embodiment, for the IO request sent by the server, the hardware logic firmware 12 receives the IO request sent by the server 20 through the PCI bus, encapsulates the IO request into an IO message, and sends the IO message to the queue, the CPU 11 takes out the IO message from the queue, parses out the IO request therein, and sends the IO request to the storage device 30; for the response request of the storage device 30 in response to the IO request, the CPU 11 receives the response request, encapsulates the response request into a response message and then sends it to the queue, the hardware logic firmware 12 takes out the response message from the queue, parses out the response request therein, and sends the IO request to the server 20 through the PCI bus.

[0059] In this embodiment, in addition to processing requests, CPU11 is also responsible for software-related configuration. CPU11 pre-creates a sub-block device, which is an operation handle established on the CPU side for the block device of the back-end storage device. A back-end block device can obtain several operation handles on the CPU side. The sub-block device establishes an access connection with the hard disk of the storage device 30 through the network. For the low-speed hard disk of the storage device 30, one sub-block device can correspond to a low-speed hard disk. For the high-speed hard disk of the storage device 30, multiple sub-block devices can correspond to a high-speed hard disk. CPU11 can include multiple processing cores, and one sub-block device is bound to one processing core. CPU11 is responsible for the configuration of binding the sub-block device bdev to the processing core of the CPU. In addition, CPU11 is also responsible for the configuration of binding the PCI address of the sub-block device bdev and the block device blk loaded by the server 20. Multiple sub-block devices bdev can be bound to the PCI address of the same block device blk, and one sub-block device bdev can also be bound to the PCI address of a block device blk.

[0060] In this embodiment, in addition to processing requests, the hardware logic firmware 12 is also responsible for hardware-related configuration, configuring performance levels, the number of queues corresponding to different performance levels, the mapping relationship between different performance levels and corresponding queues, queue depth, and data distribution strategy.

[0061] The performance level may include three levels: high, medium, and low, or two levels: high and low, or only high, or only low. The configurations of queues corresponding to different performance levels are also different. For the low performance level, one queue is configured for it, for the high performance level, multiple queues are configured for it, and for the medium performance level, multiple queues are configured for it, and the number of queues for the medium performance level is less than the number of queues for the high performance level. For example, the high performance level is configured with 3 queues, the medium performance level is configured with 2 queues, and the low performance level is configured with 1 queue. To configure the number of queues for each performance level, it is also necessary to configure the number of queues between each performance level and the queues. A mapping relationship is established to ensure that IO requests with different performance level requirements use different queue configurations to meet the needs of the corresponding performance level. As an implementation method, for example, high performance level corresponds to queues 1, 2, and 3, medium performance level corresponds to queues 4 and 5, and low performance level corresponds to queue 6. As another implementation method, for example, high performance level corresponds to queues 1, 2, and 3, medium performance level corresponds to queues 2 and 3, and low performance level corresponds to queue 4. Low performance is assigned to a separate queue to prevent low-performance IO requests from affecting high-performance IO requests. By controlling the number of queues, different performance levels can be refined to make the control of performance levels more flexible. In addition, for situations where a performance level corresponds to multiple queues, a data distribution strategy can be configured for it so that multiple IO requests can be distributed in parallel to multiple queues to achieve concurrent processing of IO requests and improve processing efficiency.

[0062] In addition to configuring the number of queues for different performance levels, different queue depths can also be configured for different performance levels. For high performance levels, since the high-speed hard disk of the storage device is processing IO requests, its queue depth is configured to be larger to prevent the smart network card 10 from becoming a bottleneck and affecting the processing efficiency of IO requests. For low performance levels, since the low-speed hard disk of the storage device is processing IO requests, its queue depth is configured to be smaller to avoid wasting the resources of the smart network card 10.

[0063] based on Figure 1 and Figure 2 This embodiment also provides an application in Figure 1 and Figure 2 For the IO request processing method of smart network card 10, please refer to Figure 3 , Figure 3 The flowchart of the IO request processing method provided in this embodiment includes the following steps:

[0064] Step S101 : the hardware logic firmware receives an IO request sent by a server through a block device according to a preset PCI address, where the PCI address is an address used by the block device for addressing using a PCI bus.

[0065] In this embodiment, the server can load multiple block devices, each of which corresponds to a PCI address, so that the PCI bus can be uniformly addressed. Each block device sends an IO request to the PCI bus according to its own pre-set PCI address.

[0066] Step S102, the hardware logic firmware sends the IO request to the queue corresponding to the performance level preset for the PCI address according to the performance level and mapping relationship preset for the PCI address. The mapping relationship represents the correspondence between the performance level and the queue. Different performance levels correspond to queues with different configurations. One queue is pre-bound to a CPU processing core for processing the IO requests in the queue.

[0067] In this embodiment, the performance level can be set in advance for the PCI address according to the performance requirements, and the configurations of queues corresponding to different performance levels are different. The different configurations may mean that the number of queues corresponding to different performance levels is different, the depth of queues corresponding to different performance levels is different, or both the number of queues and the queue depth corresponding to different performance levels are different.

[0068] As an implementation, the performance level includes a first level and a second level, the performance of the first level is greater than the performance of the second level, the number of queues corresponding to the first level is greater than the number of queues corresponding to the second level and / or the depth of the queues corresponding to the first level is greater than the depth of the queues corresponding to the second level. It should be noted that the first level and the second level are relative, for example, the performance level is divided into three levels: high, medium and low. For the high and medium levels, the high level is the first level and the medium level is the second level. For the medium and low levels, the medium level is the first level and the low level is the second level.

[0069] In this embodiment, a queue is pre-bound to a CPU processing core, and each processing core is only used to process IO requests in the queue bound to it. Different performance levels correspond to queues with different configurations, thereby achieving control of IO requests with different performance level requirements and ensuring that IO requests of different performance levels do not affect each other.

[0070] Step S103: The CPU sends the IO requests in the queue corresponding to the performance level preset for the PCI address to the storage device.

[0071] In this embodiment, the hardware logic firmware sends the IO request to the corresponding queue, the CPU takes the IO request from the queue, and then sends the IO request to the storage device, thereby realizing the sending of the IO request between the server and the storage device.

[0072] The above method provided in this embodiment pre-sets the performance level for the block device, and sets queues with different configurations for different performance levels. One queue is bound to a processing core of a CPU. As a result, IO requests sent by block devices of different performance levels are sent to corresponding configuration queues and processed by processing cores of different CPUs, thereby avoiding that IO requests sent by block devices of different performance levels affect each other and affect the efficiency of the smart network card in processing IO requests.

[0073] In an optional implementation, the hardware logic firmware sends the IO request to the queue corresponding to the performance level preset for the PCI address according to the performance level and mapping relationship preset for the PCI address. One implementation method is:

[0074] First, the hardware logic firmware encapsulates the PCI address and IO request into an IO message;

[0075] Secondly, the hardware logic firmware sends the IO message to the queue corresponding to the performance level of the PCI address.

[0076] In this embodiment, the PCI address is encapsulated in the message header of the IO message, and the IO request serves as the message content of the IO message. The IO request can be one or more, and the queue corresponding to the performance level of the PCI address can be one or more. If there is only one queue corresponding to the performance level of the PCI address, no matter whether there are one or more IO requests, each IO request is encapsulated into an IO message and sent to this queue. If there are multiple queues corresponding to the performance level of the PCI address, the queue to which the IO request should be sent can be determined according to the pre-set data distribution strategy. After each IO request is encapsulated into an IO message, it is sent to the determined queue according to the data distribution strategy. This embodiment provides a processing method for multiple IO requests and multiple queues corresponding to the performance level of the PCI address:

[0077] The hardware logic firmware sends multiple IO messages in parallel to multiple queues corresponding to the performance levels of the PCI addresses.

[0078] In this embodiment, the data distribution strategy may be, but is not limited to, a HASH method, a round-robin method, etc. For example, multiple queues are numbered in advance, and the HASH method can perform a HASH calculation on the PCI address to determine the number of the queue that the IO request enters, and then send the IO request to the queue with the corresponding number.

[0079] In an optional implementation, the CPU pre-creates a sub-block device, a binding relationship exists between the sub-block device and the PCI address, the CPU communicates with the storage device through the sub-block device, the performance level corresponds to the queue, one queue is bound to a CPU processing core, and the CPU processing core bound to the queue corresponding to the performance level of the PCI address is used to process the IO messages in the queue. One implementation method is:

[0080] First, the CPU uses the processing core of the CPU bound to the queue corresponding to the performance level of the PCI address to take out the IO message from the queue corresponding to the performance level of the PCI address and parse the PCI address and IO request from the IO message.

[0081] Secondly, the CPU uses the CPU processing core bound to the queue corresponding to the performance level of the PCI address to determine the sub-block device bound to the parsed PCI address according to the parsed IO request, the PCI address and the binding relationship between the sub-block device and the PCI address;

[0082] Finally, the CPU uses the processing core of the CPU bound to the queue corresponding to the performance level of the PCI address to send the parsed IO request to the storage device through the sub-block device bound to the parsed PCI address.

[0083] In this embodiment, after receiving an IO request, the storage device 30 processes the data that the IO request needs to access, and also needs to return a response request for the IO request. The response request includes, but is not limited to, the result of the IO request processing. For example, for a write IO request, the response request includes whether the write is successful. If the write fails, the response request also includes an error code for the write failure. For a read IO request, the response request includes whether the read is successful. If the read is successful, the response request also includes the read data. If the read fails, the response request also includes an error code for the read failure. In order to correctly return the response request to the server, this embodiment also provides an implementation method:

[0084] First, the CPU uses the pre-created sub-block device to receive a response request from the storage device in response to the IO request, and the sub-block device receiving the response request is the same as the sub-block device sending the IO request;

[0085] In this embodiment, the storage device returns a response request of the IO request to the sub-block device from which the storage device receives the IO request.

[0086] Secondly, the CPU uses the processing core of the CPU bound to the sub-block device that responds to the request to send the response request to the queue bound to the processing core of the CPU bound to the sub-block device that responds to the request;

[0087] In an optional implementation, the sub-block device is bound to the processing core of the CPU, and the response request received by the sub-block device is processed by the processing core of the CPU to which the sub-block device is bound. The sub-block device is bound to a pre-set PCI address of the block device. In order to correctly return the response request to the server, it is also necessary to encapsulate the PCI address bound to the sub-block device and the response request into a response message and send it to a queue. An implementation method for sending the response request to the queue is: the CPU uses the processing core of the CPU bound to the sub-block device that responds to the request to encapsulate the PCI address of the block device bound to the sub-block device that responds to the request and the response request into a response message; the CPU uses the processing core of the CPU bound to the sub-block device that responds to the request to send the response message to the queue bound to the processing core of the CPU bound to the sub-block device that responds to the request.

[0088] Finally, the hardware logic firmware takes out the response request from the queue bound to the processing core of the CPU bound to the sub-block device that responds to the request, and sends it to the block device through the PCI bus.

[0089] In this embodiment, after the response message is sent to the queue, the hardware logic firmware takes the response message from the queue, processes it, and then sends it to the block device through the PCI bus. One implementation method is:

[0090] First, the hardware logic firmware takes out the response message from the queue bound to the processing core of the CPU bound to the sub-block device that responds to the request and parses the PCI address and the response request from the response message;

[0091] Secondly, the hardware logic firmware sends the parsed response request to the block device through the PCI bus according to the parsed PCI address.

[0092] In this embodiment, one implementation method is: the hardware logic firmware sends a response request to the PCI bus according to the parsed PCI address, and the block device set to the parsed PCI address obtains the response request from the PCI bus. At this point, the response request reaches the block device and will eventually be returned to the server.

[0093] In order to more clearly illustrate the IO request processing method provided by the embodiment of the present invention, this embodiment provides a specific example to illustrate the above processing process. Figure 4 , Figure 4 This is an example diagram of the IO request processing process provided in this embodiment. Figure 4In the example, the server loads three block devices: blk1, blk2, and blk3. The PCI address corresponding to blk1 is blk1_pf1, the PCI address corresponding to blk2 is blk2_pf2, and the PCI address corresponding to blk3 is blk1_pf3. blk1 and blk2 correspond to the backend high-speed storage hard disk, and blk3 corresponds to the low-speed storage hard disk. The smart network card has four hardware queues q1 to q4. The CPU of the smart network card includes four processing cores c1 to c4, which process the data in the four hardware queues respectively. The performance level M includes three levels: high MH, medium MM, and low ML. The high MH=3 occupies three queues; the medium MM=2 occupies two queues; the low ML=1 occupies one queue. blk1 is MH level, and blk1 is configured to be mapped to queues q1, q2, and q3; blk2 is MM level, mapped to queues q1 and q2; blk3 is ML level, mapped to queue q4. Since blk1 and blk2 are mapped to multiple queues, their data distribution strategy is set to round-robin.

[0094] For blk1, configure three sub-devices bdev11, bdev12, and bdev13 for it, bind bdev11 to c1, bdev12 to c2, and bdev13 to c3; bind bdev11, bdev12, and bdev13 to the PCI address bdev1_pf1 of blk1; for blk2, configure two sub-devices bdev21 and bdev22, bind bdev21 to c1, and bdev22 to c2; bind bdev21 and bdev22 to the PCI address bdev2_pf2 of blk2; for blk3, configure one sub-device bdev34, bind bdev34 to c4; bind bdev34 to the PCI address bdev3_pf3 of blk3.

[0095] For the IO request sent from the server to the storage device, the hardware logic firmware receives the IO request of blk1, searches for the corresponding queue according to the value of bdf1_pf1, and encapsulates the IO request into an IO message according to the set data distribution policy and distributes it to the q1, q2, and q3 queues in sequence; the hardware logic firmware receives the IO request of blk2, searches for the corresponding queue according to the value of bdf2_pf2, and encapsulates the IO request into an IO message according to the set data distribution policy and distributes it to the q1 and q2 queues in sequence; the hardware logic firmware receives the IO request of blk3, searches for the corresponding queue according to the value of bdf3_pf3, and encapsulates the IO request into an IO message and distributes it to the q4 queue.

[0096] The SmartNIC shares four CPU cores c1 to c4 to process queue data, binds the thread of c1 to q1, the thread of c2 to q2, the thread of c3 to q3, and the thread of c4 to q4. Each CPU core reads and processes data from the bound queue in a polling manner.

[0097] c1 takes out an IO message from q1 and parses out the PCI address and IO request. First, it finds out whether it is bdev11 or bdev21 according to the PCI address value in the IO message. If it is bdev11, it forwards the parsed IO request to the back-end blk1 high-speed disk and waits for the response request asynchronously. If bdev11 receives a response request, it wakes up c1, and c1 encapsulates the response request and the PCI address bound to bdev11 into a response message, and then sends the processed response message to the queue bound to c1. The hardware logic firmware takes out the response message from the queue bound to c1, parses out the PCI address and response request, and returns the response request to blk1 of the host according to the PCI address.

[0098] The processing methods of the other processing cores c2 to c4 are similar and will not be described again here.

[0099] From the above processing flow, we can see that: the IO request of blk1 will be sent to three queues q1, q2, and q3 concurrently, processed by c1, c2, and c3 of the smart network card concurrently, and then sent to the high-speed hard disk 1 at the back end concurrently; while the IO request of blk2 will be sent to two queues q1 and q2 concurrently, processed by c1 and c2 of the smart network card concurrently, and then sent to the high-speed hard disk 2 at the back end concurrently; the IO request of blk3 will be sent to one queue q4, processed by c4 of the smart network card, and then sent to the low-speed hard disk 3 at the back end; the high-speed queue and the low-speed queue in the smart network card do not affect each other. And at any time, no matter how many IO requests of blk there are, they will be processed by the smart network card at a high load, and there will be no waste of resources on the smart network card, which can achieve the best performance of a single blk and the highest performance of the whole machine.

[0100] This embodiment also provides a working principle of a smart network card 10, the smart network card 10 includes a hardware logic firmware and a CPU, the hardware logic firmware is connected to the server through a PCI bus, the server is loaded with a block device, and the CPU is connected to the storage device through a network;

[0101] The hardware logic firmware is used to receive an IO request sent by the server through the block device according to a preset PCI address, where the PCI address is an address used by the block device for addressing using a PCI bus;

[0102] The hardware logic firmware is further used to send the IO request to a queue corresponding to the performance level preset for the PCI address according to the performance level and mapping relationship preset for the PCI address, wherein the mapping relationship represents the corresponding relationship between the performance level and the queue, and different performance levels correspond to queues with different configurations, and one queue is pre-bound to a CPU processing core for processing the IO request in the queue;

[0103] The CPU is used to send the IO request in the queue corresponding to the performance level preset at the PCI address to the storage device.

[0104] In an optional implementation manner, the hardware logic firmware, when used to send the IO request to a queue corresponding to the performance level preset for the PCI address according to the performance level and mapping relationship preset for the PCI address, is further used to:

[0105] The hardware logic firmware is further used to encapsulate the PCI address and the IO request into an IO message;

[0106] The hardware logic firmware is also used to send the IO message to a queue corresponding to the performance level of the PCI address.

[0107] In an optional implementation, there are multiple IO requests, each of which is encapsulated into an IO message, a message header of each of which carries the PCI address, and there are multiple queues corresponding to the performance level of the PCI address. When the hardware logic firmware is used to send the IO message to the queue corresponding to the performance level of the PCI address, it is also used to:

[0108] The hardware logic firmware is also used for the hardware logic firmware to send multiple IO messages in parallel to multiple queues corresponding to the performance level of the PCI address.

[0109] In an optional implementation manner, the CPU pre-creates a sub-block device, the sub-block device communicates with the storage device, and the CPU is further configured to:

[0110] The CPU is further configured to use a processing core of the CPU bound to a queue corresponding to the performance level of the PCI address to take out the IO message from the queue corresponding to the performance level of the PCI address and parse the PCI address and the IO request from the IO message;

[0111] The CPU is further configured to determine the sub-block device bound to the parsed PCI address according to the parsed IO request, the PCI address, and the binding relationship between the sub-block device and the PCI address by using the processing core of the CPU bound to the queue corresponding to the performance level of the PCI address;

[0112] The CPU is further configured to utilize a CPU processing core bound to a queue corresponding to a performance level of the PCI address to send the parsed IO request to the storage device via a sub-block device bound to the parsed PCI address.

[0113] In an optional embodiment, the performance level includes a first level and a second level, the performance of the first level is greater than the performance of the second level, the number of queues corresponding to the first level is greater than the number of queues corresponding to the second level and / or the depth of the queues corresponding to the first level is greater than the depth of the queues corresponding to the second level.

[0114] In an optional implementation manner, the CPU pre-creates a sub-block device, and a binding relationship exists between the sub-block device and the processing core of the CPU:

[0115] The CPU is further configured to receive a response request from the storage device in response to the IO request by using a pre-created sub-block device, wherein the sub-block device receiving the response request is the same as the sub-block device sending the IO request;

[0116] The CPU is further configured to send the response request to a queue bound to the processing core of the CPU bound to the sub-block device that responds to the request, using the processing core of the CPU bound to the sub-block device that responds to the request;

[0117] The hardware logic firmware is further used to take out the response request from the queue bound to the processing core of the CPU bound to the sub-block device of the response request, and send it to the block device through the PCI bus.

[0118] In an optional implementation manner, the sub-block device is bound to a preset PCI address of the block device, and the CPU, when used to send the response request to a queue bound to the processing core of the CPU bound to the sub-block device that responds to the request using the processing core of the CPU bound to the sub-block device that responds to the request, is further used to:

[0119] The CPU is further configured to encapsulate the PCI address of the block device bound to the sub-block device of the response request and the response request into a response message by using the processing core of the CPU bound to the sub-block device of the response request;

[0120] The CPU is further configured to send the response message to a queue bound to the processing core of the CPU bound to the sub-block device that responds to the request, using the processing core of the CPU bound to the sub-block device that responds to the request;

[0121] When the hardware logic firmware is used to take out the response request from the queue bound to the processing core of the CPU bound to the sub-block device of the response request and send it to the block device through the PCI bus, it is also used to:

[0122] The hardware logic firmware is further used to take out the response message from the queue bound to the processing core of the CPU bound to the sub-block device of the response request and parse out the PCI address and the response request from the response message;

[0123] The hardware logic firmware is also used to send the parsed response request to the block device through the PCI bus according to the parsed PCI address.

[0124] An embodiment of the present invention provides a server, the server includes a peripheral device interconnection PCI bus, and the server establishes a connection with the smart network card as described in the above-mentioned implementation method through the PCI bus.

[0125] An embodiment of the present invention provides a server system, which includes the server described in the above embodiment and the smart network card described in the above embodiment.

[0126] In summary, an embodiment of the present invention provides an IO request processing method, an intelligent network card, a server and a server system, wherein the intelligent network card includes hardware logic firmware and a central processing unit CPU, wherein the hardware logic firmware is communicatively connected to the server through a peripheral device interconnection PCI bus, the server is loaded with a block device, and the CPU is communicatively connected to a storage device through a network, and the method includes: the hardware logic firmware receives an IO request sent by the server through the block device according to a preset PCI address, wherein the PCI address is an address used by the block device for addressing using the PCI bus; the hardware logic firmware sends the IO request to a queue corresponding to the performance level preset by the PCI address according to the performance level and mapping relationship preset by the PCI address, wherein the mapping relationship represents the correspondence between the performance level and the queue, the block devices of the PCI addresses corresponding to different performance levels correspond to storage hard disks with different performances, and a queue is pre-bound to a processing core of a CPU for processing IO requests in the queue; the CPU sends the IO request in the queue corresponding to the performance level preset by the PCI address to the storage device. Compared with the prior art, this embodiment has at least the following advantages: (1) The IO of a high-performance block device will be distributed to multiple high-performance queues and processed by multiple CPU cores. After processing, it will be sent to the back-end storage device concurrently, thereby achieving the purpose of high-performance access to the back-end high-performance disk; while the IO of a low-performance block device will only be distributed to a low-performance queue and processed by only one CPU core, and will not compete with the high-performance device for hardware queue resources, software queue resources, and CPU resources, so it will not block the access of the high-performance device; (2) Through such diversion and load sharing, high-performance IO can occupy more resources and have higher performance, while low-performance IO will not occupy excess resources and will not block the access of other high-performance IO, thereby achieving the effect of reasonable use of resources, making the overall performance of the smart network card higher and applicable to more storage access scenarios.

[0127] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for processing IO requests, characterized in that: Applied to a smart network card, the smart network card includes a hardware logic firmware and a central processing unit CPU, the hardware logic firmware is connected to a server through a peripheral device interconnection PCI bus, the server is loaded with a block device, and the CPU is connected to a storage device through a network. The method includes: The hardware logic firmware receives the IO request sent by the server through the block device according to a preset PCI address, where the PCI address is an address used by the block device for addressing using a PCI bus; The hardware logic firmware sends the IO request to a queue corresponding to the performance level preset for the PCI address according to the performance level and mapping relationship preset for the PCI address, wherein the mapping relationship represents the corresponding relationship between the performance level and the queue, and different performance levels correspond to queues with different configurations, and one queue is pre-bound to a CPU processing core for processing the IO request in the queue; The CPU sends the IO request in the queue corresponding to the performance level preset at the PCI address to the storage device.

2. The IO request processing method according to claim 1, characterized in that: The step of sending the IO request to a queue corresponding to the performance level preset at the PCI address according to the performance level and mapping relationship preset at the PCI address by the hardware logic firmware comprises: The hardware logic firmware encapsulates the PCI address and the IO request into an IO message; The hardware logic firmware sends the IO message to a queue corresponding to a performance level of the PCI address.

3. The IO request processing method according to claim 2, characterized in that: There are multiple IO requests, each of which is encapsulated into an IO message, a message header of each IO message carries the PCI address, there are multiple queues corresponding to the performance level of the PCI address, and the step of the hardware logic firmware sending the IO message to the queue corresponding to the performance level of the PCI address includes: The hardware logic firmware sends multiple IO messages in parallel to multiple queues corresponding to the performance levels of the PCI addresses.

4. The IO request processing method according to claim 2, characterized in that: The CPU pre-creates a sub-block device, the sub-block device communicates with the storage device, and the step of the CPU sending the IO request in the queue corresponding to the performance level preset by the PCI address to the storage device includes: The CPU uses a processing core of the CPU bound to a queue corresponding to the performance level of the PCI address to take out the IO message from the queue corresponding to the performance level of the PCI address and parse the PCI address and the IO request from the IO message; The CPU determines the sub-block device bound to the parsed PCI address according to the parsed IO request, the PCI address, and the binding relationship between the sub-block device and the PCI address by using the CPU processing core bound to the queue corresponding to the performance level of the PCI address; The CPU uses a processing core of the CPU bound to a queue corresponding to a performance level of the PCI address to send the parsed IO request to the storage device through a sub-block device bound to the parsed PCI address.

5. The IO request processing method according to claim 1, characterized in that: The performance levels include a first level and a second level, the performance of the first level is greater than the performance of the second level, the number of queues corresponding to the first level is greater than the number of queues corresponding to the second level and / or the depth of the queues corresponding to the first level is greater than the depth of the queues corresponding to the second level.

6. The IO request processing method according to claim 1, characterized in that: The CPU has a sub-block device created in advance, and there is a binding relationship between the sub-block device and the processing core of the CPU. The method further includes: The CPU receives a response request from the storage device in response to the IO request by using a pre-created sub-block device, wherein the sub-block device receiving the response request is the same as the sub-block device sending the IO request; The CPU uses the processing core of the CPU bound to the sub-block device that responds to the request to send the response request to the queue bound to the processing core of the CPU bound to the sub-block device that responds to the request; The hardware logic firmware takes out the response request from the queue bound to the processing core of the CPU bound to the sub-block device of the response request, and sends it to the block device through the PCI bus.

7. The IO request processing method according to claim 6, characterized in that: The sub-block device is bound to a preset PCI address of the block device, and the CPU uses the processing core of the CPU bound to the sub-block device that responds to the request to send the response request to a queue bound to the processing core of the CPU bound to the sub-block device that responds to the request, including: The CPU uses a processing core of the CPU bound to the sub-block device that responds to the request to encapsulate the PCI address of the block device bound to the sub-block device that responds to the request and the response request into a response message; The CPU uses the processing core of the CPU bound to the sub-block device that responds to the request to send the response message to the queue bound to the processing core of the CPU bound to the sub-block device that responds to the request; The step of the hardware logic firmware taking out the response request from the queue bound to the processing core of the CPU bound to the sub-block device of the response request, and sending it to the block device through the PCI bus includes: The hardware logic firmware takes out the response message from the queue bound to the processing core of the CPU bound to the sub-block device that responds to the request, and parses the PCI address and the response request from the response message; The hardware logic firmware sends the parsed response request to the block device through the PCI bus according to the parsed PCI address.

8. A smart network card, characterized in that: The smart network card includes a hardware logic firmware and a central processing unit CPU, the hardware logic firmware is connected to the server through a peripheral device interconnection PCI bus, the server is loaded with a block device, and the CPU is connected to the storage device through a network; The hardware logic firmware is used to receive an IO request sent by the server through the block device according to a preset PCI address, where the PCI address is an address used by the block device for addressing using a PCI bus; The hardware logic firmware is further used to send the IO request to a queue corresponding to the performance level preset for the PCI address according to the performance level and mapping relationship preset for the PCI address, wherein the mapping relationship represents the corresponding relationship between the performance level and the queue, and different performance levels correspond to queues with different configurations, and one queue is pre-bound to a CPU processing core for processing the IO request in the queue; The CPU is used to send the IO request in the queue corresponding to the performance level preset at the PCI address to the storage device.

9. A server, characterized in that: The server comprises a peripheral component interconnection PCI bus, and the server establishes a connection with the intelligent network card as claimed in claim 8 through the PCI bus.

10. A server system, characterized in that: The server system comprises the server according to claim 9 and the smart network card according to claim 8.

Images