A method, device, system, and storage medium for I / O offloading in a cloud environment

Abstract

The embodiments of the present application provide an I / O offloading method, device, system and storage medium in a cloud environment. In the embodiments of the present application, a queue component is added to the CPU of the host, and it is innovatively proposed to use the queue component in the I / O processing process. Based on this, a virtual I / O device and a device queue corresponding to the virtual I / O device can be provided for the instances on the host through the queue component; within the CPU, the device queue is used to schedule the I / O requests that occur between the instance and the virtual I / O device; and the original I / O offloading card only needs to be responsible for listening to the I / O requests in the device queue and transferring the data related to the I / O requests. In this way, the device queue in the I / O processing process can be moved up to the CPU, so that in the I / O scheduling process, it can联动 with each component in the CPU, obtain the real-time loads of each core in the CPU, and thus more reasonably implement I / O scheduling instead of blindly scheduling, which can effectively improve the I / O processing efficiency. It should be noted that the "联动" in the original text seems to be a misspelling. It might be "联动" which is not a standard English word. I've translated it as "联动" for now. If it has a specific correct English expression, please let me know and I can adjust it.

Description

Technical Field

[0001] This application relates to the field of cloud computing technology, and in particular to an I / O offloading method, device, system and storage medium in a cloud environment. Background Technology

[0002] As CPU performance increases, marginal costs rise sharply, leading to the development of I / O offloading cards. I / O offloading cards, also known as DPUs or IPUs, provide an I / O computing engine for high-bandwidth, low-latency, and data-intensive computing scenarios.

[0003] Currently, I / O processing is completely offloaded to the I / O offload card. The I / O processing on the I / O offload card is separated from the CPU data link of the host machine. This separation structure often leads to problems such as data overflow during the current I / O processing, resulting in poor processing efficiency. Summary of the Invention

[0004] This application provides a method, device, system, and storage medium for I / O offloading in a cloud environment to improve I / O processing efficiency in a cloud environment.

[0005] This application provides an I / O offloading system in a cloud environment, including: a CPU of a host machine and an I / O offloading card plugged into the host machine, wherein the CPU is equipped with a queue component;

[0006] The queue component is used to provide virtual I / O devices and device queues corresponding to the virtual I / O devices for instances on the host machine; within the CPU, the device queues are used to schedule I / O requests occurring between the instances and the virtual I / O devices;

[0007] The I / O offloading card is used to listen for I / O requests in the device queue and to transmit data related to the listened I / O requests between the instance and the physical I / O device corresponding to the virtual I / O device.

[0008] This application embodiment also provides an I / O offloading method in a cloud environment, applicable to a CPU in a host machine, wherein the CPU is equipped with a queue component, and the method includes:

[0009] The queue component is used to provide virtual I / O devices for instances on the host machine;

[0010] Configure the corresponding device queue for the virtual I / O device using the queue component;

[0011] Within the CPU, the device queue is used to schedule I / O requests between the instance and the virtual I / O device, so that the I / O offloading card plugged into the host machine can listen to the I / O requests in the device queue and transmit the data related to the listened I / O requests between the instance and the physical I / O device corresponding to the virtual I / O device.

[0012] This application embodiment also provides an I / O offloading method in a cloud environment, applicable to I / O offloading cards plugged into a host machine, wherein the CPU of the host machine is equipped with a queue component, and the method includes:

[0013] Listen for I / O requests between the instance on the host machine and the virtual I / O device provided by the queue component for the instance from the device queue in the queue component;

[0014] Retrieve data related to the monitored I / O requests;

[0015] Data related to the detected I / O requests is transmitted between the instance and the physical I / O device corresponding to the virtual I / O device.

[0016] This application embodiment also provides a processor CPU installed in a host machine. The CPU is equipped with a queue component and is used to execute the one or more computer instructions for:

[0017] The queue component is used to provide virtual I / O devices for instances on the host machine;

[0018] Configure the corresponding device queue for the virtual I / O device using the queue component;

[0019] Within the CPU, the device queue is used to schedule I / O requests between the instance and the virtual I / O device, so that the I / O offloading card plugged into the host machine can listen to the I / O requests in the device queue and transmit the data related to the listened I / O requests between the instance and the physical I / O device corresponding to the virtual I / O device.

[0020] This application embodiment also provides an I / O offloading card, which is plugged into a host machine. The CPU of the host machine is equipped with a queue component, and the I / O offloading card includes a memory and a processor.

[0021] The memory is used to store one or more computer instructions;

[0022] The processor is coupled to the memory and is used to execute the one or more computer instructions for:

[0023] Listen for I / O requests that occur between an instance on the host and the virtual I / O devices provided by the queue component for the instance from the device queues in the queue component;

[0024] Obtain data related to the listened I / O requests;

[0025] Transfer data related to the listened I / O requests between the instance and the physical I / O device corresponding to the virtual I / O device.

[0026] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to execute the foregoing I / O offloading method in a cloud environment.

[0027] In an embodiment of the present application, a queue component is added to the CPU of the host, and it is innovatively proposed to use the queue component in the process of I / O processing. Based on this, the queue component can provide virtual I / O devices and corresponding device queues for instances on the host; within the CPU, the device queues are used to schedule I / O requests that occur between the instance and the virtual I / O device; and the original I / O offloading card only needs to be responsible for listening to the I / O requests in the device queue and transferring the data related to the I / O requests. In this way, the device queue in the I / O processing process can be moved up to the CPU, so that in the I / O scheduling process, it can联动 with various components within the CPU, obtain the real-time loads of each core within the CPU, and thus can more reasonably implement I / O scheduling instead of blindly scheduling, which can effectively improve the I / O processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings described herein are used to provide a further understanding of the present application, form a part of the present application, and the schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0029] Figure 1 It is a schematic structural diagram of an I / O offloading system in a cloud environment provided by an exemplary embodiment of the present application;

[0030] Figure 2 It is a schematic structural diagram corresponding to an optional implementation solution of an I / O offloading system provided by an exemplary embodiment of the present application;

[0031] Figure 3 It is a schematic structural diagram of an optional implementation solution of an I / O offloading system provided by an exemplary embodiment of the present application;

[0032] Figure 4 It is a schematic logical diagram of an instance creation process provided by an exemplary embodiment of the present application;

[0033] Figure 5 A logic schematic diagram of an instance destruction process provided by an exemplary embodiment of the present application;

[0034] Figure 6 A flowchart of an I / O offloading method in a cloud environment provided by another exemplary embodiment of the present application;

[0035] Figure 7 A flowchart of another I / O offloading method in a cloud environment provided by another exemplary embodiment of the present application;

[0036] Figure 8 A schematic diagram of the structure of a processor CPU provided by another exemplary embodiment of the present application;

[0037] Figure 9 A schematic diagram of the structure of an I / O offloading card provided by an exemplary embodiment of the present application. Detailed implementation manners

[0038] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Apparently, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0039] Currently, the processing work of I / O is completely offloaded to the I / O offloading card, and the processing efficiency of I / O is not good. Therefore, in some embodiments of the present application: a queue component is added to the CPU of the host, and it is innovatively proposed to use the queue component in the processing process of I / O offloading. Based on this, a virtual I / O device and a device queue corresponding to the virtual I / O device can be provided for the instances on the host through the queue component; within the CPU, the device queue is used to schedule the I / O requests occurring between the instance and the virtual I / O device; and the original I / O offloading card only needs to be responsible for listening to the I / O requests in the device queue and transmitting the data related to the I / O requests. In this way, the device queue in the I / O offloading processing process can be moved up to the CPU, so that it can be linked with various components within the CPU during the I / O scheduling process, obtain the real-time load of each core within the CPU, and thus can more reasonably implement I / O scheduling instead of blindly scheduling, which can effectively improve the processing efficiency of I / O.

[0040] The following will detail the technical solutions provided by each embodiment of the present application in conjunction with the drawings.

[0041] Figure 1This is a schematic diagram of the structure of an I / O offloading system in a cloud environment, provided as an exemplary embodiment of this application. Figure 1 As shown, the system includes: the host machine's CPU and an I / O offloading card plugged into the host machine.

[0042] The I / O offloading card can be implemented using chips such as DPU and IPU; however, this embodiment is not limited to these. The host machine can be a physical machine in a cloud environment, see reference [link / reference]. Figure 1 In a cloud environment, multiple instances can run on a single host machine. The core technology of cloud-native computing is virtualization. Based on virtualization, several virtual CPUs can be created on top of the host machine's CPU, and instances can then run by relying on these VCPUs.

[0043] This embodiment proposes adding a queue component to the host machine's CPU. This queue component can be implemented using either hardware or software. Optionally, this embodiment can use hardware such as an Application-Specific Integrated Circuit (ASIC) to construct the queue component; however, this embodiment is not limited to this. Furthermore, the queue component added to the CPU in this embodiment, besides providing queue functionality for I / O processing in this embodiment, can also be reused in other scenarios to provide queue functionality for other applications.

[0044] In this embodiment, the queue component can access the CPU's internal bus. This embodiment does not limit the form in which the queue component accesses the CPU's internal bus; the access method can be selected according to the actual situation. For example, the queue component can connect to the CPU's PCIe controller to access the CPU's internal bus. Alternatively, the queue component can connect to other CPU components via the UCIe bus. Furthermore, the queue component can also connect to the CPU's memory controller or optional acceleration modules to interact with the CPU's memory or acceleration modules. On the other hand, the queue component can also interconnect with an I / O offload card plugged into the host machine. Optionally, the queue component can interconnect with the I / O offload card via a cache coherence bus protocol such as CXL. Thus, referring to... Figure 1 The queue component added to the CPU can support the linkage between the I / O offloading card and the host CPU, changing the status quo of the separation of the I / O offloading card and CPU data link in the traditional I / O processing scheme.

[0045] refer to Figure 1In this embodiment, the queue component can be used to provide virtual I / O devices and corresponding device queues for instances on the host machine. In this embodiment, the queue component can perform I / O virtualization on physical I / O devices that need to perform I / O with the instance, generating virtual I / O devices (such as disk devices and network devices as shown in the figure) corresponding to the physical I / O devices. The virtualized disk devices may include, but are not limited to, blk devices, and the virtualized network devices may include, but are not limited to, net devices. The queue component can employ various I / O virtualization schemes to provide virtual I / O devices to instances on the host machine; specific schemes will be detailed later. It should be understood that the virtual I / O devices here are defined at the operating system (OS) level; that is, what the host machine's OS sees is the virtual I / O device, which represents the various physical devices involved in the I / O process. Furthermore, this embodiment does not limit the implementation form of the device queue; it can adopt implementation forms such as first-in-first-out (FIFO) queues. In this embodiment, device queues can be used to manage I / O requests. For ease of description, this embodiment will explain the I / O offloading process from the perspective of a single instance. However, it should be understood that the host machine can host multiple instances, and the I / O offloading scheme for each instance is consistent.

[0046] It is worth noting that, for reference Figure 1 In this embodiment, the queue component can provide several virtual I / O devices for a single instance. For ease of description, the I / O offloading scheme will be described from the perspective of a single virtual I / O device in the following text. However, it should be understood that the same optimization scheme can be used to optimize the I / O processing involved in other virtual I / O devices provided for a single instance.

[0047] Based on this, I / O requests can occur between the instance on the host machine and the virtual I / O devices provided by the queue component. Preferably, the I / O requests between the instance and the virtual I / O devices use the io-uring protocol. By adopting a general protocol such as io-uring, the problem of adapting to different device queues implemented by different CPU platforms when the I / O offloading card interacts with different CPU platforms can be avoided, that is, there is no need to consider compatibility issues. Of course, this is only a preferred option. In this embodiment, the I / O requests between the instance and the virtual I / O devices can also use other protocols, as long as both parties agree in advance. The queue component can schedule the I / O requests between the instance and the virtual I / O devices within the CPU using the device queue. The scheduling algorithm can be a 5-tuple hash, secret key hash, etc., which is not limited in this embodiment. In this way, the device queues involved in I / O processing are moved up to the CPU, and the queue component can work in conjunction with various components within the CPU. This makes the I / O request scheduling work performed by the queue component based on the device queue no longer blind, but can be based on the real-time load of each core within the CPU. This allows for more reasonable scheduling of I / O requests, especially read requests issued by instances to virtual I / O devices, thus avoiding data overflow problems caused by unreasonable scheduling during read requests.

[0048] As mentioned earlier, the queue component also interconnects with the I / O offload card plugged into the host machine. In this way, the I / O offload card no longer needs to handle I / O request scheduling; this task is moved up to the CPU and handled by the queue component. Other functions of the I / O offload card can be retained. The I / O offload card can interface with the device queue provided by the queue component for virtual I / O devices and listen for I / O requests in the device queue. Accordingly, the I / O offload card can monitor I / O requests occurring between the instance and virtual I / O devices.

[0049] Thus, based on the queue component configured for the CPU, the device queue interfaces with the virtual I / O device and the I / O offload card respectively. In this embodiment, the data link is as follows: the virtual I / O device passes the I / O requests that occur between it and the instance to the device queue, and the device queue passes the I / O requests to the I / O offload card so that the I / O offload card can be aware of the I / O requests.

[0050] refer to Figure 1 The I / O offloading card can also be used to transfer data related to detected I / O requests between the instance and the physical I / O device corresponding to the virtual I / O device. The I / O offloading card can act as middleware between the physical I / O device and the instance on the host machine, providing data interaction support for both parties.

[0051] In addition, in this embodiment, the queue component may also provide an acceleration module, which can be used to perform accelerated processing on data related to I / O requests occurring between the instance and the virtual I / O device. The acceleration processing in this embodiment may include, but is not limited to, encryption / decryption, compression, or statistical offloading. Similarly, the I / O offloading card may also provide an acceleration module, which can also be used to perform accelerated processing on data related to I / O requests occurring between the instance and the virtual I / O device. Based on the acceleration module provided in the queue component:

[0052] 1. Customers can configure specified virtual I / O devices to enable encryption / decryption, compression, and statistical offloading functions. The VF bound to the device queue corresponding to the virtual I / O device can be associated with a specified acceleration module. The memory address allocated by the customer to the virtual I / O device can also be associated with the specified acceleration module. In this way, when reading and writing to this memory address, the data will be processed through the specified acceleration module for accelerated processing.

[0053] 2. When the application in the example initiates a write request to the virtual I / O device, the acceleration module can preprocess the data before it enters the memory space allocated by the client.

[0054] 3. The I / O calculation engine of the I / O offloading card receives data from the communication component and DMAs the data to the client-pre-allocated memory space. Before the data enters the client-allocated memory space, the acceleration module can preprocess the data.

[0055] Thus, based on the acceleration module provided in the queue component, the data link in this embodiment becomes: the virtual I / O device passes the I / O requests occurring between itself and the instance to the device queue; the device queue triggers the acceleration module to access the memory address and accelerate the data related to the I / O request; the device queue passes the I / O request to the I / O offload card, so that the I / O offload card can detect the I / O request and read the accelerated data from the memory address. This allows the acceleration module to cover the entire data path. For example, the encryption / decryption module provided by the queue component can ensure that the data is encrypted throughout the entire path, changing the traditional solution where data acceleration can only be performed by the I / O offload card at the end of the data transmission path.

[0056] In summary, in this embodiment, a queue component is added to the CPU of the host, and it is innovatively proposed to use the queue component in the I / O processing process. Based on this, the queue component can provide virtual I / O devices and device queues corresponding to the virtual I / O devices for the instances on the host; within the CPU, the device queue is used to schedule the I / O requests that occur between the instance and the virtual I / O device; and the original I / O offloading card only needs to be responsible for listening to the I / O requests in the device queue and transmitting the data related to the I / O requests. In this way, the device queue in the I / O processing process can be moved up to the CPU, so that it can联动 with various components within the CPU during the I / O scheduling process, obtain the real-time loads of each core within the CPU, and thus can more reasonably implement I / O scheduling instead of blindly scheduling, which can effectively improve the I / O processing efficiency.

[0057] Figure 2 It is a schematic structural diagram corresponding to an optional implementation solution of an I / O offloading system provided by an exemplary embodiment of the present application. Refer to Figure 2 , in the above or the following embodiments, the device queue provided by the queue component may include a first-layer queue and a second-layer queue. Among them, the first-layer queue can be docked to the VCPU under the instance, and the second-layer queue can be docked to the physical I / O device through the I / O offloading card. Refer to Figure 2 , for one of the virtual I / O devices provided for the instance, the queue component can associate a first specified number of first-layer queues and a second specified number of second-layer queues with the virtual I / O device; establish a mapping relationship between the first-layer queue and the second-layer queue associated with the virtual I / O device; dock the first-layer queue associated with the virtual I / O device to each VCPU under the instance; and dock the second-layer queue associated with the virtual I / O device to the physical I / O device corresponding to the virtual I / O device through the I / O offloading card.

[0058] Figure 3 It is a schematic structural diagram of an optional implementation solution of an I / O offloading system provided by an exemplary embodiment of the present application, which shows the implementation solution of docking the first-layer queue associated with the virtual I / O device to the VCPU under the instance and the implementation solution of docking the second-layer queue associated with the virtual I / O device to the physical I / O device corresponding to the virtual I / O device.

[0059] Refer to Figure 3As mentioned earlier, the queue component can employ various I / O virtualization schemes to provide virtual I / O devices to instances on the host machine. In one exemplary implementation: SRIOV (Single Root I / O Virtualization) technology can be used to create VF devices for physical I / O devices that need to perform I / O with the instance. These VF devices are used to exchange data with various VCPUs within the instance. The instance's operating system registers the VF devices as virtual I / O devices of a specified type. For example, the instance's operating system can determine the type of virtual I / O device based on the VF's PCIe...<vender ID、device ID> The Virtual Functions (VFs) are registered as disks, network devices, etc. In this embodiment, the physical I / O devices may include, but are not limited to, high-performance I / O devices such as cloud network cards and cloud disks (e.g., solid-state drives, SSDs). In practical applications, each SR-I / OV device can have one physical function (PF), and each PF can have up to 64,000 associated virtual functions (VFs). A PF can create VFs through registers designed with attributes specifically for this purpose. Once SR-I / OV is enabled in a PF, the PCI configuration space of each VF can be accessed through the PF's bus, device, and function number. Each VF has a PCI memory space used to map its register set. The VF device driver operates on the register set to enable its function and appears as an actual existing PCI device. After VF creation, it can be directly assigned to various applications within an instance, allowing VF devices in the queue component to exchange data with various VCPUs within the instance.

[0060] Based on this, refer to Figure 3 The first-level queue associated with the virtual I / O device can be bound to the VF device corresponding to the virtual I / O device to exchange data with each VCPU in the instance.

[0061] Continue to refer to Figure 3For the I / O offloading card, physical I / O devices can be simulated to generate simulated devices corresponding to the physical I / O devices. Based on this, the second-level queue associated with the virtual I / O device can be bound to the simulated device, thus connecting the second-level queue associated with the virtual I / O device to the physical I / O device corresponding to the virtual I / O device. The I / O offloading card can use software or hardware simulation to simulate the physical I / O device; this embodiment does not limit this. It should be understood that in a cloud environment, the deployment of physical I / O devices may be distributed or clustered. The I / O offloading card simulates simulated devices on the corresponding distributed system or cluster to participate in the I / O processing. Ultimately, the I / O offloading card can accurately deliver I / O requests to the physical I / O device through the simulated devices. That is, the I / O computing engine in the I / O offloading card can provide a specified number of simulated devices, and each simulated device is bound to a specified number of second-level queues in the device queue within the CPU to read or send data. Of course, in this embodiment, other implementation schemes can also be used to connect the second-level queue associated with the virtual I / O device to the physical I / O device corresponding to the virtual I / O device, and it is not limited to the simulation scheme. These will not be listed exhaustively here. Figure 4 This is a logical diagram illustrating an instance creation process provided for an exemplary embodiment of this application. Figure 5 This is a logical diagram illustrating an instance destruction process provided as an exemplary embodiment of this application. The queue docking scheme described above will be explained below through the instance creation / destruction process.

[0062] refer to Figure 4 An exemplary instance creation process could be:

[0063] 1. Customers initiate instance creation requests via the console or openAPI.

[0064] 2. The console can schedule creation requests to the most suitable host machine.

[0065] 3. An instance management agent can run on the host machine to create simulated devices corresponding to physical I / O devices such as cloud network cards or cloud disks on the I / O computing engine of the I / O offloading card, and bind them to a specified number of second-level queues in the device queue.

[0066] 4. The queue component in the host machine's CPU can create multiple VFs as required on the device queue, bind them to the specified first-level queue, and establish a mapping relationship with the second-level queue in step 3.

[0067] 5. The instance management agent on the host machine can create instances by calling the Hypervisor.

[0068] refer to Figure 5An exemplary instance destruction process could be:

[0069] 1. Customers can initiate instance destruction requests via the console or OpenAPI.

[0070] 2. The console can locate the host machine where the instance is located, and the deletion command can be issued through the instance management agent on the host machine.

[0071] 3. The instance management agent can delete the simulated devices corresponding to physical I / O devices such as cloud network cards or cloud disks on the I / O computing engine of the I / O offloading card, and delete the related VFs in the queue component of the host machine's CPU.

[0072] 4. The instance management agent calls the Hypervisor to delete the instance.

[0073] from Figure 4 and Figure 5 As can be seen, in this embodiment, during the creation / destruction of the instance, multiple virtual I / O devices and multiple simulated devices are created at the granularity of a single instance, and a two-level queue docking process is performed. It is worth noting that during the normal operation of the instance, if physical I / O devices are added or removed, configuration can be performed at the granularity of a single device. Figure 3 A single virtual I / O device manages a string of entity objects (VF device - first-level queue - second-level queue - simulated device). In other words, each additional physical I / O device that needs to perform I / O with the instance adds [a certain number of virtual I / O devices]. Figure 3 One virtual I / O device and a list of entity objects managed by that virtual I / O device.

[0074] Furthermore, it's worth noting that in this embodiment, the virtual I / O device is generated by virtualizing the physical I / O device, primarily used to support instance discovery of physical I / O devices on the host machine. The emulated device in the I / O offloading card is generated by simulating the physical I / O device, primarily used to simulate the hardware behavior of the physical I / O device. Based on the various optional implementation schemes provided throughout this document, in this embodiment, the entities involved in the I / O processing may include: VF device - first-layer queue - second-layer queue - emulated device - network interface card (NIC) - physical I / O device. The aforementioned virtual I / O device represents these entities involved in the I / O processing. The emulated device in the I / O offloading card represents the last physical I / O device mentioned among these entities. Both the virtual I / O device and emulated device functions in this embodiment are used to support I / O virtualization in cloud-native environments. Thus, in this embodiment, the first-level queue associated with the virtual I / O device can be bound to the VF device and each VCPU under the instance for data exchange; the second-level queue associated with the virtual I / O device can be bound to the simulated device in the I / O offload card for data exchange. The first-level queue associated with the virtual I / O device can be mapped to its associated second-level queue in an N:1 ratio, that is, the number of first-level queues associated with the virtual I / O device is usually greater than the number of its associated second-level queues. Based on this two-level queue structure, write requests initiated by the instance to the virtual I / O device can be scheduled to the appropriate physical I / O device by mapping from the first-level queue to the second-level queue; read requests initiated by the instance to the virtual I / O device can be scheduled to the appropriate VCPU by mapping from the second-level queue to the first-level queue.

[0075] The scheduling process based on a two-layer queue structure will be described in detail below.

[0076] refer to Figure 2The queue component can read the load information of each VCPU under the instance during the process of the instance initiating a read request to the virtual I / O device. Based on the load information, it schedules the read request to the first queue in the first-level queue associated with the virtual I / O device, so that the VCPU connected to the first queue can process the read request. Optionally, the first queue can be the queue connected to the VCPU with the lowest load. In this way, read requests can be scheduled to the VCPU with the optimal load for processing, thereby improving I / O processing efficiency. During this process, the I / O offloading card can obtain the response message corresponding to the read request; it adds the metadata information in the response message to a designated queue in the second-level queue associated with the virtual I / O device. The queue component can then schedule the metadata information in the designated queue to the first queue based on the load information. Additionally, the host machine can allocate memory space for the instance, and the I / O offloading card can use this memory space to transfer data related to the monitored I / O requests between the instance and the physical I / O device corresponding to the virtual I / O device. In one exemplary scheme: the I / O offloading card can write the data portion of the response message into the memory space corresponding to the instance; it adds the memory address where the data portion is located to the metadata information; based on this, the queue component can trigger the VCPU connected to the target queue to read the data portion of the response message according to the memory address in the metadata information. This completes the read request initiated by the instance to the virtual I / O device.

[0077] In this embodiment, when an instance reads or writes to a virtual I / O device, the system call can allocate memory space similar to sk_buf in the network. The device queue accesses and references this memory space via address, and the I / O offloading card can access this memory space via DMA or a mode similar to Intel SVM.

[0078] For example, a practical application could be:

[0079] 1. The application in the example can open virtual I / O devices based on the io-uring protocol (e.g., Figure 2 For disk devices and network devices, use io_uring_smp_store_release to submit a read request, and use io_uring_enter to trigger the receiver to retrieve data from the device queue.

[0080] 2. The I / O calculation engine of the I / O offloading card is removed from communication components (such as...) Figure 2 The network interface card (NIC) receives a response message and, based on the message's metadata, determines the emulator to which the message belongs. The data portion of the message is then written to the emulator's DMA memory space. During the data writing process, the acceleration module of the I / O offload card can be used for data preprocessing.

[0081] 3. The I / O calculation engine generates metadata information for read requests in Io-uring format, which includes the DMA memory address to which the data portion is written. The I / O calculation engine can write the metadata information of the read request to the second-level queue of the CPU's device queue.

[0082] 4. The queue scheduler running in the queue component can read the power consumption, time slice utilization, and PMU of each VCPU of the instance, and perform real-time statistics on the real-time load of each VCPU. Combined with the set scheduling strategies such as five-tuple hash and secret key hash, it schedules read requests in the second-level queue to the target queue in the first-level queue.

[0083] 5. The instance's operating system can wake up the application that submitted the read request, read the metadata information of the read request from the target queue of the device queue, read the memory address where the data part is located from the metadata information, and read the data from that memory address.

[0084] refer to Figure 2 The queue component adds the metadata information of write requests initiated by the instance to the virtual I / O device to the first-level queue associated with the virtual I / O device; it then schedules the metadata information of the write request to the second queue in the second-level queue associated with the virtual I / O device; the I / O offloading card can then process the write request using the physical I / O device connected to the second queue. During this process, the queue component writes the data portion corresponding to the write request to the instance's corresponding memory space; it adds the memory address of the data portion to the metadata information; the I / O offloading card reads the metadata information from the second queue; it retrieves the data portion of the write request according to the memory address in the metadata information; and it sends the data portion to the physical I / O device connected to the second queue.

[0085] For example, a practical application could be:

[0086] 1. The application in the example can open virtual I / O devices based on the io-uring protocol (e.g., Figure 2 For disk devices and network devices, use io_uring_smp_store_release to submit write requests and use io_uring_enter to trigger the receiver to retrieve data from the device queue.

[0087] 2. The data portion of the write request resides in the memory space allocated to the application, and the metadata information of the write request enters the first-level queue of the device queue.

[0088] 3. The queue scheduler running in the queue component can send write requests in the first-level queue to the specified second-level queue based on the mapping relationship between the first-level queue and the second-level queue, the weight of the first-level queue, and the number of free queue entries in the second-level queue.

[0089] 4. Use the acceleration module within the CPU for data preprocessing, such as data encryption and data compression.

[0090] 5. The I / O calculation engine of the I / O offloading card reads the metadata information of the write request from the second-level queue of the queue component in the CPU to obtain information such as the memory access address of the data part.

[0091] 6. The I / O computing engine adds metadata headers for virtual network cards and cloud disks, sending application-submitted data to the target host or backend storage cluster via the I / O offload card's communication components. During data transmission, the I / O offload card's acceleration module can be used for data preprocessing.

[0092] Accordingly, the queue component can be used to rationally schedule I / O requests between instances and virtual I / O devices. Moving the device queue to the CPU allows for configuring scheduling policies for CPU-based device queues directly on the instance using the toolkit, eliminating the need for console configuration as in traditional solutions. The scheme for configuring scheduling policies in the device queue provided in this embodiment can increase the configured QPS by several orders of magnitude, solving the problem of low real-time performance in traditional solutions and their inability to meet the rapid creation and destruction requirements of container and serverless scenarios. Furthermore, during the transmission of I / O request-related data by the I / O offloading card, the data can be encrypted using the CPU's encryption / decryption module before entering the I / O offloading card. This ensures that the I / O offloading card cannot see the plaintext data, significantly improving user data security.

[0093] Figure 6 This is a flowchart illustrating an I / O offloading method in a cloud environment, provided as another exemplary embodiment of this application. The method can be implemented by a CPU in the host machine of the aforementioned system embodiment, the CPU being equipped with a queue component. (Reference) Figure 6 The method may include:

[0094] Step 600: Utilize the queue component to provide virtual I / O devices for instances on the host machine;

[0095] Step 601: Configure the corresponding device queue for the virtual I / O device using the queue component;

[0096] Step 602: Within the CPU, the I / O requests occurring between the instance and the virtual I / O device are scheduled using a device queue, so that the I / O offloading card plugged into the host machine can listen to the I / O requests in the device queue and transfer the data related to the listened I / O requests between the instance and the physical I / O device corresponding to the virtual I / O device.

[0097] In an alternative embodiment, a queue component is constructed using an application-specific integrated circuit (ASIC) and the queue component is connected to the CPU's internal bus.

[0098] In one optional embodiment, the device queue interfaces with both the virtual I / O device and the I / O offloading card. The virtual I / O device passes I / O requests between itself and the instance to the device queue, which then passes the I / O requests to the I / O offloading card so that the I / O offloading card becomes aware of the I / O requests.

[0099] In an optional embodiment, the device queue includes a first-level queue and a second-level queue, and the method further includes:

[0100] Associate a first specified number of first-level queues and a second specified number of second-level queues with virtual I / O devices;

[0101] Establish a mapping relationship between the first-level queue and the second-level queue associated with virtual I / O devices;

[0102] Connect the first-level queue associated with the virtual I / O device to each VCPU under the instance;

[0103] The second-level queue associated with the virtual I / O device is connected to the physical I / O device corresponding to the virtual I / O device through the I / O offload card.

[0104] In an optional embodiment, the step of scheduling I / O requests occurring between the instance and the virtual I / O device includes:

[0105] During the process of an instance initiating a read request to a virtual I / O device, the load information of each VCPU under the instance is read;

[0106] Based on the load information, read requests are scheduled to the first queue in the first-level queue associated with the virtual I / O device, so that the VCPU connected to the first queue can process the read requests.

[0107] In an optional embodiment, the step of scheduling read requests to a first queue in a first-level queue associated with a virtual I / O device based on load information includes:

[0108] Based on the load information, the metadata information in the specified queue is scheduled to the first queue; wherein, the I / O offloading card obtains the response message corresponding to the read request and adds the metadata information in the response message to the specified queue in the second-level queue associated with the virtual I / O device.

[0109] In an optional embodiment, the method further includes:

[0110] The VCPU that triggers the target queue connection reads the data portion of the response message according to the memory address in the metadata information; the I / O offload card writes the data portion of the response message into the memory space corresponding to the instance; and the memory address where the data portion is located is added to the metadata information.

[0111] In an optional embodiment, the step of scheduling I / O requests occurring between the instance and the virtual I / O device using a device queue includes:

[0112] Add the metadata information of the write requests initiated by the instance to the virtual I / O device to the first-level queue associated with the virtual I / O device;

[0113] The metadata information of the write request is scheduled to the second queue in the second-level queue associated with the virtual I / O device;

[0114] Write requests are handled by the physical I / O devices connected to the second queue via the I / O offloading card.

[0115] In an optional embodiment, the method further includes:

[0116] Write the data portion corresponding to the write request to the memory space corresponding to the instance; add the memory address where the data portion is located to the metadata information;

[0117] Specifically, the I / O offloading card reads metadata information from the second queue; retrieves the data portion of the write request according to the memory address in the metadata information, and sends the data portion to the physical I / O device connected to the second queue.

[0118] In an alternative embodiment, the step of providing a virtual I / O device for the instance on the host machine includes:

[0119] I / O virtualization is performed on physical I / O devices that need to perform I / O with the instance to create virtual I / O devices corresponding to the physical I / O devices.

[0120] In an optional embodiment, the step of I / O virtualization of the physical I / O devices that need to perform I / O with the instance includes:

[0121] Using SRIOV technology, VF devices are created for physical I / O devices that need to perform I / O with the instance. VF devices are used to exchange data with each VCPU under the instance.

[0122] Register the VF device as a virtual I / O device of a specified type using the instance's operating system.

[0123] In one alternative embodiment, I / O requests between the instance and the virtual I / O device are made using the io-uring protocol.

[0124] In an optional embodiment, the method further includes:

[0125] The acceleration module performs accelerated processing on data related to I / O requests between the instance and the virtual I / O device. The accelerated processing includes one or more of the following: encryption / decryption, compression, and statistical offloading.

[0126] In one optional embodiment, the acceleration module is bound to the memory address allocated by the host machine to the virtual I / O device. The virtual I / O device passes I / O requests between itself and the instance to the device queue. The device queue triggers the acceleration module to access the memory address and accelerate the processing of data related to the I / O request. The device queue passes the I / O request to the I / O offload card so that the I / O offload card can detect the I / O request and read the accelerated data from the memory address.

[0127] It is worth noting that the technical details of the above-mentioned embodiments of the I / O offloading method can be found in the relevant descriptions of the CPU in the foregoing system embodiments. To save space, they will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0128] Figure 7 This is a flowchart illustrating another exemplary embodiment of an I / O offloading method in a cloud environment. This method can be implemented by an I / O offloading card plugged into the host machine in the aforementioned system embodiments. The host machine's CPU is equipped with a queue component. (See reference...) Figure 7 The method may include:

[0129] Step 700: Listen for I / O requests between the instance on the host machine and the virtual I / O device provided by the queue component for the instance from the device queue in the queue component;

[0130] Step 701: Obtain data related to the monitored I / O requests;

[0131] Step 702: Transfer the data related to the detected I / O requests between the instance and the physical I / O device corresponding to the virtual I / O device.

[0132] In an alternative embodiment, a queue component is constructed using an application-specific integrated circuit (ASIC) and the queue component is connected to the CPU's internal bus.

[0133] In one optional embodiment, the device queue includes a first-level queue and a second-level queue, and the queue component associates a first specified number of first-level queues and a second specified number of second-level queues with the virtual I / O device; establishes a mapping relationship between the first-level queues and the second-level queues associated with the virtual I / O device; interfaces the first-level queues associated with the virtual I / O device to each VCPU under the instance; and interfaces the second-level queues associated with the virtual I / O device to the physical I / O device corresponding to the virtual I / O device through an I / O offload card.

[0134] In an optional embodiment, the method may further include:

[0135] Simulate physical I / O devices to generate simulated devices corresponding to the physical I / O devices;

[0136] Bind the second-level queue associated with the virtual I / O device to the emulated device to interface the second-level queue associated with the virtual I / O device to the physical I / O device.

[0137] In an optional embodiment, the method may further include:

[0138] Obtain the response message corresponding to the read request; add the metadata information in the response message to the specified queue in the second-level queue associated with the virtual I / O device, so that the queue component can schedule the metadata information in the specified queue to the first queue according to the load information.

[0139] In one optional embodiment, the step of transmitting data related to the listened I / O request between the instance and the physical I / O device corresponding to the virtual I / O device may include: writing the data portion of the response message into the memory space corresponding to the instance; adding the memory address where the data portion is located to the metadata information so that the queue component can trigger the VCPU connected to the target queue to read the data portion of the response message according to the memory address in the metadata information.

[0140] In an optional embodiment, the queue component may add the metadata information of write requests initiated by an instance to a virtual I / O device to a first-level queue associated with the virtual I / O device; schedule the metadata information of the write requests to a second queue in a second-level queue associated with the virtual I / O device; the method further includes:

[0141] Write requests are handled using the physical I / O devices connected to the second queue.

[0142] In one optional embodiment, the step of transmitting data related to the listened I / O request between the instance and the physical I / O device corresponding to the virtual I / O device may include: reading metadata information from the second queue; obtaining the data portion of the write request according to the memory address in the metadata information; and sending the data portion to the physical I / O device connected to the second queue.

[0143] Specifically, the queue component writes the data portion corresponding to the write request into the memory space corresponding to the instance; and adds the memory address where the data portion is located to the metadata information.

[0144] In one alternative embodiment, I / O requests between the instance and the virtual I / O device are made using the io-uring protocol.

[0145] It is worth noting that the technical details of the above-mentioned embodiments of the I / O offloading method can be found in the relevant descriptions of the I / O offloading card in the aforementioned system embodiments. To save space, these details will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0146] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 700, 701, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different queue layers, queues, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0147] Figure 8 This is a schematic diagram of the structure of a processor CPU provided as another exemplary embodiment of this application. The CPU is installed in a host machine, and a queue assembly 80 is assembled within the CPU. Figure 8 As shown, the CPU can be used to execute one or more computer instructions for:

[0148] The queue component 80 is used to provide virtual I / O devices for instances on the host machine;

[0149] The queue component 80 is used to configure a corresponding device queue for the virtual I / O device;

[0150] Within the CPU, the device queue is used to schedule I / O requests between the instance and the virtual I / O device, so that the I / O offloading card plugged into the host machine can listen to the I / O requests in the device queue and transmit the data related to the listened I / O requests between the instance and the physical I / O device corresponding to the virtual I / O device.

[0151] In an alternative embodiment, a queue component 80 is constructed using an application-specific integrated circuit (ASIC) and is connected to the internal bus of the CPU.

[0152] In one optional embodiment, the device queue interfaces with both the virtual I / O device and the I / O offloading card. The virtual I / O device passes I / O requests between itself and the instance to the device queue, which then passes the I / O requests to the I / O offloading card so that the I / O offloading card becomes aware of the I / O requests.

[0153] In an alternative embodiment, the device queue includes a first-level queue and a second-level queue, and the queue component 80 can also be used for:

[0154] Associate a first specified number of first-level queues and a second specified number of second-level queues with virtual I / O devices;

[0155] Establish a mapping relationship between the first-level queue and the second-level queue associated with virtual I / O devices;

[0156] Connect the first-level queue associated with the virtual I / O device to each VCPU under the instance;

[0157] The second-level queue associated with the virtual I / O device is connected to the physical I / O device corresponding to the virtual I / O device through the I / O offload card.

[0158] In an optional embodiment, the queue component 80, during the scheduling of I / O requests occurring between the instance and the virtual I / O device, can be used to:

[0159] During the process of an instance initiating a read request to a virtual I / O device, the load information of each VCPU under the instance is read;

[0160] Based on the load information, read requests are scheduled to the first queue in the first-level queue associated with the virtual I / O device, so that the VCPU connected to the first queue can process the read requests.

[0161] In an optional embodiment, the queue component 80, during the process of scheduling read requests to a first queue in the first-level queue associated with the virtual I / O device based on load information, can be used to:

[0162] Based on the load information, the metadata information in the specified queue is scheduled to the first queue; wherein, the I / O offloading card obtains the response message corresponding to the read request and adds the metadata information in the response message to the specified queue in the second-level queue associated with the virtual I / O device.

[0163] In an alternative embodiment, the queue component 80 may also be used for:

[0164] The VCPU that triggers the target queue connection reads the data portion of the response message according to the memory address in the metadata information; the I / O offload card writes the data portion of the response message into the memory space corresponding to the instance; and the memory address where the data portion is located is added to the metadata information.

[0165] In an optional embodiment, the queue component 80, during the scheduling of I / O requests between the instance and the virtual I / O device using the device queue, can be used to:

[0166] Add the metadata information of the write requests initiated by the instance to the virtual I / O device to the first-level queue associated with the virtual I / O device;

[0167] The metadata information of the write request is scheduled to the second queue in the second-level queue associated with the virtual I / O device;

[0168] Write requests are handled by the physical I / O devices connected to the second queue via the I / O offloading card.

[0169] In an alternative embodiment, the queue component 80 may also be used for:

[0170] Write the data portion corresponding to the write request to the memory space corresponding to the instance; add the memory address where the data portion is located to the metadata information;

[0171] Specifically, the I / O offloading card reads metadata information from the second queue; retrieves the data portion of the write request according to the memory address in the metadata information, and sends the data portion to the physical I / O device connected to the second queue.

[0172] In an alternative embodiment, the queue component 80 may be used to: provide virtual I / O devices for instances on the host machine

[0173] I / O virtualization is performed on physical I / O devices that need to perform I / O with the instance to create virtual I / O devices corresponding to the physical I / O devices.

[0174] In an optional embodiment, the queue component 80 can be used during the process of I / O virtualization of physical I / O devices that need to perform I / O with the instance, to:

[0175] Using SRIOV technology, VF devices are created for physical I / O devices that need to perform I / O with the instance. VF devices are used to exchange data with each VCPU under the instance.

[0176] Register the VF device as a virtual I / O device of a specified type using the instance's operating system.

[0177] In one alternative embodiment, I / O requests between the instance and the virtual I / O device are made using the io-uring protocol.

[0178] In an optional embodiment, the queue component 80 may further include an acceleration module, and the queue component 80 may also be used for:

[0179] The acceleration module performs accelerated processing on data related to I / O requests between the instance and the virtual I / O device. The accelerated processing includes one or more of the following: encryption / decryption, compression, and statistical offloading.

[0180] In one optional embodiment, the acceleration module is bound to the memory address allocated by the host machine to the virtual I / O device. The virtual I / O device passes I / O requests between itself and the instance to the device queue. The device queue triggers the acceleration module to access the memory address and accelerate the processing of data related to the I / O request. The device queue passes the I / O request to the I / O offload card so that the I / O offload card can detect the I / O request and read the accelerated data from the memory address.

[0181] It is worth noting that the technical details of the CPU embodiments described above can be found in the relevant descriptions of the CPU in the foregoing system embodiments. To save space, they will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0182] Figure 9 This application provides a schematic diagram of an I / O offloading card as an exemplary embodiment. The I / O offloading card is plugged into a host machine, and the host machine's CPU is equipped with a queueing component. (Reference) Figure 9 The I / O offloading card may include a memory 90 and a processor 91. The memory 90 is used to store one or more computer instructions; the processor 91 is coupled to the memory 90 and is used to execute one or more computer instructions for:

[0183] Listen for I / O requests between instances on the host machine and virtual I / O devices provided by the queue component for the instances from the device queue in the queue component;

[0184] Retrieve data related to the monitored I / O requests;

[0185] Data related to detected I / O requests is transferred between the instance and the physical I / O device corresponding to the virtual I / O device.

[0186] In an alternative embodiment, a queue component is constructed using an application-specific integrated circuit (ASIC) and the queue component is connected to the CPU's internal bus.

[0187] In one optional embodiment, the device queue includes a first-level queue and a second-level queue, and the queue component associates a first specified number of first-level queues and a second specified number of second-level queues with the virtual I / O device; establishes a mapping relationship between the first-level queues and the second-level queues associated with the virtual I / O device; interfaces the first-level queues associated with the virtual I / O device to each VCPU under the instance; and interfaces the second-level queues associated with the virtual I / O device to the physical I / O device corresponding to the virtual I / O device through an I / O offload card.

[0188] In an optional embodiment, processor 91 may also be used for:

[0189] Simulate physical I / O devices to generate simulated devices corresponding to the physical I / O devices;

[0190] Bind the second-level queue associated with the virtual I / O device to the emulated device to interface the second-level queue associated with the virtual I / O device to the physical I / O device.

[0191] In an optional embodiment, processor 91 may also be used for:

[0192] Obtain the response message corresponding to the read request; add the metadata information in the response message to the specified queue in the second-level queue associated with the virtual I / O device, so that the queue component can schedule the metadata information in the specified queue to the first queue according to the load information.

[0193] In an optional embodiment, during the process of transmitting data related to the monitored I / O request between the instance and the physical I / O device corresponding to the virtual I / O device, the processor 91 can be used to: write the data portion of the response message into the memory space corresponding to the instance; add the memory address where the data portion is located to the metadata information so that the queue component can trigger the VCPU connected to the target queue to read the data portion of the response message according to the memory address in the metadata information.

[0194] In an optional embodiment, the queue component may add the metadata information of write requests initiated by the instance to the virtual I / O device to a first-level queue associated with the virtual I / O device; schedule the metadata information of the write requests to a second queue in a second-level queue associated with the virtual I / O device; the processor 91 may also be used to:

[0195] Write requests are handled using the physical I / O devices connected to the second queue.

[0196] In an optional embodiment, during the process of transmitting data related to the monitored I / O request between the instance and the physical I / O device corresponding to the virtual I / O device, the processor 91 can be used to: read metadata information from the second queue; obtain the data portion of the write request according to the memory address in the metadata information; and send the data portion to the physical I / O device connected to the second queue.

[0197] Specifically, the queue component writes the data portion corresponding to the write request into the memory space corresponding to the instance; and adds the memory address where the data portion is located to the metadata information.

[0198] In one alternative embodiment, I / O requests between the instance and the virtual I / O device are made using the io-uring protocol.

[0199] Furthermore, such as Figure 9 As shown, the I / O offloading card also includes other components such as a communication component 92 and a power supply component 93. Figure 9 The diagram only shows a portion of the components and does not imply that the I / O offloading card only includes... Figure 9 The components shown.

[0200] It is worth noting that the technical details of the above-mentioned embodiments of the I / O offloading card can be referred to the relevant descriptions of the I / O offloading card in the foregoing system embodiments. To save space, they will not be repeated here, but this should not cause any loss to the scope of protection of this application.

[0201] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be executed by the CPU or I / O offloading card in the above method embodiments.

[0202] The above Figure 9 The memory in a computer is used to store computer programs and can be configured to store various other data to support operation on a computing platform. Examples of this data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc. The memory can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disks, or optical disks.

[0203] The above Figure 9The communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as Wi-Fi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0204] The above Figure 9 The power supply component provides power to the various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides.

[0205] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0206] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0207] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0208] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0209] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0210] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0211] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0212] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0213] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An I / O offloading system in a cloud environment, comprising: The host machine's CPU and an I / O offloading card plugged into the host machine, wherein the CPU is equipped with a queue component; The queue component is used to provide virtual I / O devices and device queues corresponding to the virtual I / O devices for instances on the host machine; within the CPU, the device queues are used to schedule I / O requests occurring between the instances and the virtual I / O devices; The I / O offloading card is used to listen for I / O requests in the device queue and to transmit data related to the listened I / O requests between the instance and the physical I / O device corresponding to the virtual I / O device.

2. The system according to claim 1, wherein the queue component is constructed using an application-specific integrated circuit (ASIC), and the queue component is connected to the internal bus of the CPU.

3. In the system according to claim 1, the device queue is respectively connected to the virtual I / O device and the I / O offloading card, the virtual I / O device transmits I / O requests occurring between itself and the instance to the device queue, and the device queue transmits the I / O requests to the I / O offloading card so that the I / O offloading card can be aware of the I / O requests.

4. The system according to claim 1, wherein the device queue comprises a first-level queue and a second-level queue, and the queue component is further configured to: Associate the virtual I / O device with a first specified number of first-level queues and a second specified number of second-level queues; Establish a mapping relationship between the first-level queue and the second-level queue associated with the virtual I / O device; Connect the first-level queue associated with the virtual I / O device to each VCPU under the instance; The second-layer queue associated with the virtual I / O device is connected to the physical I / O device corresponding to the virtual I / O device through the I / O offload card.

5. The system according to claim 4, wherein the I / O offloading card is further configured to: The physical I / O devices are simulated to generate simulated devices corresponding to the physical I / O devices; The second-level queue associated with the virtual I / O device is bound to the simulated device to interface the second-level queue associated with the virtual I / O device to the physical I / O device.

6. The system according to claim 4, wherein the queue component, during the process of scheduling I / O requests occurring between the instance and the virtual I / O device using the device queue, is configured to: During the process of the instance initiating a read request to the virtual I / O device, the load information of each VCPU under the instance is read; Based on the load information, the read request is scheduled to the first queue in the first-level queue associated with the virtual I / O device, so that the VCPU connected to the first queue can process the read request.

7. The system according to claim 6, wherein the I / O offloading card is further configured to: obtain a response message corresponding to the read request; add the metadata information in the response message to a designated queue in the second-level queue associated with the virtual I / O device; In the process of scheduling the read request to the first queue in the first-level queue associated with the virtual I / O device according to the load information, the queue component is used to: schedule the metadata information in the specified queue to the first queue according to the load information.

8. In the system according to claim 7, during the process of transmitting data related to the monitored I / O request between the instance and the physical I / O device corresponding to the virtual I / O device, the I / O offloading card is used to: write the data portion of the response message into the memory space corresponding to the instance; and add the memory address where the data portion is located to the metadata information; The queue component is also used to: trigger the VCPU of the target queue to read the data portion of the response message according to the memory address in the metadata information.

9. The system according to claim 4, wherein during the scheduling of I / O requests between the instance and the virtual I / O device using the device queue, the queue component is configured to: add the metadata information of the write request initiated by the instance to the virtual I / O device to the first-level queue associated with the virtual I / O device; and schedule the metadata information of the write request to the second queue in the second-level queue associated with the virtual I / O device; The I / O offloading card is also used to: process the write request using the physical I / O device connected to the second queue.

10. The system according to claim 9, wherein the queue component is further configured to: write the data portion corresponding to the write request into the memory space corresponding to the instance; and add the memory address where the data portion is located to the metadata information; During the process of transmitting data related to the monitored I / O request between the instance and the physical I / O device corresponding to the virtual I / O device, the I / O offloading card is used to: read the metadata information from the second queue; obtain the data portion of the write request according to the memory address in the metadata information; and send the data portion to the physical I / O device connected to the second queue.

11. The system of claim 1, wherein the queue component, in providing virtual I / O devices to instances on the host machine, is configured to: I / O virtualization is performed on the physical I / O devices that need to perform I / O with the instance to generate virtual I / O devices corresponding to the physical I / O devices.

12. The system of claim 11, wherein the queue component, during the process of I / O virtualization of physical I / O devices that need to perform I / O with the instance, is used to: Using SRIOV technology, VF devices are created for physical I / O devices that need to perform I / O with the instance. The VF devices are used to exchange data with each VCPU under the instance. The VF device is registered as a virtual I / O device of a specified type using the operating system of the example.

13. The system according to claim 1, wherein the I / O requests between the instance and the virtual I / O device adopt the io-uring protocol.

14. The system according to claim 1, wherein the queue component further comprises an acceleration module, the acceleration module being used for: Perform acceleration processing on data related to I / O requests occurring between the instance and the virtual I / O device, the acceleration processing including: One or more of the following processes: encryption / decryption, compression, and statistical unloading.

15. The system according to claim 14, wherein the acceleration module is bound to a memory address allocated by the host machine for the virtual I / O device, the virtual I / O device passes I / O requests occurring between itself and the instance to the device queue; the device queue triggers the acceleration module to access the memory address and accelerate the data related to the I / O request; the device queue passes the I / O request to the I / O offload card, so that the I / O offload card senses the I / O request and reads the accelerated data from the memory address.

16. An I / O offloading method in a cloud environment, applicable to a CPU in a host machine, wherein the CPU is equipped with a queue component, the method comprising: The queue component is used to provide virtual I / O devices for instances on the host machine; Configure the corresponding device queue for the virtual I / O device using the queue component; Within the CPU, the device queue is used to schedule I / O requests between the instance and the virtual I / O device, so that the I / O offloading card plugged into the host machine can listen to the I / O requests in the device queue and transmit the data related to the listened I / O requests between the instance and the physical I / O device corresponding to the virtual I / O device.

17. An I / O offloading method in a cloud environment, applicable to an I / O offloading card plugged into a host machine, wherein the CPU of the host machine is equipped with a queue component, the method comprising: Listen for I / O requests between the instance on the host machine and the virtual I / O device provided by the queue component for the instance from the device queue in the queue component; Retrieve data related to the monitored I / O requests; Data related to the detected I / O requests is transmitted between the instance and the physical I / O device corresponding to the virtual I / O device.

18. A processor CPU, installed in a host machine, the CPU having a queue assembly therein, the CPU being configured to execute one or more computer instructions for: The queue component is used to provide virtual I / O devices for instances on the host machine; Configure the corresponding device queue for the virtual I / O device using the queue component; Within the CPU, the device queue is used to schedule I / O requests between the instance and the virtual I / O device, so that the I / O offloading card plugged into the host machine can listen to the I / O requests in the device queue and transmit the data related to the listened I / O requests between the instance and the physical I / O device corresponding to the virtual I / O device.

19. An I / O offloading card, plugged into a host machine, wherein the CPU of the host machine is equipped with a queue component, and the I / O offloading card includes a memory and a processor; The memory is used to store one or more computer instructions; The processor is coupled to the memory and is used to execute the one or more computer instructions for: Listen for I / O requests between the instance on the host machine and the virtual I / O device provided by the queue component for the instance from the device queue in the queue component; Retrieve data related to the monitored I / O requests; Data related to the detected I / O requests is transmitted between the instance and the physical I / O device corresponding to the virtual I / O device.

20. A computer-readable storage medium storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the I / O offloading method in a cloud environment as described in any one of claims 16 or 17.

Images