A data acquisition method, device, equipment and storage medium

Abstract

The embodiments of the present application relate to the field of data processing technology, and specifically to a data acquisition method, device, equipment and storage medium, which are intended to ensure reasonable resource allocation when multiple virtual machines share an RDMA network card. The method comprises: receiving a data request through a target network card; determining a virtual interface and a queue pair corresponding to the data request according to a pre-stored memory mapping table; obtaining data corresponding to the data request from a virtual machine corresponding to the data request through the queue pair and the virtual interface; and sending the data corresponding to the data request to a requesting device.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and more specifically, to a data acquisition method, device, equipment and storage medium. Background Art

[0002] RDMA (Remote Direct Memory Access) technology allows computers to directly access the memory of one system from another without the intervention of the operating system, thereby reducing latency and increasing data transfer speeds. In cloud computing and virtualization environments, the high-performance characteristics of RDMA are very valuable for efficient data transfer between virtual machines.

[0003] In the related art, the management of RDMA network cards and queue pair (QP) resources is for the physical host, which causes problems of resource contention and insufficient isolation when multiple virtual machines share one RDMA network card. Summary of the invention

[0004] The embodiments of the present application provide a data acquisition method, apparatus, device and storage medium, which are intended to ensure reasonable allocation of resources when multiple virtual machines share an RDMA network card.

[0005] A first aspect of an embodiment of the present application provides a data acquisition method, the method comprising: Receive data request through target network card; Determine the virtual interface and queue pair corresponding to the data request according to a pre-stored memory mapping table; Obtaining data corresponding to the data request from the virtual machine corresponding to the data request through the queue pair and the virtual interface; The data corresponding to the data request is sent to the requesting device.

[0006] Optionally, before determining the virtual interface and queue pair corresponding to the data request according to the pre-stored memory mapping table, the method further includes: Parsing the data request to determine the business corresponding to the data request; The virtual machine corresponding to the data request is determined according to a pre-stored correspondence relationship between the service and the virtual machine.

[0007] Optionally, determining the virtual interface and queue pair corresponding to the data request according to a pre-stored memory mapping table includes: In the memory mapping table, according to the virtual machine number of the virtual machine corresponding to the data request, searching for the corresponding virtual interface number and queue pair number; The virtual interface and the queue pair corresponding to the virtual interface number and the queue pair number are determined as the virtual interface and the queue pair corresponding to the data request.

[0008] Optionally, obtaining data corresponding to the data request from a virtual machine corresponding to the data request through the queue pair and the virtual interface includes: adding the request to a sending queue in the queue pair; Sending the data request from the sending queue to the virtual machine through the virtual interface; receiving, through the virtual interface, the data returned by the virtual machine according to the data request; The data is placed into a receive queue in the queue pair.

[0009] Optionally, sending the data corresponding to the data request to the requesting device includes: Parsing the data to determine the address of the requesting device corresponding to the data; Adding the data to a sending queue in the queue pair; The data is sent from the sending queue to the requesting device according to the address of the requesting device.

[0010] Optionally, before receiving the data request through the target network card, the method further includes: Creating a preset number of the virtual machines in the host; registering memory for the virtual machine; Initializing the virtual machine; Invoking a preset queue pair creation interface in the target network card to create the queue pair; specifying the corresponding virtual interface for the queue pair; Establishing a mapping between the virtual interface and the virtual machine; Recording a mapping relationship between the queue pair, the virtual interface and the virtual machine in a configuration file of the virtual machine; Loading the configuration file through the virtual machine so that the virtual machine determines the corresponding virtual interface and the queue pair; The mapping relationship between the virtual interface and the queue pair is recorded in the memory mapping table.

[0011] Optionally, the method further comprises: When a newly added virtual machine is detected in the host, a corresponding newly added queue pair is created in the target network card for the newly added virtual machine; Specifying a corresponding new virtual interface for the new queue pair; Establishing a mapping relationship between the newly added virtual interface and the newly added queue pair; The mapping relationship between the newly added virtual interface and the newly added queue pair is recorded in the memory mapping table.

[0012] Optionally, the method further comprises: When the virtual machine is detected to be invalid, the memory occupied by the virtual machine is released, and the virtual interface and the queue pair corresponding to the virtual machine are deleted.

[0013] A second aspect of an embodiment of the present application provides a data acquisition device, the device comprising: A data request receiving module, used for receiving a data request through a target network card; A mapping relationship determination module, used to determine the virtual interface and queue pair corresponding to the data request according to a pre-stored memory mapping table; A data acquisition module, used for acquiring data corresponding to the data request from the virtual machine corresponding to the data request through the queue pair and the virtual interface; The data sending module is used to send the data corresponding to the data request to the requesting device.

[0014] Optionally, the device further comprises: A data request parsing module, used to parse the data request and determine the business corresponding to the data request; The virtual machine determination module is used to determine the virtual machine corresponding to the data request according to the pre-stored correspondence between the business and the virtual machine.

[0015] Optionally, the mapping relationship determination module includes: A number corresponding submodule, used for searching the corresponding virtual interface number and queue pair number in the memory mapping table according to the virtual machine number of the virtual machine corresponding to the data request; The mapping relationship determination submodule is used to determine the virtual interface and the queue pair corresponding to the virtual interface number and the queue pair number as the virtual interface and the queue pair corresponding to the data request.

[0016] Optionally, the data acquisition module includes: A first queue adding submodule, used for adding the request to the sending queue in the queue pair; A first data sending submodule, configured to send the data request from the sending queue to the virtual machine through the virtual interface; A data receiving submodule, configured to receive, through the virtual interface, the data returned by the virtual machine according to the data request; The second queue adding submodule is used to put the data into the receiving queue in the queue pair.

[0017] Optionally, the data sending module includes: An address determination submodule, used to parse the data and determine the address of the requesting device corresponding to the data; A third queue adding submodule, used for adding the data to a sending queue in the queue pair; The data sending submodule is used to send the data from the sending queue to the requesting device according to the address of the requesting device.

[0018] Optionally, the device further comprises: A virtual machine creation module, used to create a preset number of virtual machines in the host; A memory registration module, used to register memory for the virtual machine; An initialization module, used for initializing the virtual machine; A queue pair creation module, used to call a preset queue pair creation interface in the target network card to create the queue pair; An interface specifying module, used for specifying the corresponding virtual interface for the queue pair; A mapping relationship establishing module, used to establish a mapping between the virtual interface and the virtual machine; A first mapping relationship recording module, used for recording a mapping relationship between the queue pair, the virtual interface and the virtual machine in a configuration file of the virtual machine; A configuration file loading module, used to load the configuration file through the virtual machine, so that the virtual machine determines the corresponding virtual interface and the queue pair; The second mapping relationship recording module is used to record the mapping relationship between the virtual interface and the queue pair in the memory mapping table.

[0019] Optionally, the device further comprises: A new queue pair creation module is used to create a corresponding new queue pair for the new virtual machine in the target network card when a new virtual machine is detected in the host; A newly added virtual interface specifying module is used to specify a corresponding newly added virtual interface for the newly added queue pair; A newly added mapping relationship establishing module is used to establish a mapping relationship between the newly added virtual interface and the newly added queue pair; A newly added mapping relationship recording module is used to record the mapping relationship between the newly added virtual interface and the newly added queue pair in the memory mapping table.

[0020] Optionally, the device further comprises: The memory release and information deletion module is used to release the memory occupied by the virtual machine and delete the virtual interface and the queue pair corresponding to the virtual machine when the virtual machine fails.

[0021] A third aspect of an embodiment of the present application provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps in the method described in the first aspect of the present application are implemented.

[0022] A fourth aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in the first aspect of the present application are implemented.

[0023] Using the data acquisition method provided by the present application, a data request is received through a target network card; according to a pre-stored memory mapping table, a virtual interface and a queue pair corresponding to the data request are determined; The data corresponding to the data request is obtained from the virtual machine corresponding to the data request through the queue pair and the virtual interface; and the data corresponding to the data request is sent to the requesting device.

[0024] In this method, a corresponding virtual machine interface and a queue pair are specified for each virtual machine. When the target network card receives a data request, data is transmitted through the corresponding queue pair according to the correspondence between the virtual machine, the virtual interface and the queue pair stored in the memory mapping table. In this way, it can be ensured that multiple virtual machines share a target network card, and the queue pairs of each virtual machine do not interfere with each other, thereby ensuring the reasonable allocation of resources, ensuring that multiple virtual machines share the same RDMA network card, and that multiple virtual machines do not interfere with each other when transmitting data. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Figure 1 is a flow chart of a data acquisition method proposed in one embodiment of the present application; Figure 2This is a schematic diagram of the host structure proposed in one embodiment of the present application; Figure 3 is a schematic diagram of a data acquisition device proposed in an embodiment of the present application; Figure 4 It is a schematic diagram of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] refer to Figure 1 , Figure 1 : is a flow chart of a data acquisition method proposed in one embodiment of the present application. Figure 1 As shown, the method comprises the following steps: S11: receiving a data request through the target network card.

[0029] In this embodiment, the target network card is an RDMA network card, and the data request is a request sent to the device where the RDMA network card is located, and is used to request data in a virtual machine on the device where the RDMA network card is located.

[0030] In the related technologies, RDMA RoCEv2: RDMA over Converged Ethernet version 2 is a widely used RDMA implementation that transmits RDMA data over Ethernet, providing a more efficient data transmission method for large-scale data centers and virtualized environments. However, RoCEv2 itself does not provide queue pair (QP) management functions for virtual machines, making it challenging to use RDMA technology in a virtualized environment.

[0031] SR-IOV technology provides a virtual interface, which is a hardware virtualization technology that allows multiple virtual machines to directly access the virtual function (Virtual Function, VF) of the physical network interface card (NIC). The specific embodiment of this virtual function is the virtual interface. Each virtual machine can be assigned one or more virtual interfaces to directly access hardware resources, such as RDMA network cards, reducing virtualization overhead and latency. However, in this technology, SR-IOV implements the virtualization of hardware resources through physical functions (Physical Function, PF) and virtual functions (Virtual Function, VF), and each virtual machine communicates directly with the physical network card through one or more VFs. Although this method reduces the abstraction level between virtual machines and physical devices and improves performance, it is relatively rigid in resource isolation and sharing. The number and configuration of VFs are fixed, and resource allocation cannot be flexibly adjusted. In SR-IOV, the isolation between different virtual functions (VFs) mainly depends on hardware support. Although there is a certain degree of isolation at the hardware level, in actual operation, when multiple virtual machines share the same physical network card, resource contention or performance interference may still occur, especially in network card queue management and cache usage. The scalability of SR-IOV is limited by physical hardware. The number of VFs that each physical network card can support is limited, which directly affects the number of virtual machines that can be supported. Once the maximum number of VFs supported by the hardware is reached, it cannot be expanded further. SR-IOV requires hardware (such as NIC) to support SR-IOV functions and requires the hardware to have corresponding virtualization functions. Dependence on specific hardware limits its versatility in different environments and devices. When SR-IOV is used, a virtualization hypervisor (such as KVM, VMware ESXi, Hyper-V, etc.) is required to manage hardware support and resource allocation, which requires the virtualization hypervisor to have corresponding support and compatibility. In some cases, different virtualization hypervisors have different levels of support for SR-IOV, resulting in limitations in application scenarios. Although SR-IOV can reduce some virtualization overhead, there is still additional performance overhead and management complexity, especially when multiple virtual machines and large amounts of data transmission are involved, the configuration and management of SR-IOV will become more complicated.

[0032] refer to Figure 2 , Figure 2 is a schematic diagram of the host structure proposed in an embodiment of the present application, such as Figure 2As shown, the RDMA network card includes a port and a DMA controller. A plurality of queue pairs (QP, Queue Pair) are arranged in the DMA controller, which is the basic communication unit of RDMA and is composed of a sending queue SQ and a receiving queue RQ. Three queue pairs are arranged in the network card, and the number of queue pairs can be expanded as needed. Every two queue pairs correspond to a virtual interface, or each queue pair corresponds to a virtual interface. The setting can be made according to the actual situation. A plurality of virtual machines, VM1, VM2, and VM3, are arranged on the host side. The DMA controller can read the corresponding data from the storage space of the corresponding virtual machine through the corresponding virtual interface according to the mapping relationship between the virtual machine, the virtual interface, and the queue pair stored in the memory mapping table, and then send the data to the external device.

[0033] In this embodiment, a data request is first received through the network card, and different services are executed in each virtual machine. When the external device needs to receive service data, it sends the data request to the network card on the host side, and obtains the corresponding service data from the virtual machine through the network card.

[0034] S12: Determine the virtual interface and queue pair corresponding to the data request according to the pre-stored memory mapping table.

[0035] In this embodiment, the memory mapping table is used to store the mapping relationship between each virtual machine and the virtual interface and the queue pair. The virtual interface is the data transmission interface on the virtual network card. The queue pair includes a receiving queue and a sending queue. The receiving queue is responsible for receiving data sent from the outside, placing the data in the receiving queue, and processing it in sequence. The sending queue is the queue where the network card sends data to the outside, and the data in the queue is sent out in sequence.

[0036] In this embodiment, after receiving a data request, the service corresponding to the data request is determined, and then the virtual machine corresponding to the service is determined, and then the virtual interface and queue pair corresponding to the virtual machine are searched in the memory mapping table according to the pre-stored memory mapping table.

[0037] In this embodiment, the specific steps of determining the virtual interface and queue pair corresponding to the data request according to the pre-stored memory mapping table include: S12-1: In the memory mapping table, according to the virtual machine number of the virtual machine corresponding to the data request, search for the corresponding virtual interface number and queue pair number.

[0038] In this embodiment, each virtual machine is set with a corresponding virtual machine number when it is created, and each virtual interface and queue pair has a number. The virtual interface number and queue pair number corresponding to each virtual machine number are stored in the memory mapping table.

[0039] In this embodiment, in the memory mapping table, according to the virtual machine number of the virtual machine corresponding to the data request, the virtual interface number and the queue pair number corresponding to the virtual machine number are searched, and each virtual interface number may correspond to multiple queue pair numbers.

[0040] For example, virtual machine 1 corresponds to virtual interface 1 and queue pair 1 and queue pair 2.

[0041] S12-2: Determine the virtual interface and the queue pair corresponding to the virtual interface number and the queue pair number as the virtual interface and the queue pair corresponding to the data request.

[0042] In this embodiment, after the virtual interface number and queue pair number corresponding to the virtual machine number are found, the virtual interface and queue pair corresponding to the virtual interface number and queue pair number are determined as the virtual interface and queue pair corresponding to the data request.

[0043] In this embodiment, by allocating one or more queue pairs to each virtual machine, data transmission between each queue pair is performed independently, and each virtual machine will not interfere with each other. Each queue pair maintains its own sending queue and receiving queue, ensuring data security and isolation, thereby preventing data leakage or unauthorized access.

[0044] S13: Obtaining data corresponding to the data request from the virtual machine corresponding to the data request through the queue pair and the virtual interface.

[0045] In this embodiment, after the virtual machine, virtual interface, and queue pair corresponding to the data request are determined, the corresponding data is obtained from the memory space in the virtual machine corresponding to the data request.

[0046] In this embodiment, the specific steps of obtaining the data corresponding to the data request from the virtual machine corresponding to the data request through the queue pair and the virtual interface include: S13-1: Add the request to the sending queue in the queue pair.

[0047] In this embodiment, when the RDMA network card receives a data request, the data request is added to the sending queue in the queue pair, and the request is prepared to be sent to the corresponding virtual machine. When in the sending queue, if there is data to be sent in the queue, it will be sent in the order of entering the queue. When there is no data to be sent in the queue, the data request will be sent directly to the virtual machine.

[0048] S13-2: Send the data request from the sending queue to the virtual machine through the virtual interface.

[0049] In this embodiment, the virtual interface is a data transmission interface between the network card and the virtual machine. The data request is sent from the sending queue to the virtual machine through the corresponding virtual interface.

[0050] S13-3: Receive the data returned by the virtual machine according to the data request through the virtual interface.

[0051] In this embodiment, after the data request is sent to the virtual machine, the virtual machine reads and parses the data request, sends the data corresponding to the data request to the corresponding virtual interface, and the RDMA network card receives the returned data through the virtual interface corresponding to the virtual machine.

[0052] S13-4: Put the data into the receiving queue in the queue pair.

[0053] In this embodiment, after receiving the data returned by the virtual machine, the network card puts the received data into the receiving queue in the queue pair. The network card will parse the data in the receiving queue in the order of reception. If there is no received data in the receiving queue, the data will be directly parsed and processed. If there is already received data in the receiving queue, the data will be queued in the receiving queue and wait. After the received data is processed, the data sent by the virtual machine will be processed.

[0054] S14: Sending data corresponding to the data request to the requesting device.

[0055] In this embodiment, after the network card receives the data corresponding to the data request, it places the data in a receiving queue, parses the received data, and then obtains the device address of the data requesting device corresponding to the data, and then places the data in a sending queue, and sends the data from the sending queue to the corresponding data requesting device.

[0056] For example, the requesting device needs to access a video data in the host, sends a data request to the host, parses the data request, and obtains that the business data corresponding to the data request is a specified video data, and then determines that the virtual machine responsible for storing the video data is virtual machine 1, and then according to the memory mapping table, determines that virtual machine 1 corresponds to virtual interface 1, queue pair 1, and then sends the request from the sending queue of queue 1 to the virtual machine, and the RDMA network card controller can access the storage space corresponding to virtual machine 1, and then obtain the corresponding video data. The virtual machine sends the video data to the receiving queue in queue pair 1 through virtual interface 1, and queue pair 1 sends the video data to the requesting device through the sending queue, and the requesting device can obtain the video data.

[0057] In this embodiment, the specific steps of sending the data corresponding to the data request to the requesting device include: S14-1: Parse the data to determine the address of the requesting device corresponding to the data.

[0058] In this embodiment, after the network card receives the data sent by the virtual machine, the data in the receiving queue is parsed to obtain the address of the requesting device corresponding to the data.

[0059] S14-2: Add the data to the sending queue in the queue pair.

[0060] In this embodiment, after determining the address of the requesting device corresponding to the data, the data is placed in a sending queue. If there is no data to be sent that has entered the sending queue first, the data is sent directly to the corresponding address. If there is pre-added data to be sent in the sending queue, the data is sent in the order in which it enters the sending queue. After the data that entered the queue first is sent, the data obtained from the virtual machine is sent.

[0061] S14-3: Send the data from the sending queue to the requesting device according to the address of the requesting device.

[0062] In this embodiment, after the address of the requesting device is determined, the data sent by the virtual machine is sent from the sending queue to the requesting device.

[0063] In this embodiment, by allocating an independent queue pair to each virtual machine, the data transmission channels between different virtual machines are isolated from each other, and multiple virtual machines can share an RDMA network card, but the RDMA operation of each virtual machine is performed through the independent queue pair allocated to it, allowing the device to achieve efficient resource utilization while maintaining high performance.

[0064] In another embodiment of the present application, before determining the virtual interface and queue pair corresponding to the data request according to the pre-stored memory mapping table, the method further includes: S21: Analyze the data request to determine the service corresponding to the data request.

[0065] In this embodiment, after receiving the data request, the network card analyzes the data request and further determines the service corresponding to the data request.

[0066] For example, the service corresponding to the data request may be a video service or a login service.

[0067] S22: Determine the virtual machine corresponding to the data request according to the pre-stored correspondence between the service and the virtual machine.

[0068] In this embodiment, each virtual machine stores different business data and is responsible for different businesses. The business that each virtual machine is responsible for is pre-stored in the host. After receiving a data request, the network card determines the business corresponding to the data request, and then determines the virtual machine corresponding to the data request based on the correspondence between the pre-stored business and the virtual machine.

[0069] In this embodiment, multiple virtual machines are set up in a host, each virtual machine stores different data or is responsible for different services. Multiple virtual machines share the same RDMA network card. When the network card receives a data request, it sends it to the corresponding virtual machine to obtain the corresponding data. During the data transmission process, it is sent through the queue pair corresponding to the virtual machine, thereby ensuring efficient transmission of business data.

[0070] In another embodiment of the present application, before receiving the data request through the target network card, the method further includes: S31: Creating a preset number of virtual machines in the host.

[0071] In this embodiment, a virtual machine refers to a complete computer system with complete hardware system functions that is simulated by software and runs in a completely isolated environment. All the work that can be done in a physical computer can be done in a virtual machine. When creating a virtual machine in a computer, part of the hard disk and memory capacity of the physical machine needs to be used as the hard disk and memory capacity of the virtual machine. Each virtual machine has an independent CMOS, hard disk and operating system, and the virtual machine can be operated like a physical machine.

[0072] In this embodiment, a preset number of virtual machines are first created in the host, and the number of virtual machines can be set according to demand. The number of virtual machines to be created and various parameters can be configured in the virtual machine creation tool.

[0073] S32: Register memory for the virtual machine.

[0074] In this embodiment, after the virtual machine is created, memory is registered for the virtual machine in the host. The size of the memory can be set according to actual conditions. After the memory is registered, the registered part of the memory is used by the virtual machine. The virtual machine can store corresponding data in the registered memory. This part of the memory can be released only when the virtual machine fails, otherwise it will be used by the virtual machine.

[0075] S33: Initialize the virtual machine.

[0076] In this embodiment, after registering the memory, the virtual machine is initialized. During the initialization process, various configurations of the virtual machine are loaded into the virtual machine so that the virtual machine can be used normally.

[0077] S34: calling a preset queue pair creation interface in the target network card to create the queue pair.

[0078] In this embodiment, the queue pair creation interface encapsulates the program required to create a queue pair. By calling the queue pair creation interface, the corresponding program can be executed to create a queue pair.

[0079] S35: Specify the corresponding virtual interface for the queue pair.

[0080] In this embodiment, the virtual interface is a network port virtualized by software virtualization technology.

[0081] In this embodiment, after the queue pair is created, a corresponding virtual interface is specified for the created queue pair. After the virtual interface is specified, data of the queue is sent and received through the virtual interface.

[0082] S36: Establish a mapping between the virtual interface and the virtual machine.

[0083] In this embodiment, after the virtual interface is specified for the queue pair, a mapping relationship between the virtual interface and the virtual machine is established, thereby completing the establishment of a mapping relationship between the queue pair-virtual interface-virtual machine.

[0084] S37: Record the mapping relationship between the queue pair, the virtual interface and the virtual machine in a configuration file of the virtual machine.

[0085] In this embodiment, after the mapping relationship among the queue pair, the virtual interface and the virtual machine is established, the mapping relationship is recorded in the configuration file of the virtual machine.

[0086] S38: Load the configuration file through the virtual machine, so that the virtual machine determines the corresponding virtual interface and the queue pair.

[0087] In this embodiment, a configuration file recording the mapping relationship is loaded through the virtual machine. After the configuration file is loaded, the virtual machine can determine the corresponding virtual interface and queue pair, and then when sending and receiving data, data is only received and sent through the corresponding virtual interface, and data is transmitted through the corresponding queue pair. Different virtual machines will not occupy each other's virtual interfaces and queue pairs.

[0088] S39: Record the mapping relationship between the virtual interface and the queue pair in the memory mapping table.

[0089] In this embodiment, after the mapping relationship between the virtual machine, the virtual machine interface and the queue pair is established, the mapping relationship is recorded in a memory mapping table. The memory mapping table stores the mapping relationship between each virtual machine in the host and the virtual interface and the queue pair. By searching the memory mapping table, the virtual interface and queue pair corresponding to each virtual machine can be determined.

[0090] In another embodiment of the present application, the data sending status and data receiving status of the queue pair are monitored in real time to ensure that the sending queue and receiving queue in each queue pair can send and receive data normally, and timely response can be made when problems occur in the queue pair to ensure stable operation of the system.

[0091] In another embodiment of the present application, the method further includes: S41: when a newly added virtual machine is detected in the host, creating a corresponding newly added queue pair for the newly added virtual machine in the target network card; S42: specifying a corresponding new virtual interface for the new queue pair; S43: Establishing a mapping relationship between the newly added virtual interface and the newly added queue pair; S44: Record the mapping relationship between the newly added virtual interface and the newly added queue pair in the memory mapping table.

[0092] In another embodiment of the present application, when a new virtual machine is added to the host, a new virtual machine is first created, and a new virtual interface and queue pair are created. At the same time, a mapping relationship between the newly added virtual machine and the newly created virtual interface and queue pair is established, and the memory mapping table is updated. The mapping relationship between the newly added virtual machine, the virtual interface and the queue pair is stored in the memory mapping table. When the number of virtual machines in the host increases, the corresponding virtual interface and queue pair can be specified for the newly added virtual machine, thereby realizing flexible capacity expansion of the host. At the same time, the operating status of each virtual machine is scanned at regular intervals. When there is a problem with the operating status of the virtual machine, an alarm message is sent in time and the virtual machine is set to a disabled state. If the virtual machine cannot be restarted and repaired, the virtual machine is deleted, and the registered memory and the corresponding virtual interface and queue pair are released, thereby ensuring the stable operation of the system.

[0093] In another embodiment of the present application, the method further includes: S51: when it is detected that the virtual machine fails, releasing the memory occupied by the virtual machine, and deleting the virtual interface and the queue pair corresponding to the virtual machine.

[0094] In this embodiment, when a virtual machine failure is detected, it is necessary to release the memory occupied by the virtual machine and delete the virtual interface and queue pair corresponding to the virtual machine.

[0095] In another embodiment of the present application, when a new virtual machine is added to the host, a new virtual machine is first created, and a new virtual interface and queue pair are created. At the same time, a mapping relationship between the newly added virtual machine and the newly created virtual interface and queue pair is established, and the memory mapping table is updated. The mapping relationship between the newly added virtual machine, virtual interface and queue pair is stored in the memory mapping table. At the same time, the running status of each virtual machine is scanned at regular intervals. When there is a problem with the running status of the virtual machine, an alarm message is sent in time and the virtual machine is set to a disabled state. If the virtual machine cannot be restarted and repaired, the virtual machine is deleted, the registered memory and the corresponding virtual interface and queue pair are released, thereby ensuring the stable operation of the system.

[0096] In the above embodiments of the present application, by establishing a mapping relationship between queue pairs and virtual machines, different queue pairs are dynamically allocated to different virtual machines for use, resource allocation can be dynamically adjusted according to the actual needs of the virtual machines, and data channels can be controlled and isolated more finely. By providing separate queue management and cache for each queue pair, the isolation between data channels is more thorough, avoiding interference and resource contention between different virtual machines. Queue isolation not only physically separates resources, but also logically ensures the security and independence of data. At the software level, it manages and allocates queue pairs more flexibly, supports larger-scale virtual machine clusters, improves the scalability of virtual machine clusters, does not need to upgrade and expand existing hardware, saves costs, and improves the scope of application and deployment flexibility of virtual machines. It simplifies the resource management and configuration complexity in the RDMA virtualization environment, and significantly improves the overall performance and management efficiency.

[0097] Based on the same inventive concept, an embodiment of the present application provides a data acquisition device. Figure 3 , Figure 3 FIG. 3 is a schematic diagram of a data acquisition device 300 proposed in an embodiment of the present application. Figure 3 As shown, the device comprises: The data request receiving module 301 is used to receive the data request through the target network card; A mapping relationship determination module 302, configured to determine the virtual interface and queue pair corresponding to the data request according to a pre-stored memory mapping table; A data acquisition module 303, configured to acquire data corresponding to the data request from the virtual machine corresponding to the data request through the queue pair and the virtual interface; The data sending module 304 is used to send the data corresponding to the data request to the requesting device.

[0098] Optionally, the device further comprises: A data request parsing module, used to parse the data request and determine the business corresponding to the data request; The virtual machine determination module is used to determine the virtual machine corresponding to the data request according to the pre-stored correspondence between the business and the virtual machine.

[0099] Optionally, the mapping relationship determination module includes: A number corresponding submodule, used for searching the corresponding virtual interface number and queue pair number in the memory mapping table according to the virtual machine number of the virtual machine corresponding to the data request; The mapping relationship determination submodule is used to determine the virtual interface and the queue pair corresponding to the virtual interface number and the queue pair number as the virtual interface and the queue pair corresponding to the data request.

[0100] Optionally, the data acquisition module includes: A first queue adding submodule, used for adding the request to the sending queue in the queue pair; A first data sending submodule, configured to send the data request from the sending queue to the virtual machine through the virtual interface; A data receiving submodule, configured to receive, through the virtual interface, the data returned by the virtual machine according to the data request; The second queue adding submodule is used to put the data into the receiving queue in the queue pair.

[0101] Optionally, the data sending module includes: An address determination submodule, used to parse the data and determine the address of the requesting device corresponding to the data; A third queue adding submodule, used for adding the data to a sending queue in the queue pair; The data sending submodule is used to send the data from the sending queue to the requesting device according to the address of the requesting device.

[0102] Optionally, the device further comprises: A virtual machine creation module, used to create a preset number of virtual machines in the host; A memory registration module, used to register memory for the virtual machine; An initialization module, used for initializing the virtual machine; A queue pair creation module, used to call a preset queue pair creation interface in the target network card to create the queue pair; An interface specifying module, used for specifying the corresponding virtual interface for the queue pair; A mapping relationship establishing module, used to establish a mapping between the virtual interface and the virtual machine; A first mapping relationship recording module, used for recording a mapping relationship between the queue pair, the virtual interface and the virtual machine in a configuration file of the virtual machine; A configuration file loading module, used to load the configuration file through the virtual machine, so that the virtual machine determines the corresponding virtual interface and the queue pair; The second mapping relationship recording module is used to record the mapping relationship between the virtual interface and the queue pair in the memory mapping table.

[0103] Optionally, the device further comprises: A new queue pair creation module is used to create a corresponding new queue pair for the new virtual machine in the target network card when a new virtual machine is detected in the host; A newly added virtual interface specifying module is used to specify a corresponding newly added virtual interface for the newly added queue pair; A newly added mapping relationship establishing module is used to establish a mapping relationship between the newly added virtual interface and the newly added queue pair; A newly added mapping relationship recording module is used to record the mapping relationship between the newly added virtual interface and the newly added queue pair in the memory mapping table.

[0104] Optionally, the device further comprises: The memory release and information deletion module is used to release the memory occupied by the virtual machine and delete the virtual interface and the queue pair corresponding to the virtual machine when the virtual machine fails.

[0105] Based on the same inventive concept, another embodiment of the present application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the data acquisition method described in any of the above embodiments of the present application.

[0106] Based on the same inventive concept, another embodiment of the present application provides an electronic device, Figure 4 It is a schematic diagram of an electronic device 400 proposed in one embodiment of the present application, including a memory 402, a processor 401 and a computer program stored in the memory and executable on the processor, and the processor, when executed, implements the steps in the data acquisition method described in any of the above embodiments of the present application.

[0107] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0108] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0109] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0110] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0111] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0112] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0113] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0114] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0115] The data acquisition method, device, equipment and storage medium provided by the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data acquisition method, characterized in that: The method comprises: Receive data request through target network card; Determine the virtual interface and queue pair corresponding to the data request according to a pre-stored memory mapping table; Obtaining data corresponding to the data request from the virtual machine corresponding to the data request through the queue pair and the virtual interface; The data corresponding to the data request is sent to the requesting device.

2. The data acquisition method according to claim 1, characterized in that: Before determining the virtual interface and queue pair corresponding to the data request according to the pre-stored memory mapping table, the method further includes: Parsing the data request to determine the business corresponding to the data request; The virtual machine corresponding to the data request is determined according to a pre-stored correspondence relationship between the service and the virtual machine.

3. The data acquisition method according to claim 1, characterized in that: Determining the virtual interface and queue pair corresponding to the data request according to the pre-stored memory mapping table includes: In the memory mapping table, according to the virtual machine number of the virtual machine corresponding to the data request, searching for the corresponding virtual interface number and queue pair number; The virtual interface and the queue pair corresponding to the virtual interface number and the queue pair number are determined as the virtual interface and the queue pair corresponding to the data request.

4. The data acquisition method according to claim 1, characterized in that: Obtaining data corresponding to the data request from the virtual machine corresponding to the data request through the queue pair and the virtual interface includes: adding the request to a sending queue in the queue pair; Sending the data request from the sending queue to the virtual machine through the virtual interface; receiving, through the virtual interface, the data returned by the virtual machine according to the data request; The data is placed into a receive queue in the queue pair.

5. The data acquisition method according to claim 1, characterized in that: The sending the data corresponding to the data request to the requesting device includes: Parsing the data to determine the address of the requesting device corresponding to the data; Adding the data to a sending queue in the queue pair; The data is sent from the sending queue to the requesting device according to the address of the requesting device.

6. The data acquisition method according to claim 1, characterized in that: Before receiving the data request through the target network card, the method further includes: Creating a preset number of the virtual machines in the host; registering memory for the virtual machine; Initializing the virtual machine; Invoking a preset queue pair creation interface in the target network card to create the queue pair; specifying the corresponding virtual interface for the queue pair; Establishing a mapping between the virtual interface and the virtual machine; Recording a mapping relationship between the queue pair, the virtual interface and the virtual machine in a configuration file of the virtual machine; Loading the configuration file through the virtual machine so that the virtual machine determines the corresponding virtual interface and the queue pair; The mapping relationship between the virtual interface and the queue pair is recorded in the memory mapping table.

7. The data acquisition method according to claim 1, characterized in that: The method further comprises: When a newly added virtual machine is detected in the host, a corresponding newly added queue pair is created in the target network card for the newly added virtual machine; Specifying a corresponding new virtual interface for the new queue pair; Establishing a mapping relationship between the newly added virtual interface and the newly added queue pair; The mapping relationship between the newly added virtual interface and the newly added queue pair is recorded in the memory mapping table.

8. The data acquisition method according to claim 1, characterized in that: The method further comprises: When the virtual machine is detected to be invalid, the memory occupied by the virtual machine is released, and the virtual interface and the queue pair corresponding to the virtual machine are deleted.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images