Data transmission method and device, equipment and storage medium

Abstract

The invention discloses a data transmission method and device, equipment and a storage medium, and relates to the field of cloud platform storage, and the method comprises the steps: abstracting a nonvolatile memory in a target data transmission system into a corresponding block device, and mounting a preset virtual disk device constructed based on a virtualization technology and a small computer system interface to a target virtual machine, and when the target virtual machine receives a data transmission request, reading target data in the block device by utilizing a virtual host service of the preset software development suite, and then transmitting the target data to the target virtual machine by utilizing an interface of the preset virtual interface device, so that the target virtual machine stores the target data into a target disk of the target virtual machine. According to the data transmission method, the virtual host service is added in the target data transmission system, and the virtual disk equipment is mounted at the virtual machine end, so that the data can be directly read from the nonvolatile memory, a redundant link during data transmission is optimized, and the storage performance of the virtual machine is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of cloud platform storage, and in particular to a data transmission method, device, equipment and storage medium. Background Art

[0002] With the growing demand for data storage, NVMe SSDs (a high-performance solid-state drive (SSD) based on the NVMe (Non-Volatile Memory Express) protocol) are becoming a mainstream storage device due to their high performance. SPDK (Storage Performance Development Kit), an open-source toolkit developed by Intel, is a user-mode, asynchronous, polled, and lock-free NVMe driver that maximizes the performance advantages of NVMe SSDs, reduces NVMe command latency, and increases IOps (Input / Output Operations Per Second) per CPU (Central Processing Unit) core, creating a cost-effective solution.

[0003] Longhorn (an open-source, lightweight, cloud-native distributed block storage system) now supports SPDK in its latest version. However, in actual use, Longhorn's current utilization of SPDK is insufficient, resulting in excessively long intermediate links and thus impacting overall performance. Therefore, optimizing the use of Longhorn and SPDK to improve data transmission remains an unresolved issue in this field. Summary of the Invention

[0004] In view of this, the present invention aims to provide a data transmission method, apparatus, device, and storage medium. By adding a virtualization host service to the target data transmission system and mounting a virtual disk device on the virtual machine side, data can be directly read from non-volatile memory, optimizing redundant links during data transmission and effectively improving the storage performance of the virtual machine. The specific solution is as follows:

[0005] In a first aspect, the present application provides a data transmission method, comprising:

[0006] Abstracting the target memory in the target data transmission system into a corresponding block device, and mounting a preset virtual disk device to the target virtual machine; the preset virtual disk device is a device built based on virtualization technology and small computer system interface, and the target memory is a non-volatile memory;

[0007] When the target virtual machine receives the data transmission request, the target virtual machine uses the virtualization host service of the preset software development kit to read the target data in the block device;

[0008] The target data is transmitted to the target virtual machine by using the interface of the preset virtual interface device, so that the target virtual machine stores the target data in its own target disk.

[0009] Optionally, abstracting the target memory in the target data transmission system into a corresponding block device includes:

[0010] Obtaining an installation package of a preset development kit, and determining the software environment and hardware resource parameters of the target data transmission system; the preset development kit is a development kit built based on a distributed block storage system and a storage performance development kit;

[0011] Determine the network configuration, storage configuration, and interactive interface configuration corresponding to the installation package based on the software environment and the hardware resource parameters;

[0012] The preset development kit is deployed to the target data transmission system according to the network configuration, the storage configuration and the interactive interface configuration, and the target memory in the target data transmission system is abstracted into the corresponding block device using the preset development kit.

[0013] Optionally, after deploying the preset development kit to the target data transmission system according to the network configuration, the storage configuration, and the interactive interface configuration, the method further includes:

[0014] Using a preset performance testing tool to construct several storage demand scenarios and read / write demand scenarios, and testing the preset development kit based on each of the storage demand scenarios and the read / write demand scenarios to obtain corresponding test results;

[0015] The network configuration, the storage configuration, and the interactive interface configuration of the preset performance testing tool in the target data transmission system are adjusted based on the test results.

[0016] Optionally, before using the virtualization host service of the preset software development kit to read the target data in the block device, the method further includes:

[0017] determining a current load of each central processing unit core in the target data transmission system;

[0018] The central processing unit core whose current load is lower than a preset load threshold is selected as a target core, and the virtualization host service of the preset software development kit is deployed to the target core.

[0019] Optionally, before using the virtualization host service of the preset software development kit to read the target data in the block device, the method further includes:

[0020] Determining a virtual machine storage requirement corresponding to the target virtual machine and determining a memory capacity of the target data transmission system;

[0021] Based on the virtual machine storage requirements and the memory capacity, a target memory space corresponding to the virtualization host service of the preset software development kit is determined, so that the virtualization host service of the preset software development kit is deployed to the target data transmission system based on the target memory space.

[0022] Optionally, mounting the preset virtual disk device to the target virtual machine includes:

[0023] Determine the operating system type corresponding to the target virtual machine;

[0024] If the operating system type is a first operating system, executing a corresponding loading command through a preset command line tool according to the system version of the first operating system to load the preset virtual disk device into the target data transmission system; the first operating system is a system in which device support for the preset virtual disk device is pre-integrated in the system kernel;

[0025] If the operating system type is a second operating system, a driver download command is obtained, and based on the driver download command and the system version of the second operating system, the corresponding driver is downloaded and installed to use the driver installation program to install the preset virtual disk device into the target data transmission system; the second operating system is a system that does not integrate the device support of the preset virtual disk device in the system kernel.

[0026] Optionally, the process of mounting the preset virtual disk device to the target virtual machine further includes:

[0027] Querying a storage device list in the target data transmission system using a preset device query command, and determining a device name corresponding to the preset virtual disk device in the storage device list;

[0028] Using a preset disk partitioning tool to perform disk partitioning on the preset virtual disk device based on the device name;

[0029] The partitions corresponding to the preset virtual disk device are formatted using a preset system formatting tool, and the formatted preset virtual disk device is mounted to the target virtual machine.

[0030] In a second aspect, the present application provides a data transmission device, comprising:

[0031] A device mounting module is used to abstract the target memory in the target data transmission system into a corresponding block device and mount a preset virtual disk device to the target virtual machine; the preset virtual disk device is a device built based on virtualization technology and small computer system interface, and the target memory is a non-volatile memory;

[0032] a data reading module, configured to read target data from the block device using a virtualization host service of a preset software development kit when the target virtual machine receives a data transmission request;

[0033] The data transmission module is configured to transmit the target data to the target virtual machine by using the interface of the preset virtual interface device, so that the target virtual machine stores the target data in its own target disk.

[0034] In a third aspect, the present application provides an electronic device comprising a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the aforementioned data transmission method.

[0035] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, which implements the aforementioned data transmission method when executed by a processor.

[0036] This application first abstracts the non-volatile memory in the target data transmission system into a corresponding block device, and mounts a preset virtual disk device built based on virtualization technology and a small computer system interface to the target virtual machine. When the target virtual machine receives a data transmission request, it uses the virtualization host service of the preset software development kit to read the target data in the block device, and then uses the interface of the preset virtual interface device to transfer the target data to the target virtual machine, so that the target virtual machine can store the target data in its own target disk. In this way, by adding a virtualization host service to the target data transmission system and mounting a preset virtual disk device built based on virtualization technology and a small computer system interface on the virtual machine side, data can be read directly from the non-volatile memory and quickly transferred to the virtual machine. The entire process does not require multiple data transfers and processing in the kernel state, which optimizes the redundant links during data transmission and effectively improves the storage performance of the virtual machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0038] Figure 1 A flow chart of a data transmission method provided by this application;

[0039] Figure 2 A schematic diagram of the data transmission principle provided by this application;

[0040] Figure 3 A schematic diagram of the structure of a data transmission device provided in this application;

[0041] Figure 4 This is a structural diagram of an electronic device provided in this application. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0043] With the continuous growth of data storage demand, high-performance solid-state drives using the NVMe SSD protocol have gradually become mainstream storage devices due to their high performance. SPDK, as a user-mode, asynchronous, polling, and lock-free NVMe driver, can maximize the performance advantages of NVMe SSD, reduce NVMe command latency, and increase the IOps of a single CPU core. However, in actual use, Longhorn currently does not make sufficient use of SPDK, and the intermediate links are too long, which affects the overall performance. This application adds a virtualization host (SPDK vhost target) service to the target data transmission system and mounts a preset virtual disk device (vhost-scsi disk) on the virtual machine side. This allows data to be read directly from non-volatile memory, optimizes redundant links during data transmission, and effectively improves the storage performance of virtual machines.

[0044] See also Figure 1 As shown, an embodiment of the present invention discloses a data transmission method, including:

[0045] Step S11, abstract the target memory in the target data transmission system into a corresponding block device, and mount the preset virtual disk device to the target virtual machine; the preset virtual disk device is a device built based on virtualization technology and small computer system interface, and the target memory is a non-volatile memory.

[0046] It should be noted that this embodiment addresses the poor virtual machine performance issue in the existing Longhorn-SPDK solution by introducing the latest version of longhorn-spdk and adding the SPDKvhost target service to mount vhost-scsi disks on the virtual machine side. SPDK is an open source toolkit and library for developing high-performance storage applications. It aims to improve storage device performance and reduce latency by bypassing the traditional operating system storage stack and interacting directly with the underlying hardware. Vhost is a mechanism for achieving efficient I / O (Input / Output) in virtualized environments, defining a general framework for providing a high-performance I / O path between virtual machines (VMs) and underlying physical devices. Longhorn is an open source container-native distributed storage system designed specifically for the Kubernetes container orchestration platform, providing a simple, reliable, and high-performance storage solution for persistence within Kubernetes clusters.

[0047] Based on the above solution, in this embodiment, the target storage in the target data transmission system is first abstracted as a corresponding block device, and the preset virtual disk device (ie Figure 2 The vhost-scsi disk shown in the figure is mounted to the target virtual machine. The preset virtual disk device is a device built based on virtualization technology (vhost technology) and the Small Computer System Interface (SCSI), and the target memory is non-volatile memory (NVM). In addition, longhorn-spdk is a storage solution that is deeply integrated with Longhorn and SPDK (Storage Performance Development Kit). It aims to improve the performance of persistent storage in Kubernetes through SPDK's user-mode driver and zero-copy technology. It is mainly used to manage NVMe SSDs and create logical volumes (lvl). Figure 2The longhorn bdev shown is a customized block device implemented by Longhorn based on the bdev module (Block Device, a core concept in the SPDK framework, referring to the block device abstraction layer) of SPDK. It is used to connect the underlying physical storage (such as NVMe SSD) and the upper-level logical volume.

[0048] Specifically, to abstract the target storage in the target data transmission system into a corresponding block device, it is first necessary to introduce the latest version of longhorn-spdk, the preset development kit. First, the installation package of the preset development kit (longhorn-spdk) can be obtained, and the software environment and hardware resource parameters of the target data transmission system can be determined. Then, based on the software environment and hardware resource parameters, the network configuration, storage configuration, and interactive interface configuration corresponding to the installation package can be determined. Based on the network configuration, storage configuration, and interactive interface configuration, the preset development kit can be deployed to the target data transmission system. The preset development kit can then be used to abstract the target storage in the target data transmission system into a corresponding block device. The preset development kit is a development kit built based on the distributed block storage system (Longhorn) and the Storage Performance Development Kit (SPDK).

[0049] That is to say, this embodiment, under the existing product architecture, needs to introduce the latest version of longhorn-spdk for the current system. First, it is necessary to comprehensively sort out the software environment that the existing product relies on, including the operating system version, various basic libraries, and runtime environment. At the same time, a detailed evaluation of hardware resources, such as CPU model and number of cores, memory capacity and bandwidth, disk controller specifications, etc., is carried out. These hardware and software conditions are carefully compared with the compatibility list provided by the official longhorn-spdk new version to ensure the compatibility of the two. After that, the latest version of the longhorn-spdk installation package and the accompanying detailed documentation are obtained through officially designated safe and reliable channels, and preliminary deployment work is carried out in an independently built test environment in strict accordance with the official established installation process. During the deployment work, the network configuration link needs to reasonably set the network parameters of Longhorn-SPDK according to the network topology of the existing products, including IP address, subnet mask, gateway and DNS (Domain Name System) server address, etc., to ensure that it can seamlessly connect with the network environment of the existing products and smoothly carry out data communication; for the storage configuration part, the storage-related parameters of Longhorn-SPDK are accurately adjusted according to the type of existing storage devices (such as different specifications of mechanical hard drives and solid-state drives), the architecture of the storage array and the division of the storage pool to ensure that it can efficiently identify and utilize existing storage resources; in terms of the interactive interface with the core components of existing products, it is necessary to carefully study the interface specifications and calling mechanisms of existing products, and make targeted adaptation adjustments to Longhorn-SPDK to ensure the accuracy and stability of data interaction between the two.

[0050] Furthermore, after deploying Longhorn-SPDK to the target data transmission system, a pre-set performance testing tool can be used to construct several storage and read / write demand scenarios. The pre-set development kit can then be tested based on each storage and read / write demand scenario to obtain corresponding test results. Based on the test results, the pre-set performance testing tool's network configuration, storage configuration, and interactive interface configuration within the target data transmission system can then be adjusted. In a specific embodiment, after Longhorn-SPDK is deployed, a performance testing tool such as FIO (Flexible I / O Tester) is used to conduct preliminary performance testing of Longhorn-SPDK's basic functions. For example, when testing storage resource allocation, various storage demand scenarios of varying scales and types are simulated to verify Longhorn-SPDK's ability to quickly and accurately allocate and reclaim storage resources, and to evaluate resource utilization and allocation efficiency during the allocation process. For data read and write operation testing, different read and write modes (e.g., sequential read, sequential write, random read, random write), read and write block sizes, and concurrent request counts are set to examine Longhorn-SPDK's data read and write performance in various complex scenarios. Then, based on the detailed test results, we conducted an in-depth analysis of possible configuration deviations during the installation process. For example, improper network parameter settings caused excessive data transmission delays, unreasonable storage configuration caused read and write performance bottlenecks, or interface adaptation problems caused by data interaction errors, etc., and made precise corrections one by one to ensure that longhorn-spdk can run stably and efficiently in the existing product environment, laying a solid foundation for fully utilizing its latest features to deeply optimize storage performance in the future.

[0051] In this embodiment, a preset virtual disk device is mounted to a target virtual machine, and the operating system type corresponding to the target virtual machine can be specifically determined. If the operating system type is a first operating system supported by a device that has the preset virtual disk device pre-integrated in the system kernel, a corresponding loading command is executed through a preset command line tool based on the system version of the first operating system to load the preset virtual disk device into the target data transmission system. If the operating system type is a second operating system supported by a device that does not have the preset virtual disk device pre-integrated in the system kernel, a driver download command is obtained, and based on the driver download command and the system version of the second operating system, a corresponding driver is downloaded and installed, so that the preset virtual disk device is installed into the target data transmission system using a driver installation program.

[0052] That is to say, in the present embodiment, it is necessary to mount the vhost-scsi disk in advance at the virtual machine operating system level. Specifically, it is first necessary to confirm the operating system type and version used by the virtual machine. It is understandable that there are certain differences in the support mode and driver installation process of different operating systems for the vhost-scsi device. In a specific embodiment, for mainstream Linux operating systems, such as Ubuntu, CentOS, etc., basic support for vhost-scsi devices has usually been integrated in the system kernel, but it may be necessary to load and configure the kernel module according to the system version situation. Therefore, in the console of the virtual machine, corresponding kernel module loading commands such as "modprobe vhost_scsi" can be executed by command line tools to ensure that the vhost-scsi kernel module can be successfully loaded into the system. After loading is completed, the system log file is checked, such as " / var / log / syslog", to check whether there is relevant information output about the vhost-scsi device to confirm whether the module loading process is normal. In another specific embodiment, for the Windows Server operating system, since the vhost-scsi device driver is not integrated by default, it is necessary to manually download and install the corresponding driver. The vhost-scsi driver installation package that matches the virtual machine operating system version is obtained from the official website of a reliable driver provider or the official website of the virtual machine management software manufacturer. Then, the driver installation program is run in the virtual machine. Follow the prompts of the installation wizard to complete the driver installation process step by step. After the installation is complete, check whether the vhost-scsi device appears under the "Storage Controller" category in the device manager of the virtual machine. If it appears, it indicates that the driver installation is successful.

[0053] Afterwards, you can also use the preset device query command to query the list of storage devices in the target data transmission system, and determine the device name corresponding to the preset virtual disk device in the storage device list, and then use the preset disk partitioning tool to partition the preset virtual disk device based on the device name, and use the preset system formatting tool to format each partition corresponding to the preset virtual disk device, and then mount the formatted preset virtual disk device to the target virtual machine. In other words, after completing the driver installation or module loading, you need to configure the virtual machine so that it can recognize and mount the vhost-scsi disk. For example: in the Linux system, use commands such as "fdisk-l" to view the list of recognized storage devices in the system, confirm the device name of the vhost-scsi disk, such as " / dev / sdb", and then use partitioning tools such as "parted" or "fdisk" to partition the vhost-scsi disk according to actual storage needs. Specifically, if you plan to partition the disk into a primary partition to store important virtual machine data files, you can use the "parted / dev / sdb mklabel msdos" command to create an MBR (Master Boot Record) partition table. Then, use the "parted / dev / sdbmkpart primary 0% 100%" command to create a primary partition that occupies the entire disk space. After partitioning, you can use a file system formatting tool, such as "mkfs.ext4 / dev / sdb1" (assuming the primary partition you created is " / dev / sdb1"), to format the partition with the ext4 file system suitable for Linux systems. In Windows Server system, open the "Disk Management" tool, find the newly identified vhost-scsi disk in the disk list, initialize the disk according to the wizard prompts, select the appropriate partition style (MBR or GPT (GUID Partition Table, Globally Unique Identifier Partition Table)), after the initialization is complete, create and format the disk partition, and in the "New Simple Volume" wizard, set the partition size, assign a drive letter, and select the file system type (such as NTFS, New Technology File System) and other parameters to complete the preparation for mounting the vhost-scsi disk. In this way, through the above series of configuration operations on the virtual machine side, such as Figure 2 As shown in the figure, the virtual machine can successfully receive NVMe SSD storage resources delivered by the SPDK vhost target service, effectively achieving a significant improvement in storage performance.

[0054] Step S12: When the target virtual machine receives the data transmission request, it uses the virtualization host service of the preset software development kit to read the target data in the block device.

[0055] In this embodiment, when a target virtual machine receives a data transmission request, it can utilize the virtualization host service of the preset software development kit (SPDK vhost target service) to read the target data from the block device. Furthermore, before reading the target data from the block device, the current load of each CPU core in the target data transmission system can be determined, and CPU cores with current loads below a preset load threshold can be selected as target cores, to which the virtualization host service of the preset software development kit is deployed. Accordingly, the virtual machine storage requirements corresponding to the target virtual machine and the memory capacity of the target data transmission system can be determined. Based on the virtual machine storage requirements and memory capacity, the target memory space corresponding to the virtualization host service of the preset software development kit can then be determined, so that the virtualization host service of the preset software development kit can be deployed to the target data transmission system based on the target memory space.

[0056] In other words, this embodiment requires adding the SPDK vhost target service within the system kernel and user space architecture. First, a thorough analysis of the system's existing I / O scheduling mechanism and device driver architecture is performed to clarify the SPDK vhost target service's position and role in the overall system I / O process, providing a theoretical basis for subsequent service integration and optimization. Specifically, the SPDK vhost target service source code can be obtained from the official SPDK open-source code repository. The source code is then compiled and adapted specifically for the system's hardware platform (e.g., x86 architecture, ARM architecture, etc.) and operating system type (e.g., Linux, Windows Server, etc.). During the compilation process, compilation parameters are appropriately adjusted, enabling optimization options that match the system's hardware characteristics, such as enabling instruction optimization for CPUs supporting the AVX instruction set, to improve service efficiency. After compilation is complete, the SPDK vhost target service is deployed to the system. During deployment, the service's operating parameters can be fine-tuned based on the system's resource usage and the performance requirements of the virtual machine. For example, according to the load status of the system CPU, the thread binding policy of the SPDK vhost target service is reasonably set to preferentially bind the service thread to the idle or lightly loaded CPU core to avoid competing for CPU resources with other critical system processes, thereby ensuring that the service can run stably and efficiently; for memory allocation parameters, sufficient and reasonable memory space is allocated to the SPDK vhosttarget service based on the expected scale of NVMe SSD storage resources required by the virtual machine and the overall memory capacity of the system to ensure that its performance will not degrade or service will not be interrupted due to insufficient memory during data transmission and processing.

[0057] In this way, the SPDK vhost target service can transparently pass the NVMe SSD to the Qemu virtual machine with minimal performance loss. Its operation is based on a user-mode driver model, bypassing the complex context switching and system call overhead of traditional kernel-mode I / O drivers. During the data transmission process, the SPDK vhost target service directly interacts with the NVMe SSD, greatly reducing data transmission latency through an efficient asynchronous I / O mechanism and a polling-based device access strategy. At the same time, the use of zero-copy technology avoids multiple data copy operations in memory, further improving data transmission efficiency. For example, when the Qemu virtual machine initiates a read request, the SPDK vhost target service can quickly capture the request and read the data directly from the NVMe SSD. Through the optimized memory mapping mechanism, the data is quickly passed to the Qemu virtual machine. The entire process does not require multiple kernel-mode data transfers and processing, thus achieving the transparent transmission of storage resources from the NVMe SSD to the Qemu virtual machine with extremely low performance loss.

[0058] Step S13: Utilize the interface of the preset virtual interface device to transmit the target data to the target virtual machine, so that the target virtual machine stores the target data in its own target disk.

[0059] In this embodiment, the target data can be transmitted to the target virtual machine by using the interface of the preset virtual interface device, so that the target virtual machine can store the target data in its own target disk.

[0060] Based on the above steps, this embodiment can also carry out comprehensive redundant link optimization work at the storage service level and the virtual machine level. For example, at the storage service level, professional network topology analysis tools and storage architecture monitoring software are used to conduct detailed sorting and analysis of the current data transmission links. First, a complete data transmission path diagram is drawn, covering the entire process from storage devices (such as NVMe SSD arrays) to storage servers, to network switches, and finally to the computing nodes where the virtual machines are located. In this process, the role and impact of each network connection device (such as routers, firewalls, etc.) and storage middleware (such as storage gateways, distributed storage software, etc.) in the data transmission process are checked. Afterwards, various performance indicators of the data transmission process are collected through monitoring tools, such as network bandwidth utilization, data transmission delay, packet loss rate, etc., and the collected data is deeply analyzed to find possible redundant links or performance bottlenecks. For example, if it is found that the bandwidth utilization of a network switch port is in a saturated state for a long time and there is a large amount of packet loss, after further analysis, it is confirmed that the link where the port is located is a redundant link. You can consider optimizing and adjusting the link. The optimization measures include but are not limited to replanning the network topology and migrating part of the data traffic to other idle or low-load links; optimizing the configuration of the network switch, such as adjusting parameters such as port rate and buffer size to improve the transmission efficiency of the link; or replacing higher-performance network equipment to improve the carrying capacity of the entire network link. At the same time, at the VM level, we conduct in-depth research on the I / O processing flow within the VM and its interaction with storage devices. Specifically, we can use the performance monitoring function provided by the VM management software to collect various performance data of the VM when performing storage I / O operations, such as I / O request queue depth and I / O response time. We can then analyze this data to identify possible redundant I / O processing links or unreasonable resource allocation strategies within the VM. For example, if we find that when a VM processes storage I / O requests, multiple unnecessary intermediate software layers repeatedly process the requests, resulting in excessively long I / O response times, we can optimize the VM's software configuration to reduce or merge these intermediate processing links. At the same time, we can rationally adjust the VM's internal I / O resource allocation strategy based on the VM's actual storage needs and system resource availability, such as optimizing the I / O scheduling algorithm, to ensure that storage I / O requests are processed promptly and efficiently. In this way, by comprehensively optimizing redundant links at the storage service level and the VM level, we can minimize unnecessary intermediate links, significantly improve VM storage performance, and provide users with a more efficient and stable VM storage service experience.

[0061] Through the above technical solution, this embodiment introduces the latest version of longhorn-spdk into the system, adds the SPDK vhost target service, and mounts the vhost-scsi disk on the virtual machine side, solving the current problem of insufficient optimization of the virtual machine qume layer. By optimizing the redundant link, the storage data transmission link is shortened, and the storage performance of the virtual machine is effectively improved. In addition, the vhost-scsi disk mounted on the virtual machine side enables the virtual machine to efficiently utilize the performance advantages of NVMe SSD, further improving the overall storage performance and providing users with a more efficient virtual machine storage experience.

[0062] See also Figure 3 As shown, the embodiment of the present application further discloses a data transmission device, including:

[0063] The device mounting module 11 is used to abstract the target memory in the target data transmission system into a corresponding block device and mount the preset virtual disk device to the target virtual machine; the preset virtual disk device is a device built based on virtualization technology and small computer system interface, and the target memory is a non-volatile memory;

[0064] A data reading module 12 is configured to read target data from the block device using a virtualization host service of a preset software development kit when the target virtual machine receives a data transmission request;

[0065] The data transmission module 13 is configured to transmit the target data to the target virtual machine by using the interface of the preset virtual interface device, so that the target virtual machine stores the target data in its own target disk.

[0066] In this embodiment, the non-volatile memory in the target data transmission system can be abstracted as a corresponding block device, and a preset virtual disk device constructed based on virtualization technology and a small computer system interface can be mounted to the target virtual machine. When the target virtual machine receives a data transmission request, it uses the virtualization host service of the preset software development kit to read the target data in the block device, and then uses the interface of the preset virtual interface device to transmit the target data to the target virtual machine, so that the target virtual machine can store the target data in its own target disk. Through the above technical solution, by adding a virtualization host service to the target data transmission system and mounting a preset virtual disk device constructed based on virtualization technology and a small computer system interface on the virtual machine side, data can be read directly from the non-volatile memory and quickly transferred to the virtual machine. The entire process does not require multiple data transfers and processing in the kernel state, which optimizes the redundant links during data transmission and effectively improves the storage performance of the virtual machine.

[0067] In some specific embodiments, the device mounting module 11 specifically includes:

[0068] a parameter determination unit, configured to obtain an installation package of a preset development kit and determine the software environment and hardware resource parameters of the target data transmission system; the preset development kit is a development kit constructed based on a distributed block storage system and a storage performance development kit;

[0069] a configuration determining unit, configured to determine a network configuration, a storage configuration, and an interactive interface configuration corresponding to the installation package based on the software environment and the hardware resource parameters;

[0070] A kit deployment unit is used to deploy the preset development kit to the target data transmission system according to the network configuration, the storage configuration and the interactive interface configuration, and use the preset development kit to abstract the target memory in the target data transmission system into the corresponding block device.

[0071] In some specific embodiments, the device mounting module 11 further includes:

[0072] A suite testing unit is used to construct a plurality of storage demand scenarios and read / write demand scenarios using a preset performance testing tool, and test the preset development suite based on each of the storage demand scenarios and the read / write demand scenarios to obtain corresponding test results;

[0073] A configuration adjustment unit is used to adjust the network configuration, the storage configuration and the interactive interface configuration of the preset performance testing tool in the target data transmission system based on the test result.

[0074] In some specific embodiments, the data transmission device further includes:

[0075] a load determination module, configured to determine a current load of each central processing unit core in the target data transmission system;

[0076] The service deployment module is used to select the central processing unit core whose current load is lower than a preset load threshold as a target core, and deploy the virtualization host service of the preset software development kit to the target core.

[0077] In some specific embodiments, the data transmission device further includes:

[0078] a memory determination module, configured to determine a virtual machine storage requirement corresponding to the target virtual machine and determine a memory capacity of the target data transmission system;

[0079] A space determination module is used to determine the target memory space corresponding to the virtualization host service of the preset software development kit based on the virtual machine storage requirements and the memory capacity, so as to deploy the virtualization host service of the preset software development kit to the target data transmission system based on the target memory space.

[0080] In some specific embodiments, the device mounting module 11 specifically includes:

[0081] A type determining unit, configured to determine an operating system type corresponding to the target virtual machine;

[0082] a device loading unit configured to, if the operating system type is a first operating system, execute a corresponding loading command through a preset command line tool according to the system version of the first operating system to load the preset virtual disk device into the target data transmission system; the first operating system is a system in which device support for the preset virtual disk device is pre-integrated in a system kernel;

[0083] A device installation unit is used to obtain a driver download command if the operating system type is a second operating system, and download and install a corresponding driver based on the driver download command according to the system version of the second operating system, so as to use the driver installation program to install the preset virtual disk device into the target data transmission system; the second operating system is a system in which the device support of the preset virtual disk device is not integrated into the system kernel.

[0084] In some specific embodiments, the device mounting module 11 further includes:

[0085] a name query unit, configured to query a storage device list in the target data transmission system using a preset device query command, and determine a device name corresponding to the preset virtual disk device in the storage device list;

[0086] A disk partitioning unit, configured to perform disk partitioning on the preset virtual disk device based on the device name using a preset disk partitioning tool;

[0087] The device mounting unit is configured to format each partition corresponding to the preset virtual disk device using a preset system formatting tool, and mount the formatted preset virtual disk device to the target virtual machine.

[0088] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram should not be considered as any limitation to the scope of application of the present application.

[0089] Figure 4This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the data transmission method disclosed in any of the aforementioned embodiments. Furthermore, the electronic device 20 in this embodiment may be a computer.

[0090] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0091] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0092] The operating system 221 is used to manage and control the hardware devices on the electronic device 20 and the computer program 222, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including computer programs that can be used to implement the data transmission method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs that can be used to perform other specific tasks.

[0093] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned data transmission method is implemented. The specific steps of this method can be referred to the corresponding contents disclosed in the aforementioned embodiments and will not be repeated here.

[0094] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0095] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0096] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0097] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0098] The above is a detailed introduction to the technical solution provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data transmission method, characterized in that: include: Abstract the target storage in the target data transmission system into a corresponding block device, and mount the preset virtual disk device to the target virtual machine; The preset virtual disk device is a device built based on virtualization technology and small computer system interface, and the target memory is a non-volatile memory; When the target virtual machine receives the data transmission request, the target virtual machine uses the virtualization host service of the preset software development kit to read the target data in the block device; The target data is transmitted to the target virtual machine by using the interface of the preset virtual interface device, so that the target virtual machine stores the target data in its own target disk.

2. The data transmission method according to claim 1, wherein: The step of abstracting the target memory in the target data transmission system into a corresponding block device includes: Obtaining an installation package of a preset development kit, and determining the software environment and hardware resource parameters of the target data transmission system; the preset development kit is a development kit built based on a distributed block storage system and a storage performance development kit; Determine the network configuration, storage configuration, and interactive interface configuration corresponding to the installation package based on the software environment and the hardware resource parameters; The preset development kit is deployed to the target data transmission system according to the network configuration, the storage configuration and the interactive interface configuration, and the target memory in the target data transmission system is abstracted into the corresponding block device using the preset development kit.

3. The data transmission method according to claim 2, wherein: After deploying the preset development kit to the target data transmission system according to the network configuration, the storage configuration, and the interactive interface configuration, the method further includes: Using a preset performance testing tool to construct several storage demand scenarios and read / write demand scenarios, and testing the preset development kit based on each of the storage demand scenarios and the read / write demand scenarios to obtain corresponding test results; The network configuration, the storage configuration, and the interactive interface configuration of the preset performance testing tool in the target data transmission system are adjusted based on the test results.

4. The data transmission method according to claim 1, wherein: Before reading the target data in the block device using the virtualized host service of the preset software development kit, the method further includes: determining a current load of each central processing unit core in the target data transmission system; The central processing unit core whose current load is lower than a preset load threshold is selected as a target core, and the virtualization host service of the preset software development kit is deployed to the target core.

5. The data transmission method according to claim 1, wherein: Before reading the target data in the block device using the virtualized host service of the preset software development kit, the method further includes: Determining a virtual machine storage requirement corresponding to the target virtual machine and determining a memory capacity of the target data transmission system; Based on the virtual machine storage requirements and the memory capacity, a target memory space corresponding to the virtualization host service of the preset software development kit is determined, so that the virtualization host service of the preset software development kit is deployed to the target data transmission system based on the target memory space.

6. The data transmission method according to any one of claims 1 to 5, characterized in that: Mounting the preset virtual disk device to the target virtual machine includes: Determine the operating system type corresponding to the target virtual machine; If the operating system type is a first operating system, executing a corresponding loading command through a preset command line tool according to the system version of the first operating system to load the preset virtual disk device into the target data transmission system; the first operating system is a system in which device support for the preset virtual disk device is pre-integrated in the system kernel; If the operating system type is a second operating system, a driver download command is obtained, and based on the driver download command and the system version of the second operating system, the corresponding driver is downloaded and installed to use the driver installation program to install the preset virtual disk device into the target data transmission system; the second operating system is a system that does not integrate the device support of the preset virtual disk device in the system kernel.

7. The data transmission method according to claim 6, characterized in that: The process of mounting the preset virtual disk device to the target virtual machine also includes: Querying a storage device list in the target data transmission system using a preset device query command, and determining a device name corresponding to the preset virtual disk device in the storage device list; Using a preset disk partitioning tool to perform disk partitioning on the preset virtual disk device based on the device name; The partitions corresponding to the preset virtual disk device are formatted using a preset system formatting tool, and the formatted preset virtual disk device is mounted to the target virtual machine.

8. A data transmission device, characterized in that: include: The device mounting module is used to abstract the target storage in the target data transmission system into a corresponding block device and mount the preset virtual disk device to the target virtual machine; The preset virtual disk device is a device built based on virtualization technology and small computer system interface, and the target memory is a non-volatile memory; a data reading module, configured to read target data from the block device using a virtualization host service of a preset software development kit when the target virtual machine receives a data transmission request; The data transmission module is configured to transmit the target data to the target virtual machine by using the interface of the preset virtual interface device, so that the target virtual machine stores the target data in its own target disk.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the data transmission method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Used to store a computer program, which, when executed by a processor, implements the data transmission method according to any one of claims 1 to 7.

Images