High-throughput file transmission test method and system and electronic equipment

Abstract

The invention provides a high-throughput file transmission testing method and system and electronic equipment, and belongs to the technical field of data transmission testing. The method comprises the steps that an operating system is configured in a virtual machine; the virtual machine is deployed in a virtual disk of the computing device; creating a virtual network card for data communication based on a protocol stack of an operating system; loading a to-be-tested high-throughput file transmission system in the virtual machine, and enabling the high-throughput file transmission system to execute a file transmission process based on the virtual disk and the virtual network card; when the file transmission process is executed, operation data of the high-throughput file transmission system are collected, and the operation data are used for representing the transmission performance of the high-throughput file transmission system. According to the method, the full-virtual test environment based on the virtual disk and the virtual network card is constructed, and the efficiency and the stability of high-throughput file transmission test are improved under the condition of not depending on physical hardware.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission testing, and in particular to a high-throughput file transmission testing method, system and electronic equipment. Background Art

[0002] With the development of big data and high-performance computing technologies, high-throughput file transfer systems for large-scale data exchange have gained widespread application in fields such as scientific research, finance, and industry. To ensure system reliability and performance before launch, it is often necessary to simulate and verify the system's operating behavior under heavy loads, long links, and complex read and write conditions in a test environment. Traditional testing methods typically rely on hardware resources such as dedicated test servers, mechanical hard drive arrays, and high-speed physical network cards. These physical devices are used to construct high-bandwidth, high-concurrency test paths, which are then used to evaluate the transmission system's processing capabilities and performance indicators.

[0003] However, these existing solutions suffer from high deployment costs, poor resource flexibility, and low scalability. In hardware-constrained or cost-sensitive scenarios, they struggle to meet the demands of rapidly building a test environment and conducting frequent iterative verification. Furthermore, the testing process is highly dependent on physical hardware performance and susceptible to uncontrollable factors such as disk I / O bottlenecks and network card interrupt processing delays, resulting in low test efficiency and unstable results.

[0004] Therefore, how to solve the problems of low efficiency and poor stability of high-throughput file transfer testing without relying on physical hardware has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The present invention provides a high-throughput file transfer test method, system and electronic device to address the defects in the prior art and improve the efficiency and stability of high-throughput file transfer testing without relying on physical hardware.

[0006] The present invention provides a high-throughput file transfer testing method, comprising the following steps: Configuring an operating system in a virtual machine; the virtual machine is deployed in a virtual disk of a computing device; Creating a virtual network card for data communication based on the protocol stack of the operating system; Loading a high-throughput file transfer system to be tested in the virtual machine, and causing the high-throughput file transfer system to perform a file transfer process based on the virtual disk and the virtual network card; While executing the file transfer process, operation data of the high-throughput file transfer system is collected, and the operation data is used to characterize the transmission performance of the high-throughput file transfer system.

[0007] According to a high-throughput file transfer testing method provided by the present invention, before configuring the operating system in the virtual machine, the method further includes: In a computing device having a multi-core processor and a large-capacity physical memory, a virtual disk for data storage is constructed, wherein the virtual disk is created based on the physical memory; A virtual machine is deployed in the virtual disk, and the entire capacity of the virtual disk is allocated to the virtual machine.

[0008] According to a high-throughput file transfer testing method provided by the present invention, creating a virtual network card for data communication based on the protocol stack of the operating system includes: Invoking a virtual network interface creation instruction provided by the protocol stack within the operating system to generate the virtual network card in a software manner; The virtual network card is used to perform data communication between the virtual machine and the host machine on which the virtual machine runs, and the data communication is performed through the protocol stack without passing through the physical network card in the computing device.

[0009] According to a high-throughput file transfer test method provided by the present invention, the high-throughput file transfer system to be tested is loaded into the virtual machine, and the high-throughput file transfer system is caused to perform a file transfer process based on the virtual disk and the virtual network card, including: In the operating system deployed on the virtual disk, installing and starting the high-throughput file transfer system in a user-mode process; Reading the file data to be sent from the operating system through the virtual disk, and sending the file data to be sent to an external node in communication with the virtual network card through the virtual network card; The to-be-received file data is received from the external node through the virtual network card, and the to-be-received file data is written into the operating system through the virtual disk.

[0010] A high-throughput file transfer testing method provided by the present invention further includes: Reading the to-be-sent file data from the virtual disk in parallel using at least one reading thread, and sending the to-be-sent file data read by the reading thread in parallel through the virtual network card using at least one sending thread; The to-be-received file data is received in parallel from the external node through the virtual network card by at least one receiving thread, and the to-be-received file data received by the receiving thread is written in parallel to the virtual disk by at least one writing thread.

[0011] According to a high-throughput file transfer test method provided by the present invention, while executing the file transfer process, collecting the operating data of the high-throughput file transfer system, including: Deploying a performance monitoring module in the virtual machine for monitoring performance indicators of the high-throughput file transfer system; During the file transmission process, the performance monitoring module periodically collects the operating parameters of the high-throughput file transmission system at preset time intervals.

[0012] According to a high-throughput file transfer testing method provided by the present invention, the operating parameters include one or more of the data throughput, file transfer rate, and transmission delay of the high-throughput file transfer system; after collecting the operating data of the high-throughput file transfer system, the method further includes: Determining statistical indicators of the high-throughput file transfer system during the file transfer process based on one or more of the data throughput, the file transfer rate, and the transmission delay; the statistical indicators including one or more of an average value, a maximum value, or a minimum value of the data throughput, the file transfer rate, and the transmission delay; Based on all the statistical indicators, a test performance report is generated to characterize the performance of the high-throughput file transfer system.

[0013] According to a high-throughput file transfer testing method provided by the present invention, configuring an operating system in a virtual machine includes: Dividing a system partition for installing an operating system on the virtual disk, and storing a pre-prepared operating system image file in the system partition; When the virtual machine is started, the virtual machine automatically loads and starts the operating system image file based on the system partition of the virtual disk to complete the operating system initialization of the virtual machine.

[0014] The present invention also provides a high-throughput file transfer test system, comprising the following modules: A first configuration module is configured to configure an operating system in a virtual machine deployed in a virtual disk of a computing device; A second configuration module is used to create a virtual network card for data communication based on the protocol stack of the operating system; A first processing module is configured to load the high-throughput file transfer system to be tested into the virtual machine, and enable the high-throughput file transfer system to perform a file transfer process based on the virtual disk and the virtual network card; The second processing module is used to collect operating data of the high-throughput file transmission system while executing the file transmission process, and the operating data is used to characterize the transmission performance of the high-throughput file transmission system.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the high-throughput file transfer test method described above is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the high-throughput file transfer test method described above is implemented.

[0017] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the high-throughput file transfer test methods described above.

[0018] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: By configuring the operating system in a virtual machine, which is deployed on the virtual disk of the computing device, the loading, running, and resource scheduling of the operating system are all completed on the high-throughput, low-latency memory-level disk, providing a stable foundation for the subsequent deployment of the high-throughput transmission system. Based on the protocol stack of the operating system, a virtual network card is created for data communication, thereby achieving the construction of the data transmission path without relying on the physical network card. By loading the high-throughput file transfer system to be tested in the virtual machine and having the high-throughput file transfer system execute the file transfer process based on the virtual disk and virtual network card, the real data reading, writing, and transmission process is completed between the high-performance memory disk and the software network channel. By collecting the operating data of the high-throughput file transfer system while executing the file transfer process, the operating data is used to characterize the transmission performance of the high-throughput file transfer system, thereby improving the efficiency and stability of the high-throughput file transfer test without relying on physical hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 This is one of the flow charts of the high-throughput file transfer test method provided by the present invention.

[0021] Figure 2 This is the second flow chart of the high-throughput file transfer testing method provided by the present invention.

[0022] Figure 3 This is the third flow chart of the high-throughput file transfer testing method provided by the present invention.

[0023] Figure 4 This is the fourth flow chart of the high-throughput file transfer testing method provided by the present invention.

[0024] Figure 5 This is the fifth flow chart of the high-throughput file transfer testing method provided by the present invention.

[0025] Figure 6 This is the sixth flow chart of the high-throughput file transfer testing method provided by the present invention.

[0026] Figure 7 This is the seventh flow chart of the high-throughput file transfer test method provided by the present invention.

[0027] Figure 8 It is a structural diagram of the high-throughput file transfer test system provided by the present invention.

[0028] Figure 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0029] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field without making creative efforts based on the embodiments of the present invention are within the scope of protection of the present invention.

[0030] It should be noted that, in the description of the present invention, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. The orientation or positional relationship indicated by the terms "upper" and "lower" is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the system or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0031] The terms "first," "second," and so forth, used herein are used to distinguish similar objects, not to describe a specific order or precedence. It should be understood that such terms are interchangeable where appropriate, allowing embodiments of the present invention to be implemented in an order other than that illustrated or described herein. Furthermore, the terms "first," "second," and so forth generally distinguish objects of a single type, and do not limit the number of objects. For example, the first object may be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the connected objects.

[0032] The following combination Figures 1-9 The present invention describes a high-throughput file transfer test method, system, and electronic device.

[0033] Figure 1 This is one of the flow charts of the high-throughput file transfer test method provided by the present invention, such as Figure 1 As shown, including but not limited to the following steps: Step 101: Configure an operating system in a virtual machine; the virtual machine is deployed in a virtual disk of a computing device.

[0034] In this example, to achieve high-performance testing of a high-throughput file transfer system, the operating system must first be configured within the virtual machine. This process, as the foundation of the test environment, aims to provide a complete and stable operating platform for the subsequent deployment and operation of the file transfer system. As a well-isolated operating container, the correct configuration of the operating system within the virtual machine is key to ensuring the functional completeness and controllability of the test system.

[0035] Specifically, "configuring an operating system in a virtual machine" should be understood as the process of booting, initializing, and establishing the operating system's runtime environment. This process aims to enable the virtual machine to properly load the target operating system upon startup and provide basic computing resource support, including the kernel, drivers, and system services. This configuration process can be achieved through mechanisms such as loading the operating system image file, initializing kernel parameters, and mounting the root file system. The specific implementation details can be flexibly adjusted based on different virtualization platforms (such as KVM, VMware, or VirtualBox) and operating system types.

[0036] A computing device is a general-purpose server or high-performance personal computer with multi-core processing capabilities and large physical memory. As a host platform, it undertakes a series of basic support tasks, including virtualization operations, virtual disk mounting, and virtual machine management. Using virtualization technology, this computing device supports the simultaneous operation of multiple virtual machines, providing excellent resource isolation and stability.

[0037] A virtual disk is a simulated storage area allocated within the physical memory of the computing device, serving as the primary storage medium for a virtual machine. In this embodiment, this virtual disk is used to store operating system image files and data resources required by high-throughput file transfer systems. Virtual disks offer extremely high read and write speeds, effectively reducing the impact of storage I / O bottlenecks on test results. They are particularly suitable for scenarios involving high-frequency, high-volume file read and write operations.

[0038] Therefore, step 101, as the core initialization operation in the test method, not only lays the foundation for the execution of the test environment, but also the technical feature it defines, "configuring the operating system in the virtual machine", should be understood as: establishing an environment in the virtual machine instance that can normally run the operating system. The environment has protocol stack support capabilities, user-mode program running capabilities, and network communication support capabilities, and serves the subsequent loading and operation of the high-throughput file transfer system.

[0039] Step 102: Create a virtual network card for data communication based on the protocol stack of the operating system.

[0040] In this embodiment, to effectively test the high-throughput file transfer system, it is necessary to establish a network interface with data communication capabilities within the operating system. Specifically, after configuring the operating system within the virtual machine, a virtual network card (NIC) for data communication is created based on the operating system's protocol stack. This step, a key preparatory step in the testing method, aims to provide a reliable network channel for subsequent file transfers, enabling the high-throughput file transfer system to simulate the sending and receiving behavior of a real-world network environment while maintaining the flexibility and isolation of the test environment.

[0041] In this application, a protocol stack refers to the set of software components within an operating system that implement network communication functions. These components typically include modules implementing the TCP / IP protocol suite, which support the exchange of data packets between user-mode programs and network interfaces. The proper operation of the protocol stack is essential for communication between the virtual network card (NIC) and the application process. In a virtualized environment, the protocol stack remains independent within the virtual machine's operating system and does not rely on the host system's network stack. This architecture ensures complete simulation of the communication link and controllable testing.

[0042] In this application, a virtual network card refers to a network device interface created by software. Its essence is a virtualized network device within the operating system. It has the functions of a standard network card, can receive data packets from applications and transmit them through the protocol stack, and can also receive packets from the network and transmit them to applications. Unlike traditional network cards that rely on physical hardware, virtual network cards are not directly bound to any physical network device. Their data path exists entirely in memory, with advantages such as low latency and high controllability. After creation, the virtual network card can be bound to the network namespace or network bridge interface of the virtual machine, thereby enabling data interaction between processes within the virtual machine and external nodes.

[0043] In a specific implementation process, based on the protocol stack of the operating system, the virtual network card can be generated in software through a virtual network interface creation instruction provided by the kernel. For example, in a Linux system, a creation command can be initiated through the TUN / TAP interface in conjunction with the iproute2 tool or the kernel API to achieve dynamic generation of a virtual network card. After the virtual network card is configured, one end is connected to the process where the user-mode application is located, and the other end is forwarded to the network interface connected to the host machine through the protocol stack, or used to simulate an end-to-end data transmission path within the virtual machine.

[0044] It's important to emphasize that the virtual network card (NIC) doesn't rely on the physical network card in the computing device during data communication. That is, all packet generation, processing, and forwarding occur within the virtual environment. This communication mechanism not only avoids the performance bottlenecks of traditional physical network devices but also provides greater flexibility in the testing process, enabling rapid replication of various network topologies, bandwidth limitations, or abnormal conditions. This facilitates a comprehensive assessment of the stability and performance of high-throughput file transfer systems in various communication scenarios.

[0045] By creating a virtual network card based on the operating system protocol stack, this step effectively establishes a network data channel in the test system, achieving accurate simulation and monitoring of the target system's transmission behavior under conditions where physical network resources are limited or uncontrollable.

[0046] Step 103: Load the high-throughput file transfer system to be tested into the virtual machine, and enable the high-throughput file transfer system to perform a file transfer process based on the virtual disk and the virtual network card.

[0047] In this embodiment, to evaluate the functional correctness and performance of a high-throughput file transfer system in a specific environment, the system under test is loaded into a virtual machine (VM) with an operating system configured and a virtual network interface card (VNIC) created. The system then executes a file transfer process based on the virtual disk and VNIC. This step is the core of the entire testing method, directly impacting whether the system under test can fully operate and effectively transfer data within the constructed virtual test environment. Its implementation is crucial to the accuracy and repeatability of the test.

[0048] In this application, loading a high-throughput file transfer system means deploying and starting the system within a virtual machine operating system, allowing it to run as an independent application process in user space. A high-throughput file transfer system is an application-level program with parallel processing capabilities, support for high-speed network data exchange, and high-concurrency file reading and writing. Its design typically includes data scheduling threads, file I / O interfaces, and network communication modules, and is suitable for the rapid synchronization or replication of large-scale data between different nodes.

[0049] The virtual disk, as a data storage path, handles all read and write operations for both sent and received data during file transfers. The virtual network card, as a communication interface, handles the actual transmission of data packets. Together, they form the I / O channel for high-throughput file transfer systems and are critical foundational resources for ensuring smooth data links and stable performance.

[0050] During the specific implementation process, the high-throughput file transfer system to be tested is installed in the virtual machine operating system in user mode, and after startup, the local file system interface is called to read the file data to be sent from the virtual disk. After the read data is processed by the application, it relies on the protocol stack of the operating system to encapsulate and send the data packets through the aforementioned configured virtual network card, and then transmits it to the external node connected to the virtual machine. At the same time, the system can also receive file data returned or pushed from the external node through the virtual network card, and write it to the corresponding file path in the virtual disk after protocol parsing and data restoration. The above-mentioned sending and receiving behaviors constitute a complete two-way file transfer process, which can stably reproduce the data interaction status in actual business scenarios in a virtual environment.

[0051] Step 104: While executing the file transfer process, collect the operation data of the high-throughput file transfer system, and the operation data is used to characterize the transmission performance of the high-throughput file transfer system.

[0052] In this embodiment, to effectively evaluate the performance of a high-throughput file transfer system, the system under test is loaded and running in a virtual machine while its operating status is dynamically monitored and operational data related to transfer behavior is collected. This step, a key follow-up to the testing method, aims to provide quantitative data reflecting the system's operating performance under specific load and environmental conditions, providing a foundation for subsequent performance analysis and optimization.

[0053] Operational data, as used in this application, refers to a collection of data representing the operating status of a file transfer system, collected through software during its operation. This data typically includes key performance parameters such as data throughput, file transfer rate, and transmission latency. This data can reflect the performance of the file transfer system from multiple perspectives, including resource utilization, data processing capabilities, and network responsiveness, and is an essential and important source of information during testing.

[0054] To ensure the accuracy and real-time nature of operational data collection, this step is performed within a virtual machine. The collection process is synchronized with the file transfer process to ensure that the test data remains consistent with the actual system operating status. The isolation of the virtual machine prevents the collection process from interfering with system behavior and facilitates repeatable comparison of system performance under different test scenarios.

[0055] The acquisition mechanism relied upon in this step should have low resource consumption, high sampling accuracy, and good system integration. The collected data can be stored on a virtual disk or transmitted to an external analysis node in real time via a virtual network interface card, thus adapting to different test requirements.

[0056] To ensure the subsequent virtual machine's file system read and write speed and overall test performance, it is necessary to first build a virtual disk in the computing device before configuring the operating system on the virtual machine. This virtual disk demarcates continuous storage space in the physical memory by calling the host operating system's memory management mechanism, and provides a read and write interface equivalent to a block device through high-speed storage mapping, forming a memory-level storage resource with high throughput and low latency. Therefore, in one possible implementation, refer to Figure 2 , Figure 2 This is a second flow chart of the high-throughput file transfer test method provided by the present invention. Before step 101, the following steps are also included: Step 201: In a computing device having a multi-core processor and a large-capacity physical memory, a virtual disk for data storage is constructed, where the virtual disk is created based on the physical memory.

[0057] Step 202: Deploy a virtual machine in the virtual disk and allocate the entire capacity of the virtual disk to the virtual machine.

[0058] Specifically, in order to enable the high-throughput file transfer system to have sufficient operating bandwidth and low-latency storage capabilities, it is preferred to create a virtual disk for data storage in a computing device with a multi-core processor and a large-capacity physical memory. The virtual disk demarcates a continuous space in the physical memory by calling the memory allocation interface of the host operating system, and provides logical access capabilities to the outside world as a memory-resident block device. The construction of the virtual disk can be based on open source virtualization tools or system-level RAM-disk mechanisms. For example, in the Linux system, the tmpfs mounting method or the ramdisk driver is used to generate the corresponding block device, and the specified capacity is set for test data writing and reading. This method does not rely on traditional disk controllers or I / O buses, can provide extremely high access rates, and is suitable for high-frequency file operation scenarios.

[0059] After creating the virtual disk, a virtual machine instance is deployed within the virtual disk. This deployment process involves mapping the virtual disk as the primary storage device for the virtual machine using a virtualization management platform (such as QEMU / KVM, VirtualBox, etc.), and configuring the disk capacity and access permissions through configuration files or command-line parameters. After deployment is complete, the entire virtual disk capacity is allocated to the virtual machine, ensuring that the virtual machine operating system installation, high-throughput file transfer system loading, and file read and write operations are all completed on the virtual disk, thereby establishing the high-performance storage infrastructure required by the test system.

[0060] In this embodiment, in order to achieve smooth loading and stable operation of the virtual machine operating system, after completing the virtual disk creation and virtual machine deployment, the virtual disk structure needs to be initialized to ensure that the operating system can be correctly identified and loaded during the virtual machine startup phase. Figure 3 , Figure 3 This is the third flow chart of the high-throughput file transfer test method provided by the present invention. In one possible implementation, step 101 specifically includes the following steps: Step 301: Divide a system partition for installing an operating system on a virtual disk, and store a pre-prepared operating system image file in the system partition.

[0061] Step 302: When the virtual machine starts, the virtual machine automatically loads and starts the operating system image file based on the system partition of the virtual disk to complete the operating system initialization of the virtual machine.

[0062] Specifically, system partitioning can be achieved through the disk management tools provided by the virtualization platform. This involves using disk partitioning commands (such as fdisk and parted) to create primary or logical partitions on the virtual disk and formatting them with a file system type that supports operating system booting, such as ext4 or xfs. The operating system image file is then written to the designated boot directory within the system partition. This image file, which can be in standard ISO or IMG format, contains the kernel image, boot loader, and basic system components, ensuring full boot and operation capabilities.

[0063] After the system partition is initialized, when the virtual machine boots up, the virtual machine's runtime environment identifies the system partition on the virtual disk based on pre-set boot parameters and automatically loads and boots the operating system image file from it. This loading process relies on the boot sequence configured by the hypervisor (such as QEMU or VirtualBox), ensuring that the system boot process completes initialization without user intervention, thereby completing the automated configuration of the virtual machine's operating system. After booting up, the virtual machine enters a standard operating state, providing a fully functional environment for subsequent deployment of the file transfer system and performance testing.

[0064] In this embodiment, to ensure that the high-throughput file transfer system can successfully complete data communication operations within the virtual machine, it is necessary to create a virtual network card for data transmission based on the protocol stack of the operating system configured in the virtual machine. This step, as a key step in establishing the network environment in the testing method, aims to provide a highly controllable, low-overhead, and decoupled communication path from physical devices to support subsequent data transmission and reception processes. In one possible implementation, step 102 specifically includes the following steps: The virtual network interface creation instruction provided by the protocol stack is called within the operating system to generate a virtual network card in software; the virtual network card is used to communicate data between the virtual machine and the host machine on which the virtual machine runs. The data communication is carried out through the protocol stack and does not pass through the physical network card in the computing device.

[0065] During implementation, the operating system first calls the virtual network interface creation instructions provided by the protocol stack to generate a virtual network card in software. Taking the Linux operating system as an example, the TUN / TAP device creation mechanism is used to register a virtual network device node in kernel space and create a corresponding virtual network card device. The created virtual network card interface appears as a standard network interface in the system and can be accessed by user-mode processes as a device file. It supports point-to-point or bridge mode configuration.

[0066] Once created, the virtual network card can be bound to the virtual machine's network namespace or connected to the host machine's virtual switch via bridging, enabling efficient data communication between the virtual machine and the host. Throughout the communication process, the virtual network card is deeply integrated with the operating system's protocol stack. All data packets transmitted through the virtual network card are encapsulated, fragmented, and validated by the protocol stack, ensuring the integrity and standard compliance of network communications.

[0067] In addition, the communication path of the virtual network card is completely established between the virtual machine and the host machine. The data communication process does not pass through the physical network card of the computing device and does not occupy external network resources, thereby effectively avoiding the impact of traditional physical network card bandwidth limitations, device conflicts or network security issues on the testing process.

[0068] In this embodiment, in order to effectively load and run the high-throughput file transfer system in the virtual machine and establish a complete data reading and writing process, it is necessary to complete the installation and startup of the high-throughput file transfer system, data reading and sending, and data receiving and writing processes in sequence. Figure 4 , Figure 4 This is a fourth flow chart of the high-throughput file transfer test method provided by the present invention. In one possible implementation, step 103 specifically includes the following steps: Step 401: In the operating system deployed on the virtual disk, a high-throughput file transfer system is installed and started in a user-mode process.

[0069] Step 402: Read the file data to be sent from the operating system through the virtual disk, and send the file data to be sent to an external node in communication with the virtual network card through the virtual network card.

[0070] Step 403: Receive the to-be-received file data from the external node through the virtual network card, and write the to-be-received file data into the operating system through the virtual disk.

[0071] Specifically, step 401 first installs and starts the high-throughput file transfer system as a user-mode process. The system initializes by calling local library files, scheduling thread resources, and loading configuration parameters. It then registers the necessary file handles and network interfaces within the operating system environment, binding storage paths and communication links. Running the transfer system in user-mode avoids the need to modify kernel space, enhancing system flexibility and security while reducing reliance on underlying virtual machine permissions, making it suitable for rapid deployment and testing integration.

[0072] Furthermore, in order to realize the complete operation process of the high-throughput file transfer system in the virtual environment and construct the actual sending and receiving path of the data transmission path, the system performs data reading and sending, as well as receiving and writing operations in sequence after startup, thereby completing the two-way transmission of files between the virtual disk and the external node.

[0073] In step 402, the system determines the location of the target file data to be sent based on the configuration parameters and reads the file data by calling the file system interface. Preferably, to improve the throughput performance of the reading and sending process, the system can adopt a multi-threaded parallel processing mechanism to divide the data block into multiple sections. Multiple reading threads can read the data from the virtual disk in parallel, and then multiple sending threads can send the reading results to the external node through the virtual network card. This thread-based concurrent model can significantly reduce the synchronization wait between serial reading and sending, improve data processing throughput, and is particularly suitable for scenarios with large files or high concurrent transmission requirements.

[0074] However, parallelism isn't the only implementation path. In resource-constrained scenarios where the test goal is functional correctness verification, a serial approach can also be used to complete step 402, performing the read and send operations sequentially within a single-threaded logic. While this approach has performance bottlenecks, it offers advantages such as clear logic and ease of debugging, making it suitable for system debugging or transmission path verification.

[0075] Correspondingly, in step 403, the system receives the target file data from the external node via the virtual network card. To improve data reception efficiency and storage concurrency, in a preferred embodiment, the system also uses a multi-threading mechanism, with multiple receiving threads concurrently reading data from the network interface and multiple writing threads writing the data to the corresponding target paths on the virtual disk. This parallel processing approach helps reduce the risk of network buffer congestion and improves the concurrent bandwidth capacity of the write channel.

[0076] Similarly, in scenarios where resources are limited or performance is not the primary focus of verification, the system can also use a single-threaded serial receive and write approach to complete this step. Using serial logic to bind the receive and write operations into synchronous actions facilitates debugging control and process tracking during testing.

[0077] In a preferred embodiment, in order to meet the stringent requirements of high-throughput file transfer systems for throughput and response efficiency during the transmission process, it is necessary to optimize the data reading, sending, receiving, and writing processes to avoid performance bottlenecks caused by single-threaded processing. Therefore, in the process of reading and sending the data to be sent (corresponding to step 402), and receiving and writing the data to be received (corresponding to step 403), it is preferred to adopt a multi-threaded parallel processing mechanism to achieve high concurrency and high-speed data processing capabilities. Therefore, referring to Figure 5 , Figure 5 This is the fifth flow chart of the high-throughput file transfer test method provided by the present invention. Steps 402 and 403 are specifically implemented in the following manner: Step 501: at least one reading thread is used to read file data to be sent from a virtual disk in parallel, and at least one sending thread is used to send the file data to be sent read by the reading thread in parallel through a virtual network card.

[0078] Step 502: using at least one receiving thread to receive file data to be received from an external node in parallel through a virtual network card, and using at least one writing thread to write the file data to be received received by the receiving thread into a virtual disk in parallel.

[0079] Specifically, in the process of processing the data to be sent, at least one reading thread is used to read the file data to be sent from the virtual disk in parallel. The reading thread can divide the entire file into several data segments based on the file size and the preset block division strategy, and pull the data from different offset positions to the cache in parallel under multi-thread scheduling, avoiding the serial delay caused by sequential I / O. After the reading is completed, the file data obtained by the reading thread is sent to the external node through the virtual network card in parallel by at least one sending thread. The sending thread can extract the corresponding data segment from the shared cache, complete data encapsulation and message scheduling with the help of the operating system protocol stack, and realize high concurrent utilization of the network link.

[0080] Simultaneously, during the receive process, at least one receive thread receives incoming file data from external nodes in parallel via the virtual network interface card (VNIC) and writes it to a local buffer or temporary storage structure. Subsequently, at least one write thread writes the data received by the receive threads in parallel to the target storage path on the virtual disk. The write threads can write data to disk in an orderly manner based on the order of receipt or block number, supporting dynamic configuration of write paths and concurrent locking mechanisms to avoid data consistency risks caused by resource contention.

[0081] In this embodiment, in order to obtain the performance of the high-throughput file transfer system during operation and provide reliable data support for subsequent performance analysis and test report generation, it is necessary to deploy a special performance monitoring mechanism during the file transfer process to collect key parameters such as data throughput, file transfer rate, and transmission delay involved in the system operation process. Figure 6 , Figure 6 This is the sixth flow chart of the high-throughput file transfer test method provided by the present invention. In one possible implementation, step 104 specifically includes the following steps: Step 601: Deploy a performance monitoring module in a virtual machine for monitoring performance indicators of a high-throughput file transfer system.

[0082] Step 602: During the file transfer process, the performance monitoring module periodically collects operating parameters of the high-throughput file transfer system at preset time intervals.

[0083] Specifically, in a virtual machine environment, the performance monitoring module runs in conjunction with the high-throughput file transfer system in the form of an independent process or embedded service. The module can capture data on multiple operating dimensions of the transmission system based on a preset sampling period. The monitoring content includes but is not limited to the amount of data transmitted per unit time (i.e., data throughput), the average transmission time of file blocks (i.e., transmission delay), the transmission speed curve, the network interface utilization rate, etc. In order to ensure the real-time and lightweight nature of monitoring, the performance monitoring module can be implemented based on kernel-mode data sources or user-mode API calls, and maintain logical isolation from the main data path of the file transfer system to avoid interfering with its operating performance.

[0084] During the sampling process, the monitoring module periodically collects target parameters at preset intervals. Each sampling step generates a snapshot of data and writes it to a local cache or intermediate result file. The collected data is structured and can be directly used for subsequent statistical analysis or visualization. This periodic collection mechanism helps reflect system performance fluctuations throughout the file transfer cycle, expanding system testing from static metrics to dynamic performance curves, enhancing the timeliness and comprehensiveness of evaluation results.

[0085] By deploying a performance monitoring module in a virtual machine and periodically collecting operating parameters, we can not only achieve full quantitative tracking of the operating status of the high-throughput file transfer system, but also improve the degree of automation of the test system in performance verification.

[0086] In one possible implementation, the operating parameters include one or more of the data throughput, file transfer rate, and transmission delay of the high-throughput file transfer system. In this embodiment, in order to quantitatively evaluate the operating effect of the high-throughput file transfer system during the test and output performance results with reference value for decision-making, after completing the collection of operating parameters, it is necessary to perform statistical analysis on the collected parameters and generate a performance report based on the analysis results. Therefore, referring to Figure 7 , Figure 7 This is the seventh flow chart of the high-throughput file transfer test method provided by the present invention. After step 104, the following steps are also included: Step 701: Determine statistical indicators of the high-throughput file transfer system during the file transfer process based on one or more of data throughput, file transfer rate, and transmission delay; the statistical indicators include one or more of the average value, maximum value, or minimum value of data throughput, file transfer rate, and transmission delay.

[0087] Step 702: Based on all statistical indicators, generate a test performance report for characterizing the performance of the high-throughput file transfer system.

[0088] To determine statistical indicators, the system extracts core metrics, including data throughput, file transfer rate, and transmission latency, from the collected operational parameters and aggregates and analyzes them over time intervals. To improve the accuracy and stability of the indicators, common statistics such as the average, maximum, and minimum values ​​are calculated for each parameter category. This data aggregation process can be automated through embedded analysis modules or external data processing scripts, supporting both phased and full-cycle analysis methods to accommodate different test scenarios.

[0089] After completing the metric calculations, the system generates a structured test performance report based on all statistical results. This report includes metric definitions, data sources, calculation methods, time spans, and charts, visually demonstrating the actual performance of the high-throughput file transfer system during the test. This report can be output in a standard document format and can be exported and archived by testers after the test is complete, allowing them to use it for performance comparison and optimization recommendations.

[0090] Through the above implementation method, a complete indicator calculation and result expression link is constructed based on the performance parameter collection, so that the performance test results of the high-throughput file transfer system are transformed from the raw data level to the indicator presentation and report output, effectively improving the professionalism and practical value of the test method.

[0091] Reference Figure 8 , Figure 8 This is a schematic diagram of the structure of the high-throughput file transfer test system provided by the present invention, the system includes: A first configuration module is used to configure an operating system in a virtual machine; the virtual machine is deployed in a virtual disk of a computing device; A second configuration module is used to create a virtual network card for data communication based on the protocol stack of the operating system; A first processing module is configured to load the high-throughput file transfer system to be tested into the virtual machine, and enable the high-throughput file transfer system to perform a file transfer process based on the virtual disk and the virtual network card; The second processing module is used to collect operating data of the high-throughput file transmission system while executing the file transmission process, and the operating data is used to characterize the transmission performance of the high-throughput file transmission system.

[0092] In a possible implementation, the first configuration module is further configured to: In a computing device having a multi-core processor and a large-capacity physical memory, a virtual disk for data storage is constructed, where the virtual disk is created based on the physical memory; Deploy a virtual machine in a virtual disk and allocate the full capacity of the virtual disk to the virtual machine.

[0093] In a possible implementation, the second configuration module is further configured to: Call the virtual network interface creation instruction provided by the protocol stack in the operating system to generate a virtual network card in software; A virtual network card is used to communicate data between a virtual machine and the host machine on which the virtual machine runs. Data communication is carried out through the protocol stack and does not pass through the physical network card in the computing device.

[0094] In a possible implementation, the first processing module is further configured to: In the operating system deployed on the virtual disk, the high-throughput file transfer system is installed and started in the user-mode process mode; Read the file data to be sent from the operating system through the virtual disk, and send the file data to be sent to the external node connected to the virtual network card through the virtual network card; The file data to be received is received from the external node through the virtual network card, and the file data to be received is written into the operating system through the virtual disk.

[0095] In a possible implementation, the first processing module is further configured to: Reading the file data to be sent from the virtual disk in parallel using at least one reading thread, and sending the file data to be sent read by the reading thread in parallel through the virtual network card using at least one sending thread; At least one receiving thread is used to receive file data to be received from an external node in parallel through a virtual network card, and at least one writing thread is used to write the file data to be received received by the receiving thread into a virtual disk in parallel.

[0096] In a possible implementation, the second processing module is further configured to: Deploy a performance monitoring module in the virtual machine to monitor the performance indicators of the high-throughput file transfer system; During the file transfer process, the performance monitoring module periodically collects the operating parameters of the high-throughput file transfer system at preset time intervals.

[0097] In a possible implementation, the second processing module is further configured to: determining statistical indicators of the high-throughput file transfer system during the file transfer process based on one or more of data throughput, file transfer rate, and transmission delay; the statistical indicators including one or more of an average value, a maximum value, or a minimum value of the data throughput, the file transfer rate, and the transmission delay; Based on all statistical indicators, a test performance report is generated to characterize the performance of the high-throughput file transfer system.

[0098] In a possible implementation, the first configuration module is further configured to: Divide the virtual disk into a system partition for installing the operating system, and store the pre-prepared operating system image file in the system partition; When the virtual machine starts, the virtual machine automatically loads and starts the operating system image file based on the system partition of the virtual disk to complete the operating system initialization of the virtual machine.

[0099] It should be noted that the high-throughput file transfer test system provided by the present invention can execute the high-throughput file transfer test method of any of the above embodiments during specific operation, which will not be described in detail in this embodiment.

[0100] Figure 9 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 9 As shown, the electronic device may include: a processor 910 (processor), a communication interface 920 (Communications Interface), a memory 930 (memory) and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other via the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute a high-throughput file transfer test method, which includes: configuring an operating system in a virtual machine; deploying the virtual machine in a virtual disk of a computing device; creating a virtual network card for data communication based on the protocol stack of the operating system; loading a high-throughput file transfer system to be tested in the virtual machine, and causing the high-throughput file transfer system to execute a file transfer process based on the virtual disk and the virtual network card; while executing the file transfer process, collecting operating data of the high-throughput file transfer system, and the operating data is used to characterize the transmission performance of the high-throughput file transfer system.

[0101] Furthermore, the logic instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0102] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the high-throughput file transfer test method provided in the above embodiments.

[0103] On the other hand, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the high-throughput file transfer test method provided in the above embodiments.

[0104] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. That is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0105] Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods of each embodiment or certain portions of the embodiments.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A high-throughput file transfer test method, characterized in that: include: Configure the operating system in the virtual machine; The virtual machine is deployed in a virtual disk of a computing device; Creating a virtual network card for data communication based on the protocol stack of the operating system; Loading a high-throughput file transfer system to be tested in the virtual machine, and causing the high-throughput file transfer system to perform a file transfer process based on the virtual disk and the virtual network card; While executing the file transfer process, operation data of the high-throughput file transfer system is collected, and the operation data is used to characterize the transmission performance of the high-throughput file transfer system.

2. The high-throughput file transfer test method according to claim 1, characterized in that: Before configuring the operating system in the virtual machine, the method further includes: In a computing device having a multi-core processor and a large-capacity physical memory, a virtual disk for data storage is constructed, wherein the virtual disk is created based on the physical memory; A virtual machine is deployed in the virtual disk, and the entire capacity of the virtual disk is allocated to the virtual machine.

3. The high-throughput file transfer test method according to claim 1, characterized in that: The creating of a virtual network card for data communication based on the protocol stack of the operating system includes: Invoking a virtual network interface creation instruction provided by the protocol stack within the operating system to generate the virtual network card in a software manner; The virtual network card is used to perform data communication between the virtual machine and the host machine on which the virtual machine runs, and the data communication is performed through the protocol stack without passing through the physical network card in the computing device.

4. The high-throughput file transfer test method according to claim 1, characterized in that: The step of loading the high-throughput file transfer system to be tested into the virtual machine and causing the high-throughput file transfer system to perform a file transfer process based on the virtual disk and the virtual network card includes: In the operating system deployed on the virtual disk, installing and starting the high-throughput file transfer system in a user-mode process; Reading the file data to be sent from the operating system through the virtual disk, and sending the file data to be sent to an external node in communication with the virtual network card through the virtual network card; The to-be-received file data is received from the external node through the virtual network card, and the to-be-received file data is written into the operating system through the virtual disk.

5. The high-throughput file transfer test method according to claim 4, characterized in that: Also includes: Reading the to-be-sent file data from the virtual disk in parallel using at least one reading thread, and sending the to-be-sent file data read by the reading thread in parallel through the virtual network card using at least one sending thread; The to-be-received file data is received in parallel from the external node through the virtual network card by at least one receiving thread, and the to-be-received file data received by the receiving thread is written in parallel to the virtual disk by at least one writing thread.

6. The high-throughput file transfer testing method according to claim 1, characterized in that: The collecting of operation data of the high-throughput file transfer system while executing the file transfer process includes: Deploying a performance monitoring module in the virtual machine for monitoring performance indicators of the high-throughput file transfer system; During the file transmission process, the performance monitoring module periodically collects the operating parameters of the high-throughput file transmission system at preset time intervals.

7. The high-throughput file transfer test method according to claim 1, characterized in that: The operating parameters include one or more of the data throughput, file transfer rate, and transmission delay of the high-throughput file transfer system; After collecting the operating data of the high-throughput file transfer system, the method further includes: Determining statistical indicators of the high-throughput file transfer system during the file transfer process based on one or more of the data throughput, the file transfer rate, and the transmission delay; the statistical indicators including one or more of an average value, a maximum value, or a minimum value of the data throughput, the file transfer rate, and the transmission delay; Based on all the statistical indicators, a test performance report is generated to characterize the performance of the high-throughput file transfer system.

8. The high-throughput file transfer testing method according to claim 1, wherein: Configuring the operating system in the virtual machine includes: Dividing a system partition for installing an operating system on the virtual disk, and storing a pre-prepared operating system image file in the system partition; When the virtual machine is started, the virtual machine automatically loads and starts the operating system image file based on the system partition of the virtual disk to complete the operating system initialization of the virtual machine.

9. A high-throughput file transfer test system, characterized in that: include: A first configuration module is configured to configure an operating system in a virtual machine deployed in a virtual disk of a computing device; A second configuration module is used to create a virtual network card for data communication based on the protocol stack of the operating system; A first processing module is configured to load the high-throughput file transfer system to be tested into the virtual machine, and enable the high-throughput file transfer system to perform a file transfer process based on the virtual disk and the virtual network card; The second processing module is used to collect operating data of the high-throughput file transmission system while executing the file transmission process, and the operating data is used to characterize the transmission performance of the high-throughput file transmission system.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the high-throughput file transfer testing method according to any one of claims 1 to 8 is implemented.

Images