Simulation test system for block equipment
Through a simulation test system for block equipment, load generation modules and scheduling strategies are used to generate workloads that fit the real environment, solving the problems of inaccurate test results and high cost in the existing technology, and achieving efficient and low-cost performance testing.
Patent Information
- Application Number
- CN202510389251.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-31
- Publication Date
- 2025-08-26
AI Technical Summary
The existing block equipment simulation test methods do not fit the real environment when the load does not fit the real environment, resulting in inaccurate test results and requires a complex test environment, which increases cost and time.
It provides a simulation test system for block equipment, including a load generation module, a load simulation test module and a performance analysis module. It receives I/O requests from the real system for optimization processing, generates workloads that fit the real environment, and uses Time or Shape scheduling strategies for simulation tests.
It realizes high-accuracy simulation testing, reduces testing costs, and can obtain reliable performance data without building a hardware environment, and performs accurate read and write performance optimization and comparison analysis.
Smart Images
Figure CN120540947A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of block device storage, and in particular to a simulation test system oriented to block devices. Background Art
[0002] With the rapid development of information technology, the demand for data generation, processing, and storage has exploded. Cloud computing, big data, artificial intelligence, and other fields are placing higher demands on the performance, capacity, and reliability of storage devices. Block devices, with their high-speed read and write capabilities and random access capabilities, can provide stable and high-performance storage for systems, flexibly manage storage resources, and accelerate data processing.
[0003] Block device development must be tailored to the real-world operating environment, necessitating testing of developed block devices. Traditionally, block device testing is performed after development is complete. This approach requires preparing the test environment, including hardware, software support, and network configuration. The cost of a developed block device storage system includes not only hardware but also software, service costs, networking equipment, and application development. In this context, simulation technology can be used to simulate the storage system's environment and behavior, generating a variety of complex workloads to emulate the access and operations of real users or applications. Through simulation testing, storage system performance data can be obtained without building an actual system, providing valuable reference and foundation for R&D personnel. Furthermore, simulation technology can reduce testing costs and time, while improving test efficiency and accuracy.
[0004] In the prior art, a commonly used block device simulation test method is to use a loopback device to treat randomly created regular files as block devices, and use a test tool to complete the read and write process. However, due to the complexity and diversity of the actual environment, using a loopback device to treat randomly created regular files as block devices only supports fixed load testing, which is inconsistent with the load of the actual production environment. Therefore, the test results of this method are difficult to accurately reflect the performance of the block device storage system to be developed. Summary of the Invention
[0005] Based on this, it is necessary to provide a simulation test system for block devices to address the above technical problems. The system can test the block device to be developed before the block device is developed. There is no need to build a test environment, which is low-cost. The test load fits the real environment and can accurately reflect the performance of the block device storage system to be developed.
[0006] The present invention provides a block device-oriented simulation test system, comprising:
[0007] The load generation module is used to receive multiple I / O requests generated by the real system and optimize them to obtain the workload of the simulation test;
[0008] A load simulation test module is used to receive and schedule the workload of the simulation test as an ordered workload, store metadata of the workload of the simulation test, and execute the ordered workload;
[0009] The performance analysis module is used to receive the workload of the simulation test generated by the load generation module and distribute the workload of the simulation test according to the request address and the size of the data segment of the block device on the real system to obtain the number of read and write times, size and sparsity of each data segment of the workload of the simulation test, as well as the total number and size of reads and writes of the workload of the simulation test; it is also used to obtain the time consumption of the read and write operations of the load simulation test module.
[0010] Preferably, the load generation module includes a receiving unit and an optimization unit;
[0011] a receiving unit, configured to receive a plurality of I / O requests generated by the real system from the real system;
[0012] The optimization unit is used to optimize multiple I / O requests generated by the real system, specifically including eliminating I / O requests whose specified locations do not exist and whose data size is 0, filling write requests before I / O requests whose corresponding locations do not have readable data, and splitting I / O requests that run across blocks according to the dividing lines of the data segments of the block devices on the real system.
[0013] Preferably, the load simulation test module includes:
[0014] A scheduling unit, configured to receive a workload of a simulation test and schedule the workload of the simulation test into an ordered workload using a scheduling strategy;
[0015] A database for storing metadata of simulation test workloads;
[0016] A process unit that receives and executes all I / O requests in an ordered workload.
[0017] Preferably, the process of executing all I / O requests in the ordered workload by the process unit is to query metadata of each I / O request in the ordered workload from a database, apply each I / O request to a storage pool for read and write operations according to the metadata of each I / O request, and save the time consumed by the read and write operations of each I / O request in the database;
[0018] The metadata includes I / O size, location, and cache status.
[0019] Preferably, the time consumption of the read and write operations of the load simulation test module is obtained from the database by the time consumption of the process unit executing all I / O request read and write operations.
[0020] Preferably, the number of read and write times and the size of each data segment are added together to obtain the total number of read and write times and the size of the workload of the simulation test.
[0021] Preferably, the scheduling strategy adopted by the scheduling unit is a Time scheduling strategy or a Shape scheduling strategy.
[0022] The beneficial effects of the present invention are:
[0023] (1) The block device simulation test system of the present invention uses a load generation module to receive multiple I / O requests generated by a real system and optimizes them to obtain the workload used for simulation testing. The test load fits the real environment, and the test accuracy is high. More realistic and reliable performance data can be obtained, thereby enabling more accurate read and write performance optimization and comparative analysis of the block device to be developed.
[0024] (2) The block device-oriented simulation test system of the present invention does not require installation and construction of a test environment during the entire test process of the block device to be developed, has no requirements for hardware devices, and is low in cost. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 Schematic diagram of the modular structure of a simulation test system for block devices provided by an embodiment of the present invention;
[0026] Figure 2 This is a schematic diagram of the Time scheduling strategy provided by an embodiment of the present invention;
[0027] Figure 3 This is a schematic diagram of the Shape scheduling strategy provided by an embodiment of the present invention;
[0028] Figure 4 It is a structural diagram of the load simulation test module provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0029] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0030] In one embodiment, Figure 1 As shown, Figure 1 FIG. 1 is a schematic diagram of the modular structure of a simulation test system for block devices provided by an embodiment of the present invention. The simulation test system for block devices of this embodiment includes:
[0031] The load generation module is used to receive multiple I / O requests generated by the real system and optimize them to obtain a workload for the simulation test. The purpose of optimizing the multiple I / O requests generated by the real system is to obtain a workload that is more consistent with the response of the block device under test.
[0032] Specifically, the load generation module includes a receiving unit and an optimization unit.
[0033] The receiving unit is configured to receive a plurality of I / O requests generated by the real system from the real system. The receiving unit is deployed in the real computer system.
[0034] The optimization unit is used to optimize multiple I / O requests generated by the real system, specifically including eliminating I / O requests whose specified locations do not exist and whose data size is 0, filling write requests before I / O requests whose corresponding locations do not have readable data, and splitting I / O requests that run across blocks according to the dividing lines of the data segments of the block devices on the real system.
[0035] Among them, eliminating I / O requests in which the specified location does not exist and the data size is 0 makes the workload of the simulation test legal, and filling write requests before I / O requests in which there is no readable data at the location corresponding to the read request makes the workload of the simulation test portable.
[0036] The load simulation test module is used to receive and schedule the workload of the simulation test as an ordered workload, store metadata of the workload of the simulation test, and execute the ordered workload.
[0037] The load simulation test module can perform the I / O request response process. Specifically, the scheduling strategy adopted by the scheduling unit is the Time scheduling strategy or the Shape scheduling strategy.
[0038] Among them, Figure 2 As shown, Figure 2This is a schematic diagram of the Time scheduling strategy provided by an embodiment of the present invention. The Time scheduling strategy arranges I / O according to the running time on the real environment device and simulates the running time and sequence on the real environment device. On the real environment device, I / O1 is the first I / O operation to be run, followed by I / O3, I / O4, I / O2, and I / O5. The I / Os arranged with the Time scheduling strategy will be implemented on the load simulation test module in this order and the start time of the run. When the Time scheduling strategy is adopted, it is necessary to retain the start time of each I / O in the real environment device, regard the time at the beginning of the simulation test as 0, and arrange them in sequence according to the original start time of each I / O as time goes by. This scheduling strategy adopts the control variable method, which mainly controls the time and running sequence to compare the performance differences of block devices with different designs.
[0039] like Figure 3 As shown, Shape I / O scheduling requires analyzing the relationships between I / O operations on real-world devices based on runtime. For example, I / O2 executes after I / O3, or I / O3 and I / O4 have runtime conflicts. The original I / O execution order is reordered based on dependencies to maximize I / O concurrency. Under the Shape scheduling strategy, I / O1, I / O4, and I / O3 are executed in parallel, I / O4 and I / O5 are executed sequentially, and I / O3 and I / O2 are also executed sequentially. This scheduling strategy maximizes resource utilization of the block device system while ensuring the original I / O semantics remain unchanged, restoring the scheduling of actual production I / O.
[0040] The performance analysis module is used to receive the workload of the simulation test generated by the load generation module and distribute the workload of the simulation test according to the request address and the size of the data segment of the block device on the real system to obtain the number of read and write times, size and sparsity of each data segment of the workload of the simulation test, as well as the total number and size of reads and writes of the workload of the simulation test; it is also used to obtain the time consumption of the read and write operations of the load simulation test module.
[0041] Specifically, the number of read and write times and the size of each data segment are added together to obtain the total number of read and write times and the size of the workload of the simulation test.
[0042] The performance parameters of the block device to be developed obtained by the performance analysis module can be used to guide the queue depth, maximum number of operation pages in the queue, cache space size, cache mechanism and data block size of the real block device to be developed.
[0043] The block device simulation test system of this embodiment uses a load generation module to receive multiple I / O requests generated by a real system and optimizes them to generate the workload for simulation testing. This test load closely matches the real environment, providing high test accuracy and obtaining more realistic and reliable performance data, enabling more precise read and write performance optimization and comparative analysis of the block device under development. Furthermore, the entire testing process requires no installation or setup of the test environment, and there are no hardware requirements, resulting in low costs.
[0044] In one embodiment, Figure 4 As shown, Figure 4 : is a schematic diagram of the structure of a load simulation test module provided by an embodiment of the present invention, the load simulation test module includes:
[0045] The scheduling unit is used to receive the workload of the simulation test and schedule the workload of the simulation test into an ordered workload using a scheduling strategy.
[0046] A database used to store metadata of simulation test workloads.
[0047] A process unit that receives and executes all I / O requests in an ordered workload.
[0048] Preferably, the process of executing all I / O requests in the ordered workload by the process unit is to query metadata of each I / O request in the ordered workload from a database, apply each I / O request to a storage pool for read and write operations according to the metadata of each I / O request, and save the time consumed by the read and write operations of each I / O request in the database;
[0049] Metadata includes I / O size, location, and cache status. Obtaining metadata allows you to determine information such as data layout, indexes, and partitions, as well as the physical location of data on storage. To simulate cache performance, the metadata section simulates the cache's operating state. Caching frequently accessed data and metadata reduces the need for physical I / O operations, resulting in access speeds closer to those experienced in real systems.
[0050] It should be noted that after each I / O request is applied to the storage pool for read and write operations, the metadata of each I / O request needs to be updated for use in subsequent I / O request operations to maintain data consistency.
[0051] For write operations, data is written to the emulated block device storage medium. A storage pool generally refers to a logically combined set of storage resources that may span multiple physical storage units. For read operations, the location of the data is determined based on metadata and retrieved from the storage pool.
[0052] Specifically, the time consumption of the read and write operations of the load simulation test module is obtained from the database as the time consumption of the process unit executing all I / O request read and write operations.
[0053] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0054] The above-described embodiments merely represent several implementation methods of the present invention. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.
Claims
1. A simulation test system for block devices, characterized in that: include: The load generation module is used to receive multiple I / O requests generated by the real system and optimize them to obtain the workload of the simulation test; A load simulation test module, configured to receive and schedule the workload of the simulation test as an ordered workload, store metadata of the workload of the simulation test, and execute the ordered workload; The performance analysis module is used to receive the workload of the simulation test generated by the load generation module and distribute the workload of the simulation test according to the request address and the size of the data segment of the block device on the real system to obtain the number of read and write times, size and sparsity of each data segment of the workload of the simulation test, as well as the total number and size of reads and writes of the workload of the simulation test; it is also used to obtain the time consumption of the read and write operations of the load simulation test module.
2. The block device-oriented simulation test system according to claim 1, characterized in that: The load generation module includes a receiving unit and an optimization unit; The receiving unit is configured to receive a plurality of I / O requests generated by the real system from the real system; The optimization unit is used to optimize multiple I / O requests generated by the real system, specifically including eliminating I / O requests whose specified locations do not exist and whose data size is 0, filling write requests before I / O requests whose locations corresponding to read requests do not have readable data, and splitting I / O requests that run across blocks according to the dividing lines of the data segments of the block devices on the real system.
3. The block device-oriented simulation test system according to claim 2, characterized in that: The load simulation test module includes: A scheduling unit, configured to receive a workload of a simulation test and schedule the workload of the simulation test into an ordered workload using a scheduling strategy; A database for storing metadata of simulation test workloads; A process unit that receives and executes all I / O requests in an ordered workload.
4. The block device-oriented simulation test system according to claim 3, characterized in that: The process unit executes all I / O requests in the ordered workload by querying the metadata of each I / O request in the ordered workload from the database, applying each I / O request to the storage pool for read and write operations based on the metadata of each I / O request, and saving the time consumed by each I / O request read and write operation in the database; The metadata includes I / O size, location, and cache status.
5. The block device-oriented simulation test system according to claim 1, characterized in that: The time consumption of the read and write operations of the load simulation test module is obtained from the database as the time consumption of the process unit executing all I / O request read and write operations.
6. The block device-oriented simulation test system according to claim 1, characterized in that: The total number of reads and writes and the size of each data segment are added together to obtain the total number of reads and writes and the size of the workload of the simulation test.
7. The block device-oriented simulation test system according to claim 3, characterized in that: The scheduling strategy adopted by the scheduling unit is a Time scheduling strategy or a Shape scheduling strategy.