Database external running environment dynamic simulation method and simulation system

Abstract

The application discloses a kind of database external operating environment dynamic simulation methods, comprising the following steps: simulating various external operating environments, according to the target scene of user self-defined to generate the resource simulation load of specified type, specified intensity, and then run load to manufacture target environment;Simulate the dynamic change of external operating environment, according to the dynamic change situation of user self-defined, manufacture corresponding dynamic environment.The application also discloses a system for implementing the above-mentioned dynamic simulation method, including client and server side;The client includes controller, parser, load generator and test report generator;The server side includes dynamic change controller, load executor, index collector, target server.The method of the application can simulate the environment load of hardware resources and database software resources from multiple dimensions, accurately simulate various resource consumption scenarios, and help evaluate the reliability and stability of database system services.

Description

Technical Field

[0001] This invention relates to the fields of database, cloud platform, database evaluation, and environment simulation technology, and in particular to a dynamic simulation method and simulation system for the external operating environment of a database. Background Technology

[0002] The application of cloud computing platforms has grown unprecedentedly in the past 20 years. According to the statistical report [1], the public cloud computing market value will reach $800 billion by 2025. More and more enterprises are choosing to run their workloads in public cloud environments. As of 2020, 61% of enterprises had migrated their workloads to the cloud, and this trend is likely to continue in the coming years, as 46% of companies reported better financial conditions. Containerized [2] cloud environments are becoming increasingly popular. Deploying applications with cloud containers is more efficient than using conventional servers. Container orchestration engines (such as Kubernetes [3]) can provide the necessary runtime environment for managing applications across different clouds, making containerized cloud environments more widely available to users. Container images contain all the dependencies required to run an application, and users can flexibly deploy them on any container-enabled host. The maturity and popularity of cloud platforms have driven the development of cloud-native software systems, including cloud-native databases.

[0003] The resource pooling capability, flexible scheduling capability, and demand-oriented seamless elastic scaling capability of cloud platforms bring convenience, but also introduce new challenges. On a cloud platform, a single physical machine can host multiple application instances[4], and a large number of intensive and highly dynamic workloads with different operation types may run simultaneously in the system. Although containers deployed by different applications may be isolated from each other through software, there is still competition for environmental resources among containers because these containers share resources in the cloud environment. In addition, in order to make full use of cloud resources, cloud service providers usually sell more resources to obtain greater profits[5]. These factors make the resource competition among cloud-native applications more serious, exacerbating the uncontrollability and instability of the database operating environment. Therefore, simulating the dynamism and instability of the real deployment environment is crucial for evaluating the service performance, service availability, and reliability of databases (especially cloud-native databases).

[0004] Database systems, as foundational supporting software, run within a system environment. Cloud databases require resources from other application software. Therefore, database systems encounter resource contention within the system environment during operation, including hardware and software resources, which in turn affects user workload performance. Furthermore, in production environments, the type and intensity of environmental loads can change dynamically, impacting database performance and even its correctness. Current work addresses this by creating complex operating environments through fault injection and simulating resource consumption.

[0005] Chaos testing tools, such as ChaosBlade[7] and Chaos-Mesh[8], simulate various system resource corruption scenarios through fault injection. While fault injection can test the database's fault tolerance and recovery capabilities, it cannot reflect the dynamic changes in the normal deployment environment and lacks the universality of environment simulation. Moreover, the impact of load on resources is stable after these tools are run, and they cannot simulate dynamically changing disturbance scenarios.

[0006] Jeong[9]'s work simulates the real operating environment of an application by generating characteristic loads. However, this work can only simulate the consumption of CPU, memory, and disk resources, without considering the consumption of network resources or simulating changes in software resources, resulting in poor comprehensiveness. Furthermore, this tool cannot simulate dynamic changes in environmental load. In addition, the load used in this work is relatively bulky, resulting in a slow speed in realizing the target scenario. Moreover, when simulating a certain type of resource consumption load, it will cause significant consumption of other hardware resources, leading to an inaccurate simulation of the scenario.

[0007] Zhang C et al. from the School of Data Science at East China Normal University independently developed a dynamic workload generator for simulating database environments

[10] , which can simulate the consumption of CPU, memory, disk I / O resources, and network bandwidth. This work proposes a dynamic environment generation algorithm that can simulate a relatively complex application running environment. However, this work is not comprehensive enough. For example, it only simulates the scenario of disk capacity occupation and sequential write I / O, and only considers the simulation of network bandwidth occupation for network resources, without considering the impact of software resources. The scenarios that can be generated are limited. This work does not avoid the obvious consumption of other resources in the load design, resulting in poor accuracy of the simulated resource consumption scenarios.

[0008] In summary, the aforementioned related works are all lacking in terms of the universality, comprehensiveness, dynamism, and accuracy of the simulation results, making the database evaluation results in these simulation environments unreliable. Summary of the Invention

[0009] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method and system for simulating the external operating environment of a database.

[0010] The challenge in simulating the external operating environment of a database lies in achieving high accuracy, efficiency, comprehensiveness, and versatility. Accuracy means that the generated environment closely resembles the target environment, accurately consuming target resources while minimizing the impact of the simulated load on other system resources. Efficiency means generating the target environment quickly. Comprehensiveness means simulating a wide range of resource types. Versatility means simulating both faulty and non-faulty states of the environmental resources involved in the database's operating environment at a given moment. This invention designs a lightweight environmental load and uses agile algorithms to control the dynamic changes of the environmental load, achieving accurate and efficient simulation of the target environment. To more comprehensively simulate the operating environment, the resource simulation scope of this invention covers both hardware resources and database-related software resources. To more meticulously characterize the database operating environment and achieve better versatility, this invention divides the database operating environment into the following two categories:

[0011] (1) Normal operating environment: Under fault-free conditions, the state of other applications consuming system resources in the operating environment; changes in the normal operating environment will test the performance stability of the database and the availability of the database service; for example, in the cloud environment, there is resource competition between other containers and the container where the database is located.

[0012] (2) Extreme operating environment: The extreme situation in which the resources of the machine where the database is located are fully occupied or even destroyed; extreme environment can help test the fault tolerance and high availability of the database; for example, in the cloud environment, the partition files in the container where the database is located are corrupted.

[0013] This invention proposes a method for dynamically simulating the external operating environment of a database, comprising the following steps:

[0014] Step 1: Simulate various external operating environments. Based on the user-defined target scenario, generate resource simulation loads of specified type and intensity, and then run the loads to create the target environment. The simulated external operating environments include:

[0015] Simulate a normal operating environment: Simulate the state of system hardware and software resource consumption by other applications in the operating environment under fault-free conditions; the hardware resources consumed include CPU, memory, disk, network, etc., and the software resources include processes, etc.

[0016] Simulate extreme operating environments: Simulate extreme situations where the hardware and software resources of the machine hosting the database are fully occupied or even damaged in the operating environment; simulated hardware resources that are fully occupied and / or damaged include CPU, memory, disk, network, host, etc., and software resources include data files, processes, network connections of database ports, etc.

[0017] Step 2: Simulate dynamic changes in the external operating environment. Based on user-defined dynamic changes, create a corresponding dynamic environment. User-defined dynamic changes include the intensity and duration of simulated load changes for each resource.

[0018] Based on the Woodpecker testing framework [6], this invention designs and implements a database external operating environment simulation system. This system can generate environmental loads of specified intensity and duration for hardware resources (such as CPU, memory, disk, network, etc.) and database software resources, achieving accurate simulation of various resource consumption scenarios, which can help evaluate the reliability and stability of database system services. After the environmental simulation ends, the system will automatically restore the resources consumed or damaged in the environmental simulation. The system is divided into two parts: client and server. The client components include a controller, parser, load generator, and test report generator. The server includes a dynamic change controller, load executor, metric collector, and target server.

[0019] The controller: The system configuration file contains information such as the environment simulation task, the target database for the load to be executed, and the test report storage directory; The controller is used to read the current environment simulation task, the target database for the load to be executed, and the test report storage directory from the system configuration file, and then start the parser in sequence; The controller can also be used to count the success and failure of the environment simulation task; After each round of simulation, the controller will call the test report generator to automatically generate a test report.

[0020] The parser described in this invention consists of a predefined syntax rule for describing environmental loads (including how to describe environmental scenario types, dynamic load changes, etc.) written in a file with the .jj extension. This file is then compiled using JavaCC to generate a parser capable of parsing this syntax rule. The parser, following this syntax rule, extracts information from the test case file, including each load simulation task (containing the target scenario, dynamic load change parameters, etc.) and the target machine to execute the load. Then, using Java's reflection mechanism, it maps the load simulation tasks to Java classes that implement specific functionalities. Finally, it calls the load generator to generate corresponding resource consumption simulation loads and resource saturation / destruction simulation loads.

[0021] The load generator is used to generate environmental loads based on the load simulation tasks sent by the parser, and send the environmental loads to the dynamic change controller on the server side. The types of loads generated by the load generator include: (1) Simulating a normal operating environment: simulating the consumption of hardware resources such as CPU, memory, disk, and network, as well as software resources such as processes. (2) Simulating an extreme operating environment: simulating the full consumption and / or damage of hardware resources such as CPU, memory, disk, network, and host, as well as software resources such as data files, processes, and network connections of database ports.

[0022] The dynamic change controller is used to invoke the load executor to perform environmental load on the machine where the database is located, and to control the intensity and duration of the environmental load.

[0023] The load executor is used to remotely log in to the target machine and execute simulated workloads. The executor includes a normal environment load executor and an extreme environment load executor. The normal environment load executor can simulate the state where other applications consume system hardware and software resources in a fault-free environment; the extreme environment load executor can simulate extreme situations where the hardware and software resources of the machine hosting the database are fully utilized or even damaged. If the target database is located on a regular host, both the normal environment load executor and the extreme environment load executor execute the workload outside the database. If the target database is located in a cloud environment, the normal environment load executor executes the workload outside the container where the database resides; the extreme environment load executor executes the workload inside the container where the database resides.

[0024] The target server: This invention applies environmental load to the target server during environment simulation. The target server contains a DBMS (database management system) and a monitor. The DBMS applies user-specified database load during environment simulation, and changes in database performance reflect the operational quality of the database service under environmental changes. The monitor periodically collects fine-grained performance metrics and system resource usage of the database during the simulated load operation.

[0025] The indicator collector is used to receive fine-grained performance indicators and system resource usage of the database during simulated load operation returned by the monitor. It performs statistical processing and formatting on the collected performance indicators (throughput, latency, etc.) (i.e., calculates the average value and 95th percentile value of the performance indicators within the time interval given by the user, and saves the statistical results in CSV format), so that database evaluation personnel can analyze the database operation based on the performance data.

[0026] The test report generator is used to automatically output test reports that include environmental factors and database system performance information.

[0027] The present invention also proposes the above-mentioned dynamic simulation method and the application of the above-mentioned dynamic simulation system in the dynamic simulation of the external operating environment of the database.

[0028] The beneficial effects of this invention include: the simulation of the external operating environment of the database proposed by this invention is comprehensive, universal, accurate, efficient and dynamic. It can simulate the environmental load on hardware resources and database software resources from multiple dimensions, achieve accurate simulation of various resource consumption scenarios, help evaluate the reliability and stability of database system services, and make the database evaluation results in the environment simulated by this invention have better reference value. Attached Figure Description

[0029] Figure 1 The overall framework of the simulation system for the external operating environment of the database.

[0030] Figure 2 This is a diagram simulating CPU usage.

[0031] Figure 3 This is an experimental graph showing the average impact of CPU load on memory.

[0032] Figure 4 This is a diagram simulating memory usage.

[0033] Figure 5 This is an experimental graph showing the average impact of memory usage load on the CPU.

[0034] Figure 6 This is a graph simulating disk IOPS usage.

[0035] Figure 7 This is an experimental graph showing the average impact of disk IOPS on CPU usage.

[0036] Figure 8 This is an experimental graph showing the average impact of disk IOPS on memory usage.

[0037] Figure 9 This is a diagram simulating the impact on network bandwidth.

[0038] Figure 10 This is an experimental graph showing the average impact of network load on the CPU. Detailed Implementation

[0039] The invention will be further described in detail below with reference to the specific embodiments and accompanying drawings. Except for the content specifically mentioned below, the processes, conditions, and experimental methods for implementing the invention are all common knowledge and general knowledge in the art, and the invention does not have any particular limitations.

[0040] The dynamic simulation method for the external operating environment of a database proposed in this invention includes the following steps:

[0041] Step 1: Simulate various external operating environments. Based on the user-defined target scenario, generate resource simulation loads of specified type and intensity, and then run the loads to create the target environment. The simulated external operating environments include:

[0042] Simulate a normal operating environment: Simulate the state of system hardware and software resource consumption by other applications in the operating environment under fault-free conditions; the hardware resources consumed include CPU, memory, disk, network, etc., and the software resources include processes, etc.

[0043] Simulate extreme operating environments: Simulate extreme situations where the hardware and software resources of the machine hosting the database are fully occupied or even damaged in the operating environment; simulated hardware resources that are fully occupied and / or damaged include CPU, memory, disk, network, host, etc., and software resources include data files, processes, network connections of database ports, etc.

[0044] The generation mechanisms for each type of environmental load are as follows:

[0045] (1) Simulate the generation mechanism of CPU load consumption:

[0046] The method of simulating CPU usage in this invention is to allocate a single-threaded process equal to the number of CPU cores required, and execute assignment operations infinitely multiple times under a while(true) condition, thereby occupying the corresponding number of CPU cores. Since each process is executing an infinite loop operation, the CPU utilization rate of the process will quickly rise to 100%. CPU-intensive loads often involve arithmetic calculations, but the execution of these calculations inevitably involves caching intermediate results, leading to significant memory consumption. This makes it impossible to accurately simulate scenarios that only heavily utilize CPU resources. Therefore, the CPU usage simulation load of this invention only incorporates simple operations such as assignment, thus avoiding caching of intermediate results and causing almost no memory consumption.

[0047] (2) Simulate the generation mechanism of memory-intensive load:

[0048] This invention simulates memory usage by creating a tmpfs filesystem and then writing data of disk size $data_size into it to occupy memory. tmpfs is a virtual memory filesystem that uses memory to store data. This invention uses the Linux built-in dd command to fill the tmpfs filesystem with data by reading data from / dev / zero and then writing it to tmpfs to achieve the effect of occupying memory. Since / dev / zero is a pseudo-device that only generates an empty character stream, reading from it does not generate I / O. The speed of writing the read data into memory is related to the data block size $bs of each operation. When $bs is set to a large multiple of the disk sector size (e.g., 512K), the extra overhead from system calls can be significantly reduced, thereby improving the speed of occupying memory. The writing process naturally does not generate disk I / O, only having a slight impact on CPU consumption, while maintaining memory occupation consumes almost no disk I / O and CPU. The command executed to simulate memory usage is as follows:

[0049] Mount tmpfs: mount-t tmpfs-o size=$data_size tmpfs / tmp / memory

[0050] Write tmpfs:ddif= / dev / zero of= / tmp / memory / block

[0051] bs=$block_size count=$[$data_size / $block_size]

[0052] (3) Mechanism for simulating disk resource load generation:

[0053] Disk space occupancy: Data equal to the required disk space ($data_size) is written to the file system, thus occupying disk space. Although the execution process involves multiple reads and writes to the data file, this invention specifies the parameter "flags=direct" in the load to control the read and write methods. This avoids the file system cache when operating on the disk, so the execution of the load has a slight impact on the CPU and consumes almost no memory. To reduce the impact on disk I / O during data writing, this load reads data from / dev / zero (a pseudo-device that only generates empty character streams), thus avoiding read I / O. The command simulating disk resource occupancy is as follows:

[0054] Occupy size:dd oflag=direct if=file read of = file write

[0055] bs=$single_size count=$[$data_size / $single_size]

[0056] Disk IOPS for sequential operations: One sequential read / write operation is performed on the disk per second (via the `dd` command), with each IO operation being a 512-byte data block. The load specifies the parameter "flags=direct" to control the read / write mode. This bypasses the file system cache during disk operations, making the number of disk blocks operated on during the load approximately equal to the number of disk IO operations. The specific algorithm is as follows: Lines 1-5 define the function for consuming disk IOPS. When generating sequential write IOs, this load simulates writing data to the file system. The data source read is (file...). read The value is / dev / zero. / dev / zero is a pseudo-device that only generates an empty character stream; reading it does not generate disk I / O. During a sequential write load, the I / O is concentrated on outputting to the file; a sequential read load copies a file of a certain size from a physical partition in the file system and outputs it to / dev / null(file). write ` / dev / null` is also a pseudo-device, essentially a black hole; writing data to this device does not generate disk I / O. During a sequential read load, the I / O is concentrated in the process of reading the file system; a sequential read-write load copies a file of a certain size from one physical partition of the file system and outputs it to a file on another physical partition. Therefore, during this load, both reading and writing files will generate I / O. Lines 6-11 declare that a specified intensity of I / O operations will be performed on the disk per second.

[0057]

[0058] Disk IOPS for random operations: One random read / write operation is performed on the disk per second, and the size of each IO operation is 512 bytes of data block, so that the number of disk blocks operated during the load execution is approximately the number of disk IO operations. The specific algorithm is similar to Algorithm 1, but the method of operating the disk (the command to write to the disk in the OCCUPYIOPS function, i.e. the content between lines 2-4) is changed to call fio

[11] .

[0059] (4) Simulate the generation mechanism of network limiting and disruptive loads:

[0060] This invention can simulate special network environments, causing bandwidth limitations, network transmission delays, and non-complete network disruptions such as packet loss and corruption. This invention simulates these special network environments by directly imposing traffic limits on the network interface card (NIC). The load balancer utilizes the Linux `tc` tool to simulate network resource consumption and network failures. `tc` establishes a queue at the network output port to implement traffic control over the Linux kernel. `tc` can create traffic limits for specific NICs quickly and with minimal CPU and memory consumption during operation.

[0061] This invention can also simulate disconnecting network connections between machines. The load balancer can disconnect the network from specific ports, serving as a tool for software reliability testing. For example, if a user needs to test the fault tolerance of a database, this load balancer can be used to disconnect the database port from the network. The implementation method of this invention is to call the `tcpkill` command to terminate the corresponding TCP connection.

[0062] (5) The mechanism for generating simulated corrupted file load is as follows:

[0063] This invention simulates data file loss by forcibly executing the system's delete command to delete user-specified files or folders. In addition, this invention can also simulate scenarios where data files are maliciously modified by outputting random characters to rewrite the data files.

[0064] (6) Simulate the mechanism for consuming and disrupting process load generation:

[0065] Simulate process occupation: Simulate the occupation of process resource space by creating multiple child processes in a loop;

[0066] Simulate process termination: Invoke system commands to terminate a process, simulating the situation where an application's process is unexpectedly terminated;

[0067] Simulate process overload: exhaust system resources by continuously executing process bombs, that is, by constantly creating new processes.

[0068] (7) Mechanism for simulating host crash load generation:

[0069] This invention can simulate various scenarios where the machine hosting the database unexpectedly crashes:

[0070] Simulate a situation where the machine's power supply is interrupted, forcing all processes within the machine to terminate: by forcibly executing a command to shut down the power.

[0071] Simulate a normal machine shutdown: by executing a normal shutdown command;

[0072] Simulate a machine losing connection: by cutting off all network connections to the machine.

[0073] Step Two: Simulate dynamic changes in the external operating environment. Based on user-defined dynamic changes, create a corresponding dynamic environment. The method for simulating dynamic load changes in this invention is shown in Algorithm 2. The dynamism is mainly achieved by the variables `targetList` and `timeList` that control the execution of the simulated load in the control environment. In Algorithm 2, Workload... type This represents the environmental load that the user is currently executing. `targetList` and `timeList` are two arrays of equal length. Each value in `timeList` represents a different time period, and the corresponding value in `targetList` is the target environmental state that the load needs to simulate within that time period. This invention uses `timeList` to control the timing of load intensity changes and uses the difference (Δadjust) between adjacent target environmental states in `targetList` to control the magnitude of load intensity changes, such as in Wordload. CPU In (3 2 4, 60 3050), targetList is (3 2 4) and timeList is (60 30 50). This load will take a total of 140 seconds (60+30+50). The simulated scenario is: 3 CPU cores are used in the first 60 seconds; only 2 CPU cores are used from 60 to 90 seconds; and 4 CPU cores are used in the last 50 seconds.

[0074]

[0075] The method of this invention is implemented through a database external operating environment simulation system, which includes the following modules:

[0076] 1. Controller: Used to read information such as the current environment simulation task, the target database to be executed load, and the test report storage directory from the system configuration file, and then start the parser in sequence; it can also be used to count the success and failure of the environment simulation task; after each round of simulation, the controller will call the test report generator to automatically generate a test report.

[0077] 2. Parser: Parses each load simulation task (including the target scenario to be simulated, parameters for dynamic load changes, etc.) and the target machine to execute the load from the test case file according to the environment load writing syntax rules.

[0078] 3. Load Generator: Used to generate environmental loads based on the simulation task (including the type of load simulation, parameters of dynamic load changes, etc.) and send them to the dynamic change controller on the server.

[0079] 4. Dynamic Change Controller: Used to invoke the load executor to perform environmental load on the machine where the database is located, and to control the intensity and duration of the environmental load.

[0080] 5. Load Executor: Used to remotely log in to the target machine to execute simulated loads.

[0081] 6. Metrics Collector: This feature receives fine-grained performance metrics and system resource usage data of the database during simulated load operation from the monitor. It then formats and statistically processes the collected data to facilitate database evaluation personnel in analyzing the database's performance.

[0082] 7. Target Server: This invention will apply environmental load to the target server during environment simulation. The target server contains a DBMS (database) and a monitor. The DBMS will perform user-specified database load during environment simulation, and changes in database performance will reflect the operational quality of the database service under environmental changes. The monitor will periodically collect fine-grained performance metrics and system resource usage of the database during the simulated load operation.

[0083] 8. Test Report Generator: Used to automatically generate test reports that include environmental factors and database system performance information.

[0084] Basic architecture

[0085] The input to the database external runtime environment dynamic simulation system includes test cases containing user-defined environment simulation tasks and system configuration files; the output is the simulated environment load. The basic architecture mainly consists of eight modules: controller, parser, load generator, dynamic change controller, load executor, metric collector, target server, and test report generator. Figure 1 As shown.

[0086] 1. Controller: Used to read information such as the current environment simulation task, the target database to be executed load, and the test report storage directory from the system configuration file, and then start the parser in sequence; it can also be used to count the success and failure of the environment simulation task; after each round of simulation, the controller will call the test report generator to automatically generate a test report.

[0087] 2. Parser: This parser extracts information from the test case file, including the target scenario, dynamically changing load parameters, and the target machine to execute the load. Then, using Java reflection, it maps the load simulation tasks to Java classes that implement the specific functionalities. Finally, it calls the load generator to generate the corresponding resource consumption and resource contamination simulation loads.

[0088] 3. Load Generator: Used to generate environmental loads based on the simulation task (including the type of load simulation, parameters of dynamic load changes, etc.) and send them to the dynamic change controller on the server. The types of loads generated by the load generator include: (1) Simulating a normal operating environment: simulating the consumption of hardware resources such as CPU, memory, disk, and network, as well as software resources such as processes. (2) Simulating an extreme operating environment: simulating the full consumption and / or damage of hardware resources such as CPU, memory, disk, network, and host, as well as software resources such as data files, processes, and network connections of database ports.

[0089] 4. Dynamic Change Controller: Used to invoke the load executor to perform environmental load on the machine where the database is located, and to control the intensity and duration of the environmental load.

[0090] 5. Load Executor: Used to remotely log in to the target machine to execute simulated loads. Executors include standard environment load executors and extreme environment load executors. Standard environment load executors can simulate the state of system resource consumption by other applications in the running environment under fault-free conditions; extreme environment load executors can simulate situations where hardware resources in the system environment are fully utilized or even damaged.

[0091] 6. Metrics Collector: This feature receives fine-grained performance metrics and system resource usage data of the database during simulated load operation from the monitor. It then formats and statistically processes the collected data to facilitate database evaluation personnel in analyzing the database's performance.

[0092] 7. Target Server: This invention will apply environmental load to the target server during environment simulation. The target server contains a DBMS (database) and a monitor. The DBMS will perform user-specified database load during environment simulation, and changes in database performance will reflect the operational quality of the database service under environmental changes. The monitor will periodically collect fine-grained performance metrics and system resource usage of the database during the simulated load operation.

[0093] 8. Test Report Generator: Used to automatically generate test reports that include environmental factors and database system performance information.

[0094] Experimental conclusions

[0095] Experimental environment

[0096] Experimental hardware configuration: One physical node with CentOS 7.9 operating system, containing 8 CPUs, model Intel(R)Xeon(R)Gold 6240M CPU@2.60GHz; 32GB of memory; 130GB of storage, RAID-5, 15000 RPM HDD disks.

[0097] Experiment 1: Comparative Experiment with Related Work

[0098] The experiments simulated the use of CPU, memory, disk, and network resources respectively, demonstrating the accuracy of the simulated load of the environment designed in this invention in terms of simulated resource usage. The results were compared with the most advanced work in the field of environment simulation (the database environment simulation tool developed by the School of Data Science and Technology of East China Normal University, referred to as Chunxi in the following text) in terms of the speed of achieving the simulation goal and the degree of usage of other environmental resources.

[0099] (1) Simulate CPU usage

[0100] This experiment set the target CPU usage for simulation to 25%, 50%, 75%, and 87.5% of the total system CPU usage, respectively. Figure 2 The results show that this invention can quickly achieve stable CPU usage for a specified number of cores, even reaching 87.5% CPU utilization within 2 seconds. In contrast, Chunxi's tool initially exceeded the target CPU utilization during the simulation, then dropped back to the target utilization after a few seconds. Furthermore, the CPU utilization fluctuated slightly in the first half of the simulation, likely due to the tool running as a Java program, which incurs a small amount of additional CPU consumption during execution.

[0101] Figure 3 The impact on memory during simulated CPU load operation is demonstrated. The workload of this invention consumes less than 0.4% of memory on average. This is because the workload designed in this invention runs as a shell script, rapidly occupying the CPU by calling a specified number of single-threaded processes to perform an infinite loop operation; this method consumes almost no memory. Chunxi's program, on the other hand, runs as a Java program, continuously performing PAI calculations during its execution, using a cache to record intermediate calculation results. Therefore, the more CPU cores required, the more significantly the workload consumes memory, even reaching approximately 50% memory usage during 87.5% CPU utilization. Both tools have almost no impact on disk I / O during operation.

[0102] (2) Simulate memory usage

[0103] This experiment simulated scenarios where memory usage was 25%, 50%, 75%, and 87.5%. Figure 4 As shown, when the workload of this invention is implemented with memory usage, setting the parameter $bs to 512K achieves the target with high efficiency. Even with 87.5% memory usage, it only takes 8 seconds, which is slightly faster than Chunxi's workload (which takes 11 seconds). Figure 5This reflects the average CPU impact during the execution of the simulated memory usage workload. This workload uses system commands to fill memory, resulting in a small CPU consumption on the system kernel during this period; however, once memory is filled, it remains in a non-released state, consuming no CPU during this time. Therefore, the average CPU consumption during this workload's execution is very low; even with 87.5% memory usage, the average CPU consumption is less than 3%. Chunxi's memory usage tool, on the other hand, runs as a C program, consuming both user-mode and system kernel CPU during memory filling; furthermore, the workload actively operates on memory during this period, leading to significant CPU consumption, exceeding 10% even with only 25% memory usage. Both tools occupy memory by directly writing data to it, resulting in almost no disk resource consumption during their execution.

[0104] (3) Simulate disk usage

[0105] This experiment simulated scenarios with disk IOPS usage of 20%, 40%, 60%, and 90%. Figure 6 As shown, the workload of this invention can accurately simulate 20%, 40%, and 60% of disk IOPS. When simulating 90% disk IOPS, the simulation effect is not very accurate at the beginning, perhaps because the dense disk operations themselves put pressure on the disk, causing fluctuations in the time of each IO operation. However, the simulated IOPS becomes stable in the middle and later stages and meets the simulation target. Since this workload directly operates on the disk, the tool consumes almost no memory during operation. Chunxi's workload will consume a certain amount of IOPS, but because this workload cannot guarantee that the operation on each data block is completed in a single IO, and the workload design does not avoid memory usage, the workload's IOPS consumption for the specified intensity is completely inaccurate. Therefore, this tool is not suitable for simulating disk IOPS.

[0106] Figure 7 and Figure 8 The figures show the average CPU utilization and memory utilization during the IOPS period. This invention avoids using file system caching during operation, resulting in minimal CPU consumption and almost no memory usage. In contrast, Chunxi's tools heavily utilize memory during operation, triggering numerous system calls and consequently leading to higher CPU utilization.

[0107] (4) Simulated network environment

[0108] Both this work and Chunxi's work can simulate network environments, but they are implemented in different ways: this work limits network bandwidth by directly controlling the machine's network card; while the comparative work uses a Java program to maintain data transmission between two servers to occupy network bandwidth. To more intuitively compare the impact of the two tools on network bandwidth, before running this invention, data transmission was maintained between the experimental server and another server to facilitate the monitoring tool recording changes in network bandwidth after the tool started; while when using Chunxi's tool, the monitoring tool directly recorded changes in network bandwidth within the system.

[0109] The experiment compared the changes in network bandwidth during the operation of the two tools to reflect their effects on network bandwidth usage. Figure 9 This demonstrates how network bandwidth changes in the system when affected by 20%, 40%, 60%, and 90% of network bandwidth. This tool achieves the target environment quickly by directly limiting the system's network interface card (NIC), reaching the simulated target value within 2 seconds and simulating a stable network environment. Chunxi's tool consumes network bandwidth by continuously transferring data between two servers. This program uses Netty technology, requiring a short time for communication to be established between the two servers. During the tool's execution, the amount of network bandwidth used fluctuated significantly. This may be because the tool encountered channel congestion during data transfer between servers, causing some data to be unable to be transmitted in time.

[0110] Figure 10 This reflects the CPU usage changes in the system during runtime load testing. This invention directly limits the system's network interface card (NIC), a method that has almost no impact on the use of other hardware resources. Since data is transmitted between two servers during this load test, which inherently incurs slight CPU consumption, the CPU usage when running this tool alone will be lower than the usage recorded in the graph. Chunxi's tool, on the other hand, uses a Java program that calls Netty technology to manage network resources. This load requires intensive calculation of the data size to be transmitted during operation, thus the network simulation workload has a significant impact on CPU usage. Both tools have almost no impact on disk I / O during operation.

[0111] Experimental data shows that, compared with related work, this invention can more accurately and efficiently simulate the target environment.

[0112] Experiment 2: Simulation of Dynamically Changing Environments

[0113] The simulation task was as follows: A simulated load was executed for 120 seconds. In the first 30 seconds, the simulated load was 50% CPU, 60% disk IOPS, and 80% network bandwidth. From 30 to 60 seconds, it was 87.5% CPU, 90% disk IOPS, and 40% network bandwidth. From 60 to 90 seconds, it was 25% CPU, 40% disk IOPS, and 60% network bandwidth. From 90 to 120 seconds, it was 62.5% CPU, 20% disk IOPS, and 10% network bandwidth. The deviation between the simulated actual environment and the target environment, as well as the effective time of different environments, were observed.

[0114]

[0115]

[0116] CPU usage 25％ 50％ 62.5％ 87.5％ Effective time (s) 2 2 2 2

[0117] Disk IOPS usage 20％ 40％ 60％ 90％ Effective time (s) 1 2 2 3

[0118] Uses network bandwidth 10％ 40％ 60％ 80％ Effective time (s) 1 1 1 1

[0119] Experimental data show that the effective time of this invention is almost consistent under different occupancy ratios, and it can accurately, efficiently and smoothly simulate dynamically changing environments.

[0120] References

[0121] [1]How Many Companies Use Cloud Computing in 2022? [EB / OL].https: / / techjury.net / blog / how-many-companies-use-cloud-computing.

[0122] [2]Containerization[EB / OL].https: / / www.ibm.com / cloud / learn / containerization.

[0123] [3]Kubernetes[EB / OL].https: / / kubernetes.io / .

[0124] [4]Ma M,Yin Z,Zhang S,et al.Diagnosing root causes of intermittentslow queries in cloud databases[J].Proceedings of the VLDB Endowment,2020,13(8):1176-1189.

[0125] [5]Merenstein A,Tarasov V,Anwar A,et al.CNSBench:A Cloud NativeStorage Benchmark[J].FAST,2021:263-275.

[0126] [6]Woodpecker[EB / OL].https: / / github.com / DBHammer / Woodpecker.

[0127] [7]chaosblade[EB / OL].https: / / github.com / chaosblade-io / chaosblade.

[0128] [8]chaos-mesh[EB / OL].https: / / github.com / chaos-mesh / chaos-mesh.

[0129] [9]JEONG H,LEE S.A workload generator for database system benchmarks[C] / / International Conference on Information Integration and Web-basedApplications&Services(iiWAS):Citeseer,2005:813-822.

[0130]

[10] Zhang C,Zhang R,Su Q,et al.Dynamic Environment Simulation forDatabase Performance Evaluation[J].APWeb-WAIM,2021:180-189.

[0131]

[11] fio[EB / OL].https: / / github.com / axboe / fio.

[0132] The scope of protection of this invention is not limited to the above embodiments. Any variations and advantages that can be conceived by those skilled in the art without departing from the spirit and scope of the inventive concept are included in this invention and are protected by the appended claims.

Claims

1. A method for dynamically simulating the external operating environment of a database, characterized in that, Includes the following steps: Step 1: Simulate various external operating environments, generate resource simulation loads of specified type and intensity based on user-defined target scenarios, and then run the loads to create the target environment; In step one, the simulation of various external operating environments includes: Simulate a normal operating environment: Simulate the state of other applications consuming system software and hardware resources in the operating environment under fault-free conditions; the hardware resources include CPU, memory, disk, and network, and the software resources include processes; Simulate extreme operating environments: Simulate situations where the hardware and software resources of the machine hosting the database are fully occupied and / or damaged in the operating environment; the hardware resources that are fully occupied and / or damaged include CPU, memory, disk, network, and host, and the software resources include data files, processes, and network connections of the database port; In simulating hardware resources, the method for simulating CPU usage is to allocate a single-threaded process equal to the number of CPU cores required. Under certain conditions, the assignment operation is executed an infinite number of times, thereby consuming the corresponding number of CPU cores; The method to simulate memory usage is to create a tmpfs file system and then write data of disk size $data_size into it to occupy memory. The method to simulate disk usage is to write data equal to the required disk space $data_size into the file system, thereby achieving the purpose of occupying disk space. The method to simulate network usage is to call the Linux tc tool to simulate network resource consumption and network failures; Methods for simulating host occupation include: simulating a situation where the machine's power supply is disrupted and all processes within the machine are forced to terminate by forcibly executing a power-off command; simulating a situation where the machine is shut down normally by executing a normal shutdown command; and simulating a situation where the machine is disconnected by cutting off all network connections. In the simulation software resources, Simulation of data files includes rewriting the data file by outputting random characters to simulate malicious tampering; and deleting user-specified files or folders by forcibly executing the system's delete command to simulate data file loss. Process simulation includes creating multiple child processes in a loop to simulate occupying process resource space; calling system commands to terminate processes to simulate the unexpected termination of an application process; and continuously executing process bombs to simulate the situation where the process is completely occupied. Simulating network connections to the database port includes calling the tcpkill command to terminate the TCP connection corresponding to the database port. Step 2: Simulate the dynamic changes of the external operating environment and create a corresponding dynamic environment based on the user-defined dynamic changes.

2. The method for dynamically simulating the external operating environment of a database as described in claim 1, characterized in that, In step two, the dynamic changes include the intensity and duration of the simulated load for each resource.

3. A dynamic simulation system for an external database operating environment, used to implement the dynamic simulation method as described in claim 1 or 2, characterized in that, The dynamic simulation system comprises two parts: a client and a server. The client includes a controller, a parser, a load generator, and a test report generator. The server includes a dynamic change controller, a load executor, an indicator collector, and a target server.

4. The database external operating environment dynamic simulation system according to claim 3, characterized in that, In the client, the controller is used to identify information from the system configuration file, including the current environment simulation task, the target database for the load to be executed, and the test report storage directory, and then start the parser in sequence; and to count the success and failure of the environment simulation task; after each round of simulation, the test report generator is called to automatically generate a test report; The parser uses the defined syntax rules for writing environment loads to parse information from the test case file, including each load simulation task and the target machine to execute the load; then it uses Java's reflection mechanism to map the load simulation tasks to Java classes that implement specific functions. The load generator is used to generate environmental load according to the load simulation task sent by the parser, and then send the environmental load to the dynamic change controller on the server side. The test report generator is used to automatically output test reports that include environmental factors and database system performance information.

5. The database external operating environment dynamic simulation system according to claim 4, characterized in that, The types of loads generated by the load generator include: Simulates a normal operating environment: Simulates the consumption of hardware resources, including CPU, memory, disk, and network, as well as software resources, including processes; Simulate extreme operating environments: Simulate the full utilization and / or destruction of hardware resources, including CPU, memory, disk, network, and host, as well as software resources, including data files, processes, database ports, and network connections.

6. The database external operating environment dynamic simulation system according to claim 3, characterized in that, On the server side, the dynamic change controller is used to call the load executor to perform environmental load on the machine where the database is located, and to control the intensity and duration of the environmental load. The load executor performs environmental load on the target server; it includes a normal environment load executor and an extreme environment load executor; the normal environment load executor is used to simulate the state of other applications consuming system hardware and software resources in the system environment under fault-free conditions; the extreme environment load executor is used to simulate the extreme situation where the hardware and software resources of the machine where the database is located in the running environment are fully occupied and / or damaged; The target server contains a database management system (DBMS) and a monitor. The DBMS will execute user-specified database load during the environment simulation, and changes in database performance will reflect the operational quality of the database service when the environment changes. The monitor will periodically collect fine-grained performance indicators and system resource usage of the database during the simulated load operation. The metric collector is used to receive fine-grained performance metrics and system resource usage of the database during simulated load operation returned by the monitor, and to format and statistically process the collected performance metrics, so that database evaluation personnel can analyze the database operation based on the performance data.

Images