RAID fault simulation method and device

Abstract

The embodiment of the invention provides an RAID fault simulation method and device, relates to the technical field of computer system structure storage and can also be used for the financial field, and the method comprises the steps that an RAID array of a corresponding level is constructed based on the preset number of RAID array hard disks and a hard disk list forming an RAID; performing parameter control on the constructed RAID array to construct a corresponding fault model, wherein the fault type of the fault model comprises at least one of a hard disk type fault, a control and configuration type fault, a logic fault and a read-write slow fault; determining a corresponding fault injection instruction and a fault recovery instruction according to the fault model, and inputting the instructions to a tested system, so that the tested system performs fault simulation; the RAID fault simulation method and device can solve the problems that the current RAID fault simulation cost is high, and hardware dependence, professional software dependence and manual operation dependence exist.

Description

Technical Field

[0001] The present invention relates to the technical field of computer system structure storage, and can also be used in the financial field, and in particular to a RAID fault simulation method and device. Background Art

[0002] RAID (Redundant Array of Independent Disks) is a technology that combines multiple hard disk drives into a virtualized logical unit for managing and storing data. Its core goal is to improve the system's data storage performance, reliability, and fault tolerance by combining multiple hard disks. Specifically, RAID can avoid data loss caused by a single hard disk failure by distributing data to multiple hard disks, while also improving data reading and writing speeds.

[0003] RAID has many different configuration levels, and each level of design has its own unique characteristics. The choice of different RAID levels depends on the user's different needs for data security, performance, and storage capacity.

[0004] In enterprise-level storage, the occurrence and recovery time of RAID failures will affect the performance and availability of important systems, especially for financial enterprises involved in transactions, where RAID failures may lead to data loss, transaction consistency, etc. It is necessary to simulate RAID failures on important systems to test their robustness and availability.

[0005] However, existing RAID simulators (such as Intel's RAID simulator and Dell's array manager) have some limitations in practical applications. First, these tools often rely on specific hardware and operating systems. For example, Intel's RAID simulator can only run on Intel servers, while Dell's array manager is only applicable to Dell servers. In addition, these tools are also limited in functionality. For example, they only support basic operations such as creating or deleting RAID volumes and resetting hard drives, but cannot simulate more complex scenarios, such as hard drive failures in RAID arrays, slow data transmission, or write failures. This limitation makes it difficult to fully test various failure scenarios of RAID systems during actual use.

[0006] This section is intended to provide a background or context to the embodiments of the invention recited in the claims. No description herein is admitted to be prior art by inclusion in this section. Summary of the invention

[0007] In view of the problems in the prior art, the present application provides a RAID fault simulation method and device, which can solve the current problems of high cost of simulating RAID faults, hardware dependence, professional software dependence and manual operation dependence.

[0008] In order to solve at least one of the above problems, the present application provides the following technical solutions:

[0009] According to a first aspect of an embodiment of the present application, the present application provides a RAID failure simulation method, comprising:

[0010] Build a RAID array of the corresponding level based on the preset number of RAID array hard disks and the list of hard disks that make up the RAID;

[0011] Performing parameter control on the constructed RAID array to construct a corresponding fault model, wherein the fault type of the fault model includes at least one of a hard disk fault, a control and configuration fault, a logic fault, and a slow read / write fault;

[0012] Corresponding fault injection instructions and fault recovery instructions are determined according to the fault model and input into the system under test, so that the system under test performs fault simulation.

[0013] According to any implementation manner of the present application, the performing parameter control on the constructed RAID array to construct a corresponding fault model includes:

[0014] Determine the hard disk to be failed according to the type and number of hard disks of the RAID array;

[0015] Removing the hard disk to be failed to simulate a hard disk failure;

[0016] Add the removed hard disk to the RAID array to achieve fault recovery.

[0017] According to any implementation manner of the present application, the performing parameter control on the constructed RAID array to construct a corresponding fault model includes:

[0018] Changing the configuration level of the RAID array to an error level to simulate a RAID configuration failure;

[0019] Restoring the configuration level of the RAID array to a correct level to achieve RAID configuration recovery;

[0020] Interrupting the hard disk network of the RAID array to simulate a RAID access interruption failure;

[0021] Restoring the hard disk network of the RAID array to achieve RAID access recovery;

[0022] Injecting consistent failure parameters into the hard disks of the RAID array to simulate a RAID data consistency failure;

[0023] Resetting consistency fault parameters in the RAID array hard disk to achieve RAID data consistency recovery;

[0024] disabling the RAID array to simulate a RAID power outage failure;

[0025] Restarting the RAID array to implement RAID power outage recovery.

[0026] According to any implementation manner of the present application, the performing parameter control on the constructed RAID array to construct a corresponding fault model includes:

[0027] Random data is written to the check block of the RAID array to simulate a RAID check information failure.

[0028] According to any implementation manner of the present application, the performing parameter control on the constructed RAID array to construct a corresponding fault model includes:

[0029] Limit the speed of the hard disk network of the RAID array, reduce the network bandwidth, or control the disk read and write speed to simulate a slow RAID write failure;

[0030] The hard disk network of the RAID array is released from speed limit, the network bandwidth is reduced, or the disk read and write speed is controlled to achieve RAID write recovery.

[0031] According to any embodiment of the present application, determining the corresponding fault injection instruction and fault recovery instruction according to the fault model and inputting them into the system under test so that the system under test performs fault simulation includes:

[0032] Determine corresponding fault injection instructions and fault recovery instructions according to the fault model;

[0033] The agent is designed by using bytecode enhancement technology so that the fault injection instruction and the fault recovery instruction can be received and parsed and executed by the system under test.

[0034] According to any implementation manner of the present application, before determining the corresponding fault injection instruction and fault recovery instruction according to the fault model and inputting them into the system under test so that the system under test performs fault simulation, the method further includes:

[0035] Perform task orchestration on the fault model based on an orchestration engine, so that the fault model is in at least one orchestration state of task serialization, task parallelization, and task suspension.

[0036] According to any embodiment of the present application, after determining corresponding fault injection instructions and fault recovery instructions according to the fault model and inputting them into the system under test so that the system under test performs fault simulation, the method further includes:

[0037] The fault response speed and fault recovery capability of the system under test are recorded according to the fault simulation result of the system under test.

[0038] According to a second aspect of an embodiment of the present application, the present application provides a RAID fault simulation device, comprising:

[0039] An array construction module, used to construct a RAID array of a corresponding level based on a preset number of RAID array hard disks and a list of hard disks constituting the RAID;

[0040] A fault design module, used to control parameters of the constructed RAID array to construct a corresponding fault model, wherein the fault type of the fault model includes at least one of a hard disk fault, a control and configuration fault, a logic fault, and a slow read and write fault;

[0041] The fault injection module is used to determine corresponding fault injection instructions and fault recovery instructions according to the fault model and input them into the system under test so that the system under test performs fault simulation.

[0042] According to any implementation manner of the present application, the fault design module includes a hard disk fault design unit, which is used to:

[0043] Determine the hard disk to be failed according to the type and number of hard disks of the RAID array;

[0044] Removing the hard disk to be failed to simulate a hard disk failure;

[0045] Add the removed hard disk to the RAID array to achieve fault recovery.

[0046] According to any implementation of the present application, the fault design module includes a control and configuration fault design unit, which is used to:

[0047] Changing the configuration level of the RAID array to an error level to simulate a RAID configuration failure;

[0048] Restoring the configuration level of the RAID array to a correct level to achieve RAID configuration recovery;

[0049] Interrupting the hard disk network of the RAID array to simulate a RAID access interruption failure;

[0050] Restoring the hard disk network of the RAID array to achieve RAID access recovery;

[0051] Injecting consistent failure parameters into the hard disks of the RAID array to simulate a RAID data consistency failure;

[0052] Resetting consistency fault parameters in the RAID array hard disk to achieve RAID data consistency recovery;

[0053] disabling the RAID array to simulate a RAID power outage failure;

[0054] Restarting the RAID array to implement RAID power outage recovery.

[0055] According to any implementation manner of the present application, the fault design module includes a logic fault design unit, which is used to:

[0056] Random data is written to the check block of the RAID array to simulate a RAID check information failure.

[0057] According to any embodiment of the present application, the fault design module includes a slow read / write fault design unit, which is used to: limit the speed of the hard disk network of the RAID array, reduce the network bandwidth, or control the disk read / write speed to simulate a slow RAID write fault;

[0058] The hard disk network of the RAID array is released from speed limit, the network bandwidth is reduced, or the disk read and write speed is controlled to achieve RAID write recovery.

[0059] According to any implementation of the present application, the fault injection module includes:

[0060] An instruction framework unit, used to determine corresponding fault injection instructions and fault recovery instructions according to the fault model;

[0061] The instruction injection unit is used to design an agent through bytecode enhancement technology so that the fault injection instruction and the fault recovery instruction can be received and parsed and executed by the system under test.

[0062] According to any implementation manner of the present application, it also includes a fault orchestration module, which is used to perform task orchestration on the fault model based on an orchestration engine so that the fault model is in at least one orchestration state of task serialization, task parallelism, and task suspension.

[0063] According to any embodiment of the present application, it also includes an information recording module, which is used to record the fault response speed and fault recovery capability of the system under test according to the fault simulation result of the system under test.

[0064] According to a third aspect of an embodiment of the present application, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the RAID fault simulation method when executing the program.

[0065] According to a fourth aspect of the embodiments of the present application, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the RAID failure simulation method are implemented.

[0066] According to a fifth aspect of the embodiments of the present application, the present application provides a computer program product, including a computer program / instruction, which implements the steps of the RAID failure simulation method when executed by a processor.

[0067] It can be seen from the above technical scheme that the present application provides a RAID fault simulation method and device, which constructs a RAID array of a corresponding level based on a preset number of RAID array hard disks and a list of hard disks that constitute the RAID; performs parameter control on the constructed RAID array to construct a corresponding fault model, and the fault type of the fault model includes at least one of hard disk faults, control and configuration faults, logical faults, and slow reading and writing faults; determines corresponding fault injection instructions and fault recovery instructions according to the fault model and inputs them into a system under test, so that the system under test performs fault simulation; and can solve the problems of high cost, hardware dependence, professional software dependence, and manual operation dependence in the current simulation of RAID faults. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0069] Figure 1 This is one of the flow charts of the RAID failure simulation method in the embodiment of the present application;

[0070] Figure 2 This is one of the schematic diagrams of the RAID fault simulation structure in the embodiment of the present application;

[0071] Figure 3 This is a second flow chart of the RAID fault simulation method in the embodiment of the present application;

[0072] Figure 4 The third flowchart of the RAID fault simulation method in the embodiment of the present application;

[0073] Figure 5 This is a fourth flow chart of the RAID fault simulation method in the embodiment of the present application;

[0074] Figure 6 FIG5 is a flowchart of a RAID fault simulation method in an embodiment of the present application;

[0075] Figure 7 This is the second schematic diagram of the RAID fault simulation structure in the embodiment of the present application;

[0076] Figure 8 This is the third schematic diagram of the RAID fault simulation structure in the embodiment of the present application;

[0077] Fig. 9 is a structural diagram of a RAID fault simulation device in an embodiment of the present application;

[0078] Fig.10 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0080] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0081] The acquisition, transmission, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0082] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0083] In order to solve the current problems of high cost of simulating RAID failures, hardware dependence, professional software dependence, and manual operation dependence, the present application provides an embodiment of a RAID failure simulation method, see Figure 1 The RAID failure simulation method specifically includes the following contents:

[0084] Step S01: constructing a RAID array of a corresponding level based on the preset number of hard disks in the RAID array and a list of hard disks constituting the RAID.

[0085] First, based on the preset number of hard disks and hard disk list, create an array corresponding to the specified RAID level. There are different levels of RAID, such as RAID 0, RAID 1, or RAID 5, and each configuration has its own specific advantages and uses. In this step, by selecting the appropriate number of hard disks and RAID level, a virtual storage system that can perform data storage and fault tolerance is constructed to simulate the data distribution and fault performance in the subsequent fault simulation.

[0086] For example, when creating a RAID array, you first need to specify the number of hard disks included in the RAID array and the specific list of hard disks that make up the RAID array. You can use the mdadm tool in the Linux system to complete this operation. The specific steps are as follows:

[0087] Use the mdadm --create command to create a RAID array and specify the RAID device path, such as / dev / md5;

[0088] --level=X parameter is used to specify the RAID level, such as RAID 0, RAID 1, or RAID 5, which represent different data storage methods (striping, mirroring, or striping with parity).

[0089] --raid-devices=X parameter is used to specify the number of hard disks that make up the RAID array. For example, 3 means using three hard disks.

[0090] The hard disk list, such as disk1, disk2, etc., represents the hard disk paths actually used in the RAID array. For example, / dev / disk1 represents the first hard disk.

[0091] Step S02: Controlling parameters of the constructed RAID array to construct a corresponding fault model, wherein the fault type of the fault model includes at least one of a hard disk fault, a control and configuration fault, a logic fault, and a slow read / write fault.

[0092] After the RAID array is built, different fault models are created by adjusting the system parameters. The fault model can simulate various types of faults, including hard disk faults (such as hard disk damage or failure), control and configuration faults (such as RAID controller failure, network interruption), logical faults (such as data inconsistency or verification errors), and slow reading and writing problems. Through the fault model, the system can predict possible fault scenarios in advance and perform necessary tests and verifications.

[0093] Step S03: determining corresponding fault injection instructions and fault recovery instructions according to the fault model and inputting them into the system under test, so that the system under test performs fault simulation.

[0094] Among them, the corresponding fault injection and recovery instructions can be generated according to the previously created fault model, and these instructions can be passed to the system under test. Specifically, the fault injection instructions simulate the problems that may occur in actual operation (such as disconnecting the hard disk or limiting the read and write speed), while the fault recovery instructions ensure that the system can be restored to a normal state after the failure. Through this operation, the RAID system is tested in a real environment. Its performance when a fault occurs, thereby verifying the system's fault tolerance, data integrity and recovery capabilities.

[0095] In an optional embodiment, if Figure 2 As shown, taking RAID5 as an example, the RAID fault simulation structure of the present application includes the top RAID fault model, which is used to define various possible RAID fault types, including single disk fault, double disk fault, multiple disk fault, controller fault, configuration fault, power supply fault, and slow writing or reading, etc. These fault models provide a basis for fault simulation.

[0096] Under the fault model are the agent and sidecar modules, which are responsible for issuing and executing fault injection and recovery instructions in the system under test. This module is designed with bytecode enhancement technology. It can parse fault instructions and perform RAID fault simulation tasks through resources such as the network, file system, and disk. This part is also responsible for injecting these instructions into the system under test to complete fault simulation and recovery.

[0097] On the right side of the fault model is the fault orchestration module, which can orchestrate multiple faults in series or in parallel as needed. Basic and complex orchestrations make the order and combination of faults more flexible, simulating multiple faults that are closer to the real environment. Through this module, you can simulate complex scenarios where multiple faults occur simultaneously or sequentially.

[0098] In the middle part, Figure 2 A RAID 5 array is shown, which includes three hard disks (Disk 1, Disk 2, Disk 3) and stores different data blocks and parity information (such as A1, A2, Ap, etc.) on them. RAID 5 uses parity to provide fault tolerance, allowing data to be recovered in the event of a hard disk failure. This application simulates the failure of the hard disks and parity data in the RAID array to evaluate the performance and recovery capabilities of the RAID system under different failures.

[0099] Figure 2The right side shows the application under test, i.e. the target system for fault simulation. For example, the application includes different types of business systems such as distributed services, distributed databases, cloud computing, and big data components. In these applications, multiple Agent modules are deployed to inject fault simulation instructions into the actual running system, thereby evaluating the fault tolerance of the system in a real environment. Figure 2 The bottom shows the infrastructure on which the system depends, including containers, physical machines, and virtual machines. These facilities provide underlying support for the operation of the RAID system and fault simulation device.

[0100] It can be seen from the above description that the RAID fault simulation method provided in the embodiment of the present application can solve the current problems of high cost of simulating RAID faults, hardware dependence, professional software dependence, and manual operation dependence.

[0101] In one embodiment of the RAID fault simulation method of the present application, see Figure 3 The parameter control of the constructed RAID array to construct a corresponding fault model may also specifically include the following contents:

[0102] Step S02A1: Determine the hard disk to be failed according to the type of the RAID array and the number of hard disks.

[0103] Based on the constructed RAID array, determine which hard disk or hard disks will be set to fail, such as single disk failure, double disk failure, or multiple disk failure. The type of RAID array (such as RAID 0, RAID 1, RAID 5, etc.) and the specific number of hard disks determine the fault tolerance and redundancy design of the system. For example, in RAID 5, one or more hard disks can be selected for failure simulation because RAID 5 allows a single hard disk failure without data loss.

[0104] Step S02A2: Remove the hard disk to be faulty to simulate a hard disk failure.

[0105] Simulates a drive failure or failure scenario by removing selected drives from the RAID array. This test the system's fault tolerance in the event of a drive failure and whether it can continue to read or write data. For example, by removing a drive, the system enters degraded mode and continues to operate with the remaining drives and parity information.

[0106] Step S02A3: Add the removed hard disk to the RAID array to achieve fault recovery.

[0107] After the simulated failure is complete, the removed hard drive is added back to the RAID array to simulate the system's automatic recovery process after the hard drive is replaced. The system recovers to normal operation by rebuilding data or parity information. This step verifies the automatic repair capability of the RAID array, ensuring that data can be successfully recovered after a failure and the system can resume normal operation.

[0108] Through the fault model design in the above steps, the fault tolerance and data recovery capabilities of the RAID array to hard disk failures can be fully tested.

[0109] In one embodiment of the RAID fault simulation method of the present application, see Figure 4 The parameter control of the constructed RAID array to construct a corresponding fault model may also specifically include the following contents:

[0110] Step S02B1: changing the configuration level of the RAID array to an error level to simulate a RAID configuration failure;

[0111] Step S02B2: Restoring the configuration level of the RAID array to a correct level to achieve RAID configuration recovery;

[0112] Step S02B3: interrupting the hard disk network of the RAID array to simulate a RAID access interruption failure;

[0113] Step S02B4: restoring the hard disk network of the RAID array to achieve RAID access recovery;

[0114] Step S02B5: injecting consistent failure parameters into the hard disks of the RAID array to simulate a RAID data consistency failure;

[0115] Step S02B6: resetting the consistency fault parameters in the RAID array hard disk to achieve RAID data consistency recovery;

[0116] Step S02B7: disabling the RAID array to simulate a RAID power outage failure;

[0117] Step S02B8: Restart the RAID array to implement RAID power failure recovery.

[0118] First, in the simulation of RAID configuration failure, by changing the configuration level of the RAID array to an incorrect configuration, the system will enter an abnormal operating state and cannot perform data operations correctly. This fault simulation can be achieved by changing the RAID configuration parameters (such as creating an incorrect RAID level). After the system detects a configuration error, it can restore the RAID array to the correct configuration level to return the system to a normal state. This recovery operation ensures that the RAID system can be readjusted and restored to a normal operating mode when the configuration is wrong.

[0119] Secondly, the simulation of RAID access interruption failure is achieved by disconnecting the network connection of the hard disk, simulating the situation where the system cannot access the storage device when the network is interrupted. This operation can be achieved through network commands (such as blocking I / O traffic). When the RAID array is inaccessible, the system's data operations will be affected. This simulation can test the system's fault tolerance when the network or storage connection is interrupted. Once the network connection is restored, the RAID system should be able to access the storage device normally, thereby achieving access recovery.

[0120] In the simulation of RAID data consistency failure, the data inconsistency or verification error on the hard disk is simulated by injecting fault parameters. For example, the RAID system can find problems during data verification by adjusting the consistency check parameters or injecting errors. This type of failure can verify the performance and recovery ability of the system in the case of data inconsistency. When the failure is recovered, the consistency failure parameters are reset, and the system will re-verify the data to ensure that the data consistency of all hard disks is restored.

[0121] Finally, the simulation of RAID power outage failure is achieved by disabling the RAID array, simulating a scenario where the system loses power or the RAID device loses power. In this case, all operations of the RAID array will stop, and the system must deal with the risk of failure caused by power outage. When power is restored, the system will restart the RAID array and ensure data integrity and normal operation of the system through automatic recovery operations. This process tests the recovery ability and stability of the RAID system in the event of power outage and recovery.

[0122] Through these simulation operations, the RAID system can be tested under different failure scenarios.

[0123] In one embodiment of the RAID fault simulation method of the present application, the parameter control of the constructed RAID array to construct a corresponding fault model may also specifically include the following contents:

[0124] Random data is written to the check block of the RAID array to simulate a RAID check information failure.

[0125] The above steps simulate RAID checksum information failure by writing random data to the checksum block of the RAID array. First, use the RAID management tool (such as mdadm) to view the detailed information of the RAID array and find the location of the checksum block. Then, by writing random data to this location, the checksum information is deliberately destroyed to simulate the scenario of checksum information damage. This operation can test the system's fault tolerance and fault recovery mechanism when the checksum information is wrong. Finally, evaluate whether the checksum error can be detected and the data can be successfully recovered, thereby verifying the robustness and reliability of the RAID system in the case of checksum information failure.

[0126] In one embodiment of the RAID fault simulation method of the present application, see Figure 5 The parameter control of the constructed RAID array to construct a corresponding fault model may also specifically include the following contents:

[0127] Step S02C1: limiting the speed of the hard disk network of the RAID array, reducing the network bandwidth or controlling the disk read and write speed to simulate a RAID slow write failure.

[0128] Step S02C2: Release the speed limit of the hard disk network of the RAID array, reduce the network bandwidth or control the disk read and write speed to achieve RAID write recovery.

[0129] The RAID slow write failure is simulated by limiting the network speed of the RAID array, reducing the network bandwidth, or controlling the read and write speed of the disk.

[0130] First, as an example, you can use the cgroup and tc tools in the Linux system. By creating a control group (cgroup) and limiting the I / O speed to 1MB / s, you can effectively simulate the write speed drop scenario in the RAID array. Simulate the response of the RAID system when the read and write speed of the disk slows down due to network or system bottlenecks.

[0131] In addition to controlling the disk I / O speed, you can also limit the output bandwidth of the network interface to 1MB / s to simulate the impact of network bottlenecks on RAID write speed, so as to evaluate the system's fault tolerance and data transmission stability when the network bandwidth is limited. By adjusting the network speed or bandwidth, the fault model can reflect data write delay problems caused by network delays or insufficient bandwidth.

[0132] Once the slow write failure simulation is completed, the write speed of the RAID system can be restored to normal levels by removing the I / O speed limit and network bandwidth limit. At this point, the RAID system should be able to write data at normal speed again to verify the system's performance when recovering from the failure.

[0133] In one embodiment of the RAID fault simulation method of the present application, see Figure 6 , the determining of the corresponding fault injection instruction and fault recovery instruction according to the fault model and inputting them into the system under test so that the system under test performs fault simulation may also specifically include the following contents:

[0134] Step S03A: Determine corresponding fault injection instructions and fault recovery instructions according to the fault model.

[0135] Step S03B: Designing an agent through bytecode enhancement technology enables the fault injection instruction and the fault recovery instruction to be received, parsed and executed by the system under test.

[0136] First, if Figure 7 As shown in the figure, according to the established RAID fault model, the corresponding fault injection instructions and recovery instructions are generated. Specifically, the fault model uses network resources, file systems or disk resources to generate a set of fault injection instructions based on the characteristics of different types of RAID faults (such as hard disk failure, network interruption, checksum error, etc.) to simulate real fault scenarios. The corresponding recovery instructions ensure that the system can perform automatic or manual recovery operations after a fault occurs.

[0137] Secondly, the agent is enabled to receive and parse fault instructions through bytecode enhancement technology. The role of the bytecode enhancement technology is to enhance the system's functionality without changing the system's own code, ensuring that the agent can intercept and process the fault injection instructions and recovery instructions passed to it.

[0138] When the agent is installed on the system under test, it will receive and parse the previously generated fault injection instructions and recovery instructions. Through these instructions, the agent can control the key resources of the system under test and perform specific fault simulation tasks.

[0139] The above steps ensure the accuracy and controllability of fault simulation. At the same time, through the combination of agent program and bytecode enhancement technology, the system can easily perform fault injection and recovery, thereby improving the fault tolerance and recovery capability test effect of the RAID system.

[0140] In one embodiment of the RAID fault simulation method of the present application, Figure 8 As shown, before determining the corresponding fault injection instruction and fault recovery instruction according to the fault model and inputting them into the system under test so that the system under test performs fault simulation, the following contents may also be specifically included:

[0141] Perform task orchestration on the fault model based on an orchestration engine, so that the fault model is in at least one orchestration state of task serialization, task parallelization, and task suspension.

[0142] Among them, the orchestration engine can have multiple functions for different types of RAID failures, such as serial failures, parallel failures, and failure hangs. In actual environments, failures usually do not occur alone, and one failure may trigger a chain reaction of others. For example, a disk failure may cause the network transmission speed to slow down, or a controller problem may cause data writing failure. Through the orchestration engine, users can simulate these complex multi-failure scenarios and observe how the system performs under different combinations of failures.

[0143] In the serial failure mode, the orchestration engine allows failures to occur one after another in a predetermined order. For example, after the first hard drive failure, network delay or controller failure failures are triggered. Through this serial mode, it is possible to observe the system's ability to cope with new failures before a failure is recovered or not fully recovered.

[0144] The parallel failure mode allows multiple failures to occur simultaneously, simulating the system's performance when facing multiple different failures (such as multiple hard disk failures or network interruptions and data inconsistencies occurring simultaneously). This parallel failure scenario is usually closer to the situation encountered by complex systems in the real world, thus helping developers verify the fault tolerance of the system under high pressure.

[0145] The suspend function provides greater flexibility for the occurrence of failures. For example, the orchestration engine can be used to suspend the occurrence of a certain failure, and then resume the simulation of the failure after the system recovers some or all functions. This suspend function can be used to analyze the performance of the system during the gradual recovery process and verify its ability to handle complex scenarios.

[0146] Through the orchestration engine, this application can accurately control the order, time points and combinations of different RAID failures, thereby building a realistic test environment. By simulating complex scenarios of multiple failures through serial, parallel and suspended operations, it can not only evaluate the stability of the system, but also help developers find potential bottlenecks or failure chains in the system, and improve the fault tolerance and recovery capabilities of the system in a real environment.

[0147] In one embodiment of the RAID fault simulation method of the present application, after determining the corresponding fault injection instruction and fault recovery instruction according to the fault model and inputting them into the system under test so that the system under test performs fault simulation, the method may further specifically include the following contents:

[0148] The fault response speed and fault recovery capability of the system under test are recorded according to the fault simulation result of the system under test.

[0149] In an optional embodiment, when a fault is injected into the RAID system, the present application can also perform fault handling and data recovery. By measuring the response speed of the tested system after the fault occurs, the agility and stability of the system in the face of different faults can be evaluated. In addition, by recording the process of fault recovery, testers can understand the time and resources required for the system to return to normal. By analyzing this data, developers can find bottlenecks in the system and improve its fault tolerance mechanism to improve the overall performance and recovery efficiency of the RAID system in real fault situations.

[0150] In order to solve the current problems of high cost of simulating RAID failures, hardware dependence, professional software dependence and manual operation dependence, the present application provides an embodiment of a RAID failure simulation device for implementing all or part of the contents of the RAID failure simulation method.

[0151] The RAID fault simulation device 002 provided in the embodiment of the present invention is shown in FIG. Fig. 9 , including an array construction module 21, a fault design module 22 and a fault injection module 23, wherein:

[0152] An array construction module 21 is used to construct a RAID array of a corresponding level based on the preset number of RAID array hard disks and the list of hard disks constituting the RAID;

[0153] A fault design module 22 is used to control parameters of the constructed RAID array to construct a corresponding fault model, wherein the fault type of the fault model includes at least one of a hard disk fault, a control and configuration fault, a logic fault, and a slow read / write fault;

[0154] The fault injection module 23 is used to determine corresponding fault injection instructions and fault recovery instructions according to the fault model and input them into the system under test, so that the system under test performs fault simulation.

[0155] According to any implementation manner of the present application, the fault design module includes a hard disk fault design unit, which is used to:

[0156] Determine the hard disk to be failed according to the type and number of hard disks of the RAID array;

[0157] Removing the hard disk to be failed to simulate a hard disk failure;

[0158] Add the removed hard disk to the RAID array to achieve fault recovery.

[0159] According to any implementation of the present application, the fault design module includes a control and configuration fault design unit, which is used to:

[0160] Changing the configuration level of the RAID array to an error level to simulate a RAID configuration failure;

[0161] Restoring the configuration level of the RAID array to a correct level to achieve RAID configuration recovery;

[0162] Interrupting the hard disk network of the RAID array to simulate a RAID access interruption failure;

[0163] Restoring the hard disk network of the RAID array to achieve RAID access recovery;

[0164] Injecting consistent failure parameters into the hard disks of the RAID array to simulate a RAID data consistency failure;

[0165] Resetting consistency fault parameters in the RAID array hard disk to achieve RAID data consistency recovery;

[0166] disabling the RAID array to simulate a RAID power outage failure;

[0167] Restarting the RAID array to implement RAID power outage recovery.

[0168] According to any implementation manner of the present application, the fault design module includes a logic fault design unit, which is used to:

[0169] Random data is written to the check block of the RAID array to simulate a RAID check information failure.

[0170] According to any embodiment of the present application, the fault design module includes a slow read / write fault design unit, which is used to: limit the speed of the hard disk network of the RAID array, reduce the network bandwidth, or control the disk read / write speed to simulate a slow RAID write fault;

[0171] The hard disk network of the RAID array is released from speed limit, the network bandwidth is reduced, or the disk read and write speed is controlled to achieve RAID write recovery.

[0172] According to any implementation of the present application, the fault injection module includes:

[0173] An instruction framework unit, used to determine corresponding fault injection instructions and fault recovery instructions according to the fault model;

[0174] The instruction injection unit is used to design an agent through bytecode enhancement technology so that the fault injection instruction and the fault recovery instruction can be received and parsed and executed by the system under test.

[0175] According to any implementation manner of the present application, it also includes a fault orchestration module, which is used to perform task orchestration on the fault model based on an orchestration engine so that the fault model is in at least one orchestration state of task serialization, task parallelism, and task suspension.

[0176] According to any embodiment of the present application, it also includes an information recording module, which is used to record the fault response speed and fault recovery capability of the system under test according to the fault simulation result of the system under test.

[0177] It can be seen from the above description that the RAID fault simulation device provided in the embodiment of the present application can solve the current problems of high cost of simulating RAID faults, hardware dependence, professional software dependence and manual operation dependence.

[0178] It should be noted that the RAID fault simulation method provided in the embodiment of the present invention can be used in the financial field, and can also be used in any technical field other than the financial field. The embodiment of the present invention does not limit the application field of the virtual reality-based bank training system and its training method.

[0179] Fig.10 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as Fig.10 As shown, the electronic device 003 includes: a processor (processor) 301, a memory (memory) 302 and a bus 303.

[0180] The processor 301 and the memory 302 communicate with each other via the bus 303 .

[0181] The processor 301 is used to call the program instructions in the memory 302 to execute the methods provided by the above method embodiments.

[0182] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned RAID failure simulation method is implemented.

[0183] An embodiment of the present invention further provides a computer program product, the computer program product comprising a computer program, and the computer program implements the above-mentioned RAID failure simulation method when executed by a processor.

[0184] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0185] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0186] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0187] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0188] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A RAID failure simulation method, characterized in that: The method comprises: Build a RAID array of the corresponding level based on the preset number of RAID array hard disks and the list of hard disks that make up the RAID; Performing parameter control on the constructed RAID array to construct a corresponding fault model, wherein the fault type of the fault model includes at least one of a hard disk fault, a control and configuration fault, a logic fault, and a slow read / write fault; Corresponding fault injection instructions and fault recovery instructions are determined according to the fault model and input into the system under test, so that the system under test performs fault simulation.

2. The RAID failure simulation method according to claim 1, characterized in that: The performing parameter control on the constructed RAID array to construct a corresponding fault model includes: Determine the hard disk to be failed according to the type and number of hard disks of the RAID array; Removing the hard disk to be failed to simulate a hard disk failure; The removed hard disk is added to the RAID array to achieve fault recovery.

3. The RAID failure simulation method according to claim 1, characterized in that: The performing parameter control on the constructed RAID array to construct a corresponding fault model includes: Changing the configuration level of the RAID array to an error level to simulate a RAID configuration failure; Restoring the configuration level of the RAID array to a correct level to achieve RAID configuration recovery; Interrupting the hard disk network of the RAID array to simulate a RAID access interruption failure; Restoring the hard disk network of the RAID array to achieve RAID access recovery; Injecting consistent failure parameters into the hard disks of the RAID array to simulate a RAID data consistency failure; Resetting consistency fault parameters in the RAID array hard disk to achieve RAID data consistency recovery; disabling the RAID array to simulate a RAID power outage failure; Restarting the RAID array to implement RAID power outage recovery.

4. The RAID failure simulation method according to claim 1, characterized in that: The performing parameter control on the constructed RAID array to construct a corresponding fault model includes: Random data is written to the check block of the RAID array to simulate a RAID check information failure.

5. The RAID failure simulation method according to claim 1, characterized in that: The performing parameter control on the constructed RAID array to construct a corresponding fault model includes: Limit the speed of the hard disk network of the RAID array, reduce the network bandwidth, or control the disk read and write speed to simulate a slow RAID write failure; The hard disk network of the RAID array is released from speed limit, the network bandwidth is reduced, or the disk read and write speed is controlled to achieve RAID write recovery.

6. The RAID failure simulation method according to claim 1, characterized in that: The step of determining corresponding fault injection instructions and fault recovery instructions according to the fault model and inputting the instructions into the system under test so as to simulate the fault of the system under test includes: Determine corresponding fault injection instructions and fault recovery instructions according to the fault model; The agent is designed by using bytecode enhancement technology so that the fault injection instruction and the fault recovery instruction can be received and parsed and executed by the system under test.

7. The RAID failure simulation method according to claim 1, characterized in that: Before determining the corresponding fault injection instruction and fault recovery instruction according to the fault model and inputting them into the system under test so that the system under test performs fault simulation, the method further includes: Perform task orchestration on the fault model based on an orchestration engine, so that the fault model is in at least one orchestration state of task serialization, task parallelization, and task suspension.

8. The RAID failure simulation method according to claim 1, characterized in that: After determining the corresponding fault injection instruction and fault recovery instruction according to the fault model and inputting them into the system under test so that the system under test performs fault simulation, the method further includes: The fault response speed and fault recovery capability of the system under test are recorded according to the fault simulation result of the system under test.

9. A RAID fault simulation device, characterized in that: The device comprises: An array construction module, used to construct a RAID array of a corresponding level based on a preset number of RAID array hard disks and a list of hard disks constituting the RAID; A fault design module, used to control parameters of the constructed RAID array to construct a corresponding fault model, wherein the fault type of the fault model includes at least one of a hard disk fault, a control and configuration fault, a logic fault, and a slow read and write fault; The fault injection module is used to determine corresponding fault injection instructions and fault recovery instructions according to the fault model and input them into the system under test so that the system under test performs fault simulation.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the RAID failure simulation method according to any one of claims 1 to 7 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the RAID failure simulation method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the RAID failure simulation method according to any one of claims 1 to 8 are implemented.

Images