A method, apparatus, device and medium for generating random test cases

Abstract

The application discloses a random test case generation method. The method comprises the following steps: generating at least one random factor, and acquiring a current test step in a plurality of type step libraries matched with a test object according to the value of each random factor; if the current test step satisfies a legality condition according to a generated test step set, the current test step is added to the generated test step set; returning to the operation of generating at least one random factor until the number of test steps in the generated test step set reaches a target number threshold; and constructing at least one random test case according to the generated test step set at the end of the cycle. Through the above technical scheme, the test case of the random scene can be automatically generated according to the test requirement, and the coverage of the test is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and in particular to a method, apparatus, device, and medium for generating random test cases. Background Technology

[0002] Fuzz testing is a software testing method that uses automated methods to generate a large number of test cases, input them into the software, and obtain the output results to discover potential software errors. Using comprehensive test cases can help developers discover unforeseen software errors in a timely and efficient manner.

[0003] Traditional methods for generating test cases often involve manually writing test cases based on the tester's experience or automatically generating test cases based on the automatic generation logic set by the tester.

[0004] However, the traditional test case generation methods mentioned above all require testers to set the test case generation logic based on historical experience, which often can only test some common scenarios, but cannot test some unpredictable abnormal scenarios. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and medium for generating random test cases, which can automatically generate test cases for random scenarios according to test requirements, effectively improving the coverage of tests.

[0006] According to one aspect of the present invention, a method for generating random test cases is provided, the method comprising:

[0007] Generate at least one random factor, and based on the value of each random factor, retrieve the current test step from a library of multiple types of steps that match the test object;

[0008] If the current test step meets the legality conditions according to the generated test step set, then the current test step is added to the generated test step set.

[0009] Return to the operation of generating at least one random factor until the number of test steps in the generated test step set reaches the target number threshold;

[0010] Based on the generated test step set at the end of the loop, construct at least one random test case.

[0011] Optionally, the test objects include distributed storage systems, and multiple types of step libraries include generator framework class step libraries and test operation class step libraries;

[0012] The test operation class step library stores memory read / write operation steps as well as multiple non-memory read / write operation steps.

[0013] The advantage of this setup is that by configuring the generator framework class step library and the test operation class step library, the probability of consecutively obtained test steps being of the same type is reduced, which can improve the rationality of the obtained test steps to a certain extent.

[0014] Optionally, at least one random factor is generated, and based on the value of each random factor, the current test step is obtained from a library of multiple types of steps matching the test object, including:

[0015] Generate a first random factor and check whether the first random factor falls within the first data range;

[0016] If so, then obtain the memory read / write operation step from the test operation class step library as the current test step; otherwise, generate a second random factor.

[0017] When the second random factor falls within the second data range, a non-memory read / write operation step is randomly selected from the test operation class step library as the current test step;

[0018] When the second random factor does not fall within the second data range, a generator framework class step is randomly selected from the generator framework class step library as the current test step.

[0019] The advantage of this setup is that by determining the range of data into which the random factor falls, the target step can be obtained from the corresponding step library as the current test step. This reduces the probability that the generated current test step does not meet the validity conditions. Under the premise of ensuring that the current test step has a high degree of randomness, it can reduce the number of times the test step is reselected.

[0020] Optionally, verify that the current test step meets the validity conditions based on the generated test step set, including:

[0021] Obtain a pre-built set of invalid step determination logic, in which at least two invalid step determination logics are defined between test steps;

[0022] Based on the currently stored generated test steps in the generated test step set, verify whether the current test step hits any invalid step judgment logic set. The generated test step set has at least one preset test step initially set.

[0023] If not, then verify that the current test step meets the legality conditions.

[0024] The advantage of this setup is that by verifying whether the current test step hits the invalid step judgment set, it is possible to filter out test steps that are meaningless in the current test scenario, thus ensuring the usability of test cases.

[0025] Optionally, after obtaining the current test step, the following may also be included:

[0026] If the current test step does not meet the validity conditions according to the generated test step set, then return to the process of generating at least one random factor until a current test step that meets the validity conditions is successfully generated.

[0027] The advantage of this setup is that by returning to the process of generating a random factor to obtain a new test step when the current test step does not meet the validity conditions, a new test step can be quickly obtained within the original test step acquisition logic, reducing the impact of invalid steps on the entire test case generation process.

[0028] Optionally, before generating at least one random factor, the following steps are also included:

[0029] Obtain test case configuration information, which includes the generation mode of test steps;

[0030] The test step generation modes include: generating a user-defined number of random test steps or generating a default number of random test steps;

[0031] The target quantity threshold is set according to the user-defined quantity or default quantity included in the generation mode of the test steps.

[0032] The advantage of this setting is that users can determine the number of test case steps by selecting the test step generation mode, and then set the target number threshold of test cases. This allows for configuring the number of test steps or randomly generating the number of test steps according to actual needs, making the number of test steps more selective and improving the randomness of the overall test cases.

[0033] Optionally, the test case configuration information may also include a generation mode for memory read / write patterns;

[0034] Among them, the memory read / write pattern generation mode includes generating a user-defined number of random access memory read / write patterns or generating a default number of random access memory read / write patterns;

[0035] After obtaining the test case configuration information, the following is also included:

[0036] In the memory read / write pattern library, at least one target memory read / write pattern that matches the user-defined number or default number in the generation mode of the memory read / write pattern is randomly selected.

[0037] Based on the generated test step set at the end of the loop, construct at least one random test case, specifically including:

[0038] Using each target memory read / write style, set each memory read / write operation step included in each generated test step set to obtain random test cases corresponding to each target memory read / write style.

[0039] The advantage of this setup is that by selecting the generation mode of memory read / write patterns and configuring the acquired target memory read / write patterns into the test step set, multiple test cases with different memory read / write patterns can be generated for a test step set, effectively increasing the test scenarios covered by the test cases.

[0040] Optionally, the test case configuration information may also include a fault injection mode;

[0041] The fault injection modes include generating a user-defined number of random faults or generating a default number of random faults.

[0042] After obtaining the test case configuration information, the following is also included:

[0043] In the fault database, at least one target fault is randomly selected that matches the user-defined or default number in the fault injection mode.

[0044] After constructing at least one random test case based on the generated test step set at the end of the loop, the process also includes:

[0045] Each of the target faults is added to each random test case to achieve the injection of each target fault with a preset trigger probability during the execution of each random test case.

[0046] The advantage of this setup is that by selecting target faults from the fault database and adding them to random test cases, and injecting each fault with a preset trigger probability during the execution of the random test cases, actual environmental factors can be considered when executing test cases, making the test results more indicative and referential.

[0047] According to another aspect of the present invention, an apparatus for generating random test cases is provided, comprising:

[0048] The test step acquisition module is used to generate at least one random factor and, based on the value of each random factor, retrieve the current test step from a library of multiple types of steps that match the test object.

[0049] The test step addition module is used to add the current test step to the generated test step set if the current test step meets the legality conditions according to the generated test step set.

[0050] The test step loop acquisition module is used to return the operation of generating at least one random factor until the number of test steps in the generated test step set reaches the target number threshold.

[0051] The random test case construction module is used to construct at least one random test case based on the generated test step set at the end of the loop.

[0052] According to another aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the random test case generation method described in any embodiment of the present invention.

[0053] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the random test case generation method according to any embodiment of the present invention.

[0054] The technical solution of this invention obtains the current test step from the step library by generating a random factor, adds the test steps that meet the legality conditions to the test step set, and cyclically obtains test steps until the number of test steps in the test step set reaches the target threshold number, until the loop ends and test cases are generated. This method can achieve the purpose of automatically generating test cases. The generated test cases not only meet the legality conditions, but also cover some unpredictable abnormal scenarios, which can effectively improve the speed of test case generation and effectively expand the test scope of test cases.

[0055] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart of a method for generating random test cases according to Embodiment 1 of the present invention;

[0058] Figure 2 This is a flowchart of a method for generating random test cases according to Embodiment 2 of the present invention;

[0059] Figure 3 This is a schematic diagram of the structure of a random test case generation device according to Embodiment 3 of the present invention;

[0060] Figure 4 This is a schematic diagram of the structure of a computer device that implements the random test case generation method of the present invention. Detailed Implementation

[0061] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0062] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0063] Example 1

[0064] Figure 1 This is a flowchart of a method for generating random test cases according to Embodiment 1 of the present invention. This embodiment is applicable to situations requiring high coverage and high adaptability in constructing random test cases. The method can be executed by a random test case generation device, which can be implemented in hardware and / or software. This device can be configured in a computer with data processing capabilities, typically executed by a computer node in a computer cluster. Figure 1 As shown, the method includes:

[0065] S110. Generate at least one random factor, and obtain the current test step from a library of multiple types of steps that match the test object based on the value of each random factor.

[0066] The test objects are various software and hardware platforms that need to be tested using test cases. Typically, they can include distributed storage systems, such as distributed block storage systems.

[0067] It is understandable that test cases can be composed of multiple test steps linked together sequentially to achieve the defined software testing functions. Correspondingly, the step library is used to store a set of multiple steps that can be selected as test steps by test cases. Among them, all available steps can be divided into multiple types according to the test functions that different steps can achieve, thereby constructing multiple types of step libraries.

[0068] Specifically, taking a distributed storage system as an example, the various types of step libraries can include generator framework class step libraries and test operation class step libraries.

[0069] The test operation class step library stores memory read / write operation steps as well as multiple non-memory read / write operation steps.

[0070] Software testing can refer to the process of executing target software to discover errors. The test object can be understood as the distributed storage system to be tested, while the test steps form the basis for building the execution process of the distributed storage system. A step library can be used to store relevant test steps for testing a distributed storage system. During the testing process, the test steps need to be executed step by step to complete the entire execution process of the target distributed storage system.

[0071] Optionally, the generator framework class step library can store node class steps in the test process, such as selecting the operation object to be executed, the start of the operation, or the end of the operation; the test operation class step library can store operation class steps in the test process. Non-memory read and write operation steps can include operations such as creating, deleting, and migrating the current operation object. Memory read and write operation steps can generate read and write tasks according to the read and write mode of the storage device and send them to the incentive layer of the distributed storage system to execute the read and write tasks.

[0072] The advantage of this setup is that by configuring the generator framework class step library and the test operation class step library, the probability of consecutively obtained test steps being of the same type is reduced, which can improve the rationality of the obtained test steps to a certain extent.

[0073] In one specific implementation, the operation object can be any storage object in the distributed storage system, such as a logical volume, snapshot volume, or clone volume; the operation object selection method for each test node can be obtained from the generator framework class step library, such as specifying the operation object as the last created storage object, specifying the operation object as the last specified operation object, or randomly selecting a storage object as the specified operation object, etc.; the start or end identifier of the operation can also be obtained; the memory read / write operation steps or non-memory read / write operation steps for the current test can be obtained from the test operation class step library.

[0074] It should also be noted that in the embodiments of the present invention, the operation objects and operation steps are not in one-to-one correspondence. The technical solution of the present invention takes into account the need to test unpredictable test scenarios. Therefore, all kinds of test steps are randomly selected, and it is possible that after obtaining an operation object, multiple operation steps are obtained in succession.

[0075] A random factor can be understood as a randomly generated element. The purpose of generating a random factor is to randomly obtain the current test step. Random factors can be in the form of numbers, letters, and other types of elements. The number of random factors can correspond to the type of the step library.

[0076] In one specific implementation, a random factor in the form of a number can be selected. The range of values ​​of a random factor can correspond to a type of step library that matches the test object. That is, when the value of the random factor falls into a certain range, the current test step can be obtained from the step library of the type corresponding to that range.

[0077] S120. If the current test step meets the legality conditions according to the generated test step set, then the current test step is added to the generated test step set.

[0078] A test step set can be understood as a collection of related test steps stored in the order they were obtained.

[0079] Validity conditions allow users to filter test steps based on pre-defined criteria for the target test object. The purpose of setting validity conditions is to filter out test steps that are meaningless in the current test scenario. Executing such meaningless test steps may lead to unexpected or destructive results for the target test object; deleting these meaningless test steps ensures the usability of the test cases.

[0080] Furthermore, since the purpose of setting legality conditions is to filter out meaningless test steps in the current test scenario, the legality conditions of each test step need to be verified in conjunction with the currently generated set of test steps.

[0081] S130, Return to the operation of generating at least one random factor until the number of test steps in the generated test step set reaches the target number threshold.

[0082] After adding the current test step to the set of generated test steps, it can be determined whether the number of test steps in the set of generated test steps has reached the target number threshold. If the target number threshold has not been reached, step S110 can be returned to continue generating at least one random factor, thereby continuing to add test steps to the set of test steps; if the target number threshold has been reached, the generation of test steps can be stopped.

[0083] Furthermore, before generating at least one random factor, a target quantity threshold can be pre-set in the test step set. The method for setting the target quantity threshold can include:

[0084] Obtain test case configuration information, which includes the generation mode of test steps;

[0085] The test step generation modes include: generating a user-defined number of random test steps or generating a default number of random test steps;

[0086] The target quantity threshold is set according to the user-defined quantity or default quantity included in the generation mode of the test steps.

[0087] This can be understood as follows: by determining the number of random test steps in the test case configuration information, a target number threshold can be set. If the number of random test steps is a user-defined number, the target number threshold can match the specific user-defined number; if the number of random test steps is a default number, the target number threshold can be the default number.

[0088] If the user does not define the number of random test steps, a default number of random test steps can be generated. Both the user-defined number and the default number of test steps need to be set by the user in advance, but the user-defined number of test steps needs to be set in advance before each random test case generation, while the default number of test steps only needs to be set once during the development phase of the random test case generation device.

[0089] The advantage of this setting is that users can determine the number of test case steps by selecting the test step generation mode, and then set the target number threshold of test cases. This allows for configuring the number of test steps or randomly generating the number of test steps according to actual needs, making the number of test steps more selective and improving the randomness of the overall test cases.

[0090] S140. Based on the generated test step set at the end of the loop, construct at least one random test case.

[0091] When the number of test steps in the test step set reaches the target threshold, the test step generation loop can be terminated.

[0092] Furthermore, the test case configuration information may also include a generation mode for memory read / write patterns;

[0093] The memory read / write pattern generation mode can include generating a user-defined number of random access memory read / write patterns or generating a default number of random access memory read / write patterns.

[0094] The memory read / write pattern can represent the size of data read or written in a single memory operation. For example, when the memory uses block storage, the memory read / write pattern can be to store data in 4k blocks of IO (Input / Output) size or in 8k blocks of IO size, etc.

[0095] Similarly, the user-defined number and random number of memory read / write patterns both need to be set by the user in advance. The user-defined number of memory read / write patterns needs to be set in advance before each random test case is generated, while the default number of memory read / write patterns can be set only once during the development phase of the random test case generation device.

[0096] Furthermore, after obtaining the test case configuration information, it may also include:

[0097] In the memory read / write pattern library, at least one target memory read / write pattern is randomly selected that matches the user-defined number or default number in the generation mode of the memory read / write pattern.

[0098] The memory read / write style library is a database that users can pre-configure based on the selectable read / write styles of the target memory.

[0099] Furthermore, based on the generated test step set at the end of the loop, at least one random test case is constructed, which may specifically include:

[0100] Using each target memory read / write style, set each memory read / write operation step included in each generated test step set to obtain random test cases corresponding to each target memory read / write style.

[0101] It is understandable that by acquiring multiple target memory read / write patterns, the generated test step set can be expanded. The test step set can be combined with at least one target memory read / write pattern to generate at least one test case, and the number of test cases can match the number of target memory read / write patterns.

[0102] The advantage of this setup is that by selecting the generation mode of memory read / write patterns and configuring the acquired target memory read / write patterns into the test step set, multiple test cases with different memory read / write patterns can be generated for a test step set, effectively increasing the test scenarios covered by the test cases.

[0103] The innovation of this invention lies not only in the random generation of test cases to cover multiple test scenarios, but also in the fact that this invention is based on black-box or gray-box testing, that is, testing is only based on the requirements and functions of the test object, rather than on the source code of the test object. The advantage of this setting is that when the source code is modified, there is no need to change the way test cases are selected, and the test scenarios can be expanded more conveniently.

[0104] The technical solution of this invention obtains the current test step from the step library by generating a random factor, adds the test steps that meet the legality conditions to the test step set, and cyclically obtains test steps until the number of test steps in the test step set reaches the target threshold number, until the loop ends and test cases are generated. This method can achieve the purpose of automatically generating test cases. The generated test cases not only meet the legality conditions, but also cover some unpredictable abnormal scenarios, which can effectively improve the speed of test case generation and effectively expand the test scope of test cases.

[0105] Example 2

[0106] Figure 2 This is a flowchart of a method for generating random test cases according to Embodiment 2 of the present invention. Based on the above embodiments, this embodiment further specifies the method for obtaining test steps. For example... Figure 2 As shown, the method includes:

[0107] S210. Obtain test case configuration information, which includes the generation mode of test steps.

[0108] The test step generation mode can include: generating a user-defined number of random test steps or generating a default number of random test steps.

[0109] Furthermore, the test case configuration information can also include the generation mode of memory read / write patterns;

[0110] The memory read / write pattern generation mode includes generating a user-defined number of random access memory read / write patterns or generating a default number of random access memory read / write patterns.

[0111] Furthermore, after obtaining the test case configuration information, it may also include:

[0112] In the memory read / write pattern library, at least one target memory read / write pattern is randomly selected that matches the user-defined number or default number in the generation mode of the memory read / write pattern.

[0113] S220. Set the target quantity threshold according to the user-defined quantity or default quantity included in the generation mode of the test steps.

[0114] S230. Generate at least one random factor, and obtain the current test step from a library of multiple types of steps that match the test object based on the value of each random factor.

[0115] Specifically, at least one random factor is generated, and based on the value of each random factor, the current test step is obtained from a library of multiple types of steps that match the test object. This may include:

[0116] Generate a first random factor and check whether the first random factor falls within the first data range;

[0117] If so, then obtain the memory read / write operation step from the test operation class step library as the current test step; otherwise, generate a second random factor.

[0118] When the second random factor falls within the second data range, a non-memory read / write operation step is randomly selected from the test operation class step library as the current test step;

[0119] When the second random factor does not fall within the second data range, a generator framework class step is randomly selected from the generator framework class step library as the current test step.

[0120] Optionally, the settings for each data range can correspond to the type of operation steps corresponding to each data range. For operation step types that users expect to execute a lot, the data range of the random factor can be set to be larger. Correspondingly, for operation step types that users expect to execute a few times, the corresponding random factor data range can be reduced. Each data range can also be set proportionally according to the proportion of the number of times each operation step type is expected to be executed by the user.

[0121] The advantage of this setup is that by determining the range of data into which the random factor falls, the target step can be obtained from the corresponding step library as the current test step. This reduces the probability that the generated current test step does not meet the validity conditions. Under the premise of ensuring that the current test step has a high degree of randomness, it can reduce the number of times the test step is reselected.

[0122] S240. Obtain the pre-built invalid step judgment logic set. Based on each generated test step currently stored in the generated test step set, verify whether the current test step hits any invalid step judgment logic set. If yes, return to execute S230; otherwise, execute S250.

[0123] Specifically, if the current test step hits any invalid step decision logic set, then the current test step is verified to not meet the legality condition; if the current test step does not hit any invalid step decision logic set, then the current test step is verified to meet the legality condition.

[0124] Each invalid step determination logic defines at least two invalid step determination logics between test steps, and the generated test step set has at least one preset test step initially set.

[0125] The invalid step determination logic set can be a dataset pre-set by the user based on historical experience. Preferably, the invalid step determination logic set can include identified invalid steps and suggested invalid steps. If the current test step hits the identified invalid step, the currently obtained test step must be discarded and the process should return to step S230 to obtain a new test step. If the current test step hits the suggested invalid step, the operation of discarding the current test step can be triggered according to the preset discard probability.

[0126] In one specific implementation, determining an invalid step may include: if the target operation object has been obtained as an initial logical volume, performing an operation step to delete the target operation object; if, in the existing test steps in the test step set, the deletion operation is equal to the creation operation, performing a deletion operation step, but not limited to the above-mentioned invalid steps; suggesting invalid steps may include: the currently obtained test step is the same as the previously obtained test step.

[0127] The advantage of this setup is that by verifying whether the current test step hits the invalid step judgment set, it is possible to filter out test steps that are meaningless in the current test scenario, thus ensuring the usability of test cases.

[0128] S250. Add the current test step to the set of generated test steps.

[0129] S260. Determine whether the number of test steps in the currently generated test step set meets the target number threshold. If yes, execute S270; otherwise, return to execute S230.

[0130] S270. Based on the generated test step set and test case configuration information at the end of the loop, construct at least one random test case.

[0131] Specifically, based on the generated test step set and test case configuration information at the end of the loop, at least one random test case is constructed. This may include: using each target memory read / write style in the test case configuration information, setting each memory read / write operation step included in each generated test step set to obtain a random test case corresponding to each target memory read / write style.

[0132] The technical solution of this invention reduces the probability that the generated current test step does not meet the legality conditions by determining the data range in which the random factor falls and obtaining the target step from the corresponding step library as the current test step. By verifying whether the current test step hits the invalid step judgment logic set, it can filter out test steps that are meaningless in the current test scenario, thus ensuring both the legality and randomness of the test cases.

[0133] Furthermore, the test case configuration information may also include a fault injection mode;

[0134] The fault injection mode can include generating a user-defined number of random faults or generating a default number of random faults;

[0135] After obtaining the test case configuration information, it may also include:

[0136] In the fault database, at least one target fault is randomly selected that matches the user-defined or default number in the fault injection mode.

[0137] After constructing at least one random test case based on the generated test step set at the end of the loop, it may also include:

[0138] Each of the target faults is added to each random test case to achieve the injection of each target fault with a preset trigger probability during the execution of each random test case.

[0139] Specifically, the purpose of fault injection in test cases is to test the usage of the test object in real-world application scenarios when some unexpected situations occur, such as whether the target test object can still obtain the expected output results when the device using the test object loses power.

[0140] The fault database is a pre-configured database based on historical experience, allowing users to set all possible fault scenarios. When configuring faults, trigger scenarios for each fault can also be configured. A target fault must be triggered within its corresponding scenario to potentially affect the normal operation of the test object. For example, configuring a network outage fault could be triggered during data read / write operations, as a network outage could cause data interruption. Configuring faults in this way makes injecting faults into test cases meaningful.

[0141] Preferably, a trigger probability can be set for each fault. When testing test cases, the injection of each target fault can be triggered according to the trigger probability. The trigger probability of each fault in each test case can be the same, or the trigger probability of each fault can be set separately according to actual needs.

[0142] The advantage of this setup is that by selecting target faults from the fault database and adding them to random test cases, and injecting each fault with a preset trigger probability during the execution of the random test cases, actual environmental factors can be considered when executing test cases, making the test results more indicative and referential.

[0143] Specific application scenarios

[0144] This specific application scenario illustrates an optional method for generating random test cases, using a distributed block storage test case generation process as an example.

[0145] Test case generation for distributed block storage can be based on three dimensions: test steps, read / write patterns, and fault injection. Before generating test cases, a preset generation mode should be configured for each of these dimensions, as shown in Table 1.

[0146] Table 1

[0147] Custom standard random No configuration Test steps a、b a b, c c Read and write styles d, e d e, f f Fault Injection h、i h i g

[0148] Each dimension includes four modes: custom, standard, random, and no configuration. Each mode includes one or two optional operations. Operations a, b, and c are test step generation operations; operations d, e, and f are read / write pattern generation operations; and operations g, h, and i are fault injection operations. The specific operation content is as follows:

[0149] The test step generation process includes:

[0150] a. Generate test steps based on the number of steps and test content pre-configured by the user;

[0151] b. Generate a preset number of random test steps;

[0152] c. Generate 20 random test steps;

[0153] The read / write pattern generation operation includes:

[0154] d. Obtain the user-configured read / write patterns;

[0155] e. Generate a preset number of read / write patterns;

[0156] f. Generate 3 read / write styles;

[0157] Fault injection operations include:

[0158] g. Injection failure;

[0159] h. Injecting a pre-defined fault;

[0160] i. Inject a preset number of faults.

[0161] Specifically, for test steps, if no mode is configured, 20 test steps are randomly generated by default; in standard mode, a preset number of operation steps are generated; in random mode, operation b or operation c is randomly selected to generate read / write patterns; in custom mode, test steps are generated based on the specific operation selected by the user. The generated steps will be executed sequentially within a single thread.

[0162] For read / write patterns, if no mode is configured, three read / write patterns are randomly generated by default; if it is the standard mode, a preset number of read / write patterns are generated; if it is the random mode, operation e or operation f is randomly selected to generate read / write patterns; if it is the custom mode, read / write patterns are generated according to the specific operation selected by the user. Different read / write patterns are executed sequentially within their respective threads.

[0163] For fault injection, if no mode is configured, no faults will be injected; in standard mode, faults will be injected probabilistically based on preset faults; in random mode, a preset number of faults will be injected probabilistically, or if no preset number is specified, 5 faults will be randomly generated for probabilistic injection; in custom mode, fault injection can be performed based on user-preset operations. During testing, the injected faults will be triggered independently based on probability.

[0164] After the above operations, it can be determined whether test cases need to be randomly generated and the number of randomly generated test cases, thereby further generating random test cases.

[0165] Both the read / write style and fault generation are based on randomly selecting a preset number of models from the model library. The logic is relatively simple and will not be described in detail in this specific application scenario.

[0166] The test operation step library can be divided into a generator framework step library and a test operation step library. The generator framework step library can include: CHG_INO_LAST, CHG_INO_RANDOM, and CHG_INO_OLD, which represent reassigning the currently operated logical volume, snapshot volume, or clone volume to the last created, randomly selected, or previously assigned volume, respectively. It also includes LOOP_START and LOOP_END, which do not participate in the random algorithm and represent the start and end markers of the random test step loop, respectively. Test operation steps can include: SEND_IO, CRT_LUN, CRT_SNAP, CRT_CLONE, DEL_INO, MIG_OUT, MIG_IN, MIG_AUTO, etc., which can be roughly divided into three categories: first, SEND_IO, which generates a certain amount of storage blocks according to the read and write pattern and sends them to the stimulus layer; second, CRT-type operations, including creating logical volumes, snapshot volumes, or clone volumes, as well as deleting the currently operated logical volume, snapshot volume, or clone volume; and third, MIG-type operations, which represent service migration out, migration in, and migration out followed by migration in, respectively.

[0167] The specific method for generating random test cases can be as follows: First, based on the user-defined generation pattern for the test step dimension, a random factor is randomly selected within the value range [0, 100]. For ease of understanding, this is referred to as the first random factor. The first random factor is then checked to see if it is greater than 70. If it is, the step SEND_IO is obtained as the current test step. If the first random factor is less than or equal to 70, another random factor is randomly selected. For ease of understanding, this is referred to as the second random factor. The second random factor is then checked to see if it is less than 30. If it is, a test step is randomly selected from the generator framework class step library as the current test step. If the second random factor is greater than or equal to 30, a test step is randomly selected from the test step library. A test step is randomly selected from the operation step library as the current test step. Each current test step undergoes a validity check. If the current test step meets the validity requirements, it is added to the generated test step set. If the current test step does not meet the validity requirements, the process returns to regenerate the first random factor and continues generating new test steps. After each new test step is added to the test step set, it is also necessary to check whether the number of test steps in the current test step set meets the defined test step generation pattern, i.e., whether the number of test steps meets the user's target number. If it does, test cases are generated based on the user-defined read / write style generation pattern; otherwise, test steps are continuously acquired.

[0168] Specifically, the following describes the test steps to facilitate understanding of the validity conditions: CHG_INO_LAST does not need to be repeated after DEL or CHG_INO_LAST; CHG_INO_LAST is not needed if LUN, SNAP, or CLONE were not created initially; the source volume cannot be deleted if CRT_SNAP is performed without CHG_INO_LAST; an initially created LUN cannot be deleted; deletion is not allowed if the number of DEL operations is about to exceed the number of CRT operations; during ROLLBACK_SNAP, ROLLBACK is not allowed if the current operation is not SNAP.

[0169] Furthermore, if the currently obtained test step is the same as the previously obtained test step, a probability of returning to regenerate the first random factor can be set. When such a condition occurs, the first random factor can be regenerated probabilistically.

[0170] Furthermore, when executing test cases, random probabilistic fault injection can be performed on the test cases according to the user-defined fault injection pattern, making the test results more indicative and referential.

[0171] Example 3

[0172] Figure 3 This is a schematic diagram of a random test case generation device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: a test step acquisition module 310, a test step addition module 320, a test step cyclic acquisition module 330, and a random test case construction module 340.

[0173] The test step acquisition module 310 is used to generate at least one random factor and, based on the value of each random factor, acquire the current test step from a library of multiple types of steps that match the test object.

[0174] The test step adding module 320 is used to add the current test step to the generated test step set if the current test step meets the legality conditions according to the generated test step set.

[0175] The test step loop acquisition module 330 is used to return the operation of generating at least one random factor until the number of test steps in the generated test step set reaches the target number threshold.

[0176] The random test case construction module 340 is used to construct at least one random test case based on the generated test step set at the end of the loop.

[0177] The technical solution of this invention obtains the current test step from the step library by generating a random factor, adds the test steps that meet the legality conditions to the test step set, and cyclically obtains test steps until the number of test steps in the test step set reaches the target threshold number, until the loop ends and test cases are generated. This method can achieve the purpose of automatically generating test cases. The generated test cases not only meet the legality conditions, but also cover some unpredictable abnormal scenarios, which can effectively improve the speed of test case generation and effectively expand the test scope of test cases.

[0178] Based on the above embodiments, the test objects include distributed storage systems, and multiple types of step libraries include generator framework class step libraries and test operation class step libraries;

[0179] The test operation class step library stores memory read / write operation steps as well as multiple non-memory read / write operation steps.

[0180] Based on the above embodiments, the test step acquisition module 310 can be specifically used for:

[0181] Generate a first random factor and check whether the first random factor falls within the first data range;

[0182] If so, then obtain the memory read / write operation step from the test operation class step library as the current test step; otherwise, generate a second random factor.

[0183] When the second random factor falls within the second data range, a non-memory read / write operation step is randomly selected from the test operation class step library as the current test step;

[0184] When the second random factor does not fall within the second data range, a generator framework class step is randomly selected from the generator framework class step library as the current test step.

[0185] Based on the above embodiments, the test step adds module 320, which can be specifically used for:

[0186] Obtain a pre-built set of invalid step determination logic, in which at least two invalid step determination logics are defined between test steps;

[0187] Based on the currently stored generated test steps in the generated test step set, verify whether the current test step hits any invalid step judgment logic set. The generated test step set has at least one preset test step initially set.

[0188] If not, then verify that the current test step meets the legality conditions.

[0189] Based on the above embodiments, a random factor repetition generation module may also be included, used after obtaining the current test step:

[0190] If the current test step does not meet the validity conditions according to the generated test step set, then return to the process of generating at least one random factor until a current test step that meets the validity conditions is successfully generated.

[0191] Based on the above embodiments, a target quantity threshold acquisition module may also be included, used before generating at least one random factor:

[0192] Obtain test case configuration information, which includes the generation mode of test steps;

[0193] The test step generation modes include: generating a user-defined number of random test steps or generating a default number of random test steps;

[0194] The target quantity threshold is set according to the user-defined quantity or default quantity included in the generation mode of the test steps.

[0195] Based on the above embodiments, the test case configuration information also includes a memory read / write pattern generation mode;

[0196] The memory read / write pattern generation mode includes generating a user-defined number of random access memory read / write patterns or generating a default number of random access memory read / write patterns.

[0197] Based on the above embodiments, a target memory read / write pattern selection unit may also be included, which, after obtaining the test case configuration information, randomly selects at least one target memory read / write pattern from the memory read / write pattern library that matches the user-defined quantity or default quantity in the generation mode of the memory read / write pattern.

[0198] Based on the above embodiments, the random test case construction module 340 can be specifically used for:

[0199] Using each target memory read / write style, set each memory read / write operation step included in each generated test step set to obtain random test cases corresponding to each target memory read / write style.

[0200] Based on the above embodiments, the test case configuration information also includes a fault injection mode;

[0201] The fault injection mode includes generating a user-defined number of random faults or generating a default number of random faults.

[0202] Based on the above embodiments, a target fault selection unit may also be included, which, after obtaining the test case configuration information, randomly selects at least one target fault from the fault library that matches the user-defined number or default number in the fault injection mode.

[0203] Based on the above embodiments, a target fault injection module may also be included, used to construct at least one random test case after generating a set of test steps at the end of the loop:

[0204] Each of the target faults is added to each random test case to achieve the injection of each target fault with a preset trigger probability during the execution of each random test case.

[0205] The random test case generation device provided in this embodiment of the invention can execute the random test case generation method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0206] Example 4

[0207] Figure 4 A schematic diagram of a computer device 40 that can be used to implement embodiments of the present invention is shown. The computer device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The computer device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0208] like Figure 4 As shown, the computer device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42 and a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the ROM 42 or loaded from storage unit 48 into the RAM 43. The RAM 43 may also store various programs and data required for the operation of the computer device 40. The processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0209] Multiple components in computer device 40 are connected to I / O interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of monitors, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows computer device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0210] Processor 41 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as the method for generating random test cases as described in the embodiments of the present invention. That is:

[0211] Generate at least one random factor, and based on the value of each random factor, retrieve the current test step from a library of multiple types of steps that match the test object;

[0212] If the current test step meets the legality conditions according to the generated test step set, then the current test step is added to the generated test step set.

[0213] Return to the operation of generating at least one random factor until the number of test steps in the generated test step set reaches the target number threshold;

[0214] Based on the generated test step set at the end of the loop, construct at least one random test case.

[0215] In some embodiments, the method for generating random test cases may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on computer device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the method for generating random test cases described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to execute the method for generating random test cases by any other suitable means (e.g., by means of firmware).

[0216] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0217] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0218] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0219] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0220] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0221] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0222] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0223] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for generating random test cases, characterized in that, include: At least one random factor is generated, and the current test step is obtained from multiple types of step libraries that match the test object based on the value of each random factor. The test object includes a distributed storage system, and the multiple types of step libraries include a generator framework class step library and a test operation class step library. The test operation class step library stores memory read and write operation steps as well as multiple non-memory read and write operation steps. If the current test step meets the legality conditions according to the generated test step set, then the current test step is added to the generated test step set. Return to the operation of generating at least one random factor until the number of test steps in the generated test step set reaches the target number threshold; Based on the set of test steps generated at the end of the loop, construct at least one random test case; This involves generating at least one random factor, and based on the values ​​of each random factor, retrieving the current test step from a library of multiple types of steps that match the test object, including: Generate a first random factor and check whether the first random factor falls within the first data range; If so, then obtain the memory read / write operation step from the test operation class step library as the current test step; otherwise, generate a second random factor. When the second random factor falls within the second data range, a non-memory read / write operation step is randomly selected from the test operation class step library as the current test step; When the second random factor does not fall within the second data range, a generator framework class step is randomly selected from the generator framework class step library as the current test step.

2. The method according to claim 1, characterized in that, Verify that the current test step meets the legality conditions based on the generated test step set, including: Obtain a pre-built set of invalid step determination logic, in which at least two invalid step determination logics are defined between test steps; Based on the currently stored generated test steps in the generated test step set, verify whether the current test step hits any invalid step judgment logic set. The generated test step set has at least one preset test step initially set. If not, then verify that the current test step meets the legality conditions.

3. The method according to claim 1, characterized in that, After obtaining the current test step, it also includes: If the current test step does not meet the validity conditions according to the generated test step set, then return to the process of generating at least one random factor until a current test step that meets the validity conditions is successfully generated.

4. The method according to any one of claims 1-3, characterized in that, Before generating at least one random factor, the process also includes: Obtain test case configuration information, which includes the generation mode of test steps; The test step generation modes include: generating a user-defined number of random test steps or generating a default number of random test steps; The target quantity threshold is set according to the user-defined quantity or default quantity included in the generation mode of the test steps.

5. The method according to claim 4, characterized in that, The test case configuration information also includes the generation mode of memory read / write patterns; Among them, the memory read / write pattern generation mode includes generating a user-defined number of random access memory read / write patterns or generating a default number of random access memory read / write patterns; After obtaining the test case configuration information, the following is also included: In the memory read / write pattern library, at least one target memory read / write pattern that matches the user-defined number or default number in the generation mode of the memory read / write pattern is randomly selected. Based on the generated test step set at the end of the loop, construct at least one random test case, specifically including: Using each target memory read / write style, set each memory read / write operation step included in each generated test step set to obtain random test cases corresponding to each target memory read / write style.

6. The method according to claim 4, characterized in that, The test case configuration information also includes a fault injection mode; The fault injection modes include generating a user-defined number of random faults or generating a default number of random faults. After obtaining the test case configuration information, the following is also included: In the fault database, at least one target fault is randomly selected that matches the user-defined or default number in the fault injection mode. After constructing at least one random test case based on the generated test step set at the end of the loop, the process also includes: Each of the target faults is added to each random test case to achieve the injection of each target fault with a preset trigger probability during the execution of each random test case.

7. A device for generating random test cases, characterized in that, include: The test step acquisition module is used to generate at least one random factor and, based on the value of each random factor, retrieve the current test step from multiple types of step libraries that match the test object. The test object includes a distributed storage system, and the multiple types of step libraries include a generator framework class step library and a test operation class step library. The test operation class step library stores memory read / write operation steps and multiple non-memory read / write operation steps. The test step addition module is used to add the current test step to the generated test step set if the current test step meets the legality conditions according to the generated test step set. The test step loop acquisition module is used to return the operation of generating at least one random factor until the number of test steps in the generated test step set reaches the target number threshold. The random test case construction module is used to construct at least one random test case based on the generated test step set at the end of the loop. The test step acquisition module is specifically used for: Generate a first random factor and check whether the first random factor falls within the first data range; If so, then obtain the memory read / write operation step from the test operation class step library as the current test step; otherwise, generate a second random factor. When the second random factor falls within the second data range, a non-memory read / write operation step is randomly selected from the test operation class step library as the current test step; When the second random factor does not fall within the second data range, a generator framework class step is randomly selected from the generator framework class step library as the current test step.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for generating random test cases as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the method for generating random test cases as described in any one of claims 1-6.

Images