A method for generating test cases for unmanned aerial vehicles based on a double-layer quality diversity optimization algorithm

Abstract

The application discloses a kind of unmanned plane test case generation method based on double-layer quality diversity optimization algorithm, it is related to software testing technical field, including steps: T1, obstacle coding is expressed as individual;T2, for unmanned plane flight task, task characteristics are obtained by analysis;Space is constructed including multiple cells;T3, multiple are randomly generated, and individual is filled to cell;T4, each cell is respectively sampled and keeps an individual;Simulation test is carried out to individual, and evaluation value is generated for individual;T5, individual is sampled based on evaluation value;For the individual sampled, disturbance is generated, and offspring individual is generated;Evaluation value is generated to offspring individual, and offspring individual is stored in space;T6, repeat step T5 until reaching iteration end condition;The individual in space is output as test case.Finally, high-quality and diverse key test cases can be explored more efficiently.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and in particular to a method for generating test cases for unmanned aerial vehicles (UAVs) based on a two-layer quality diversity optimization algorithm. Background Technology

[0002] With the rapid development of drone technology, drones are increasingly being used in logistics delivery, inspection, and agricultural monitoring. However, as the complexity of drone-related tasks increases, the demand for automated drone execution software is also growing. The efficiency and robustness of this software are crucial; therefore, key test cases for optimizing drone software have become paramount.

[0003] In existing technologies, UAV test case generation mainly involves constructing diverse task scenarios. Under multiple preset constraints, a small number of obstacles are generated to effectively induce malfunctions in the UAV's automated execution system, thereby exposing potential faults in the UAV's automated system. Specifically, the main characteristics and challenges include:

[0004] (1) Multiple constraints: Obstacles need to meet various constraints, such as non-overlapping and area satisfaction.

[0005] (2) Cost: Each UAV simulation test requires a lot of time and resources, so the testing process is expensive and it is necessary to generate as many key test cases as possible within limited resources.

[0006] (3) Diversity: The generated test cases need to be diverse in order to test the robustness of the UAV automated execution software.

[0007] However, existing test case generation methods are insufficient to generate a small number of diverse key test cases within a limited budget, making it difficult to efficiently test and expose the faults of drone automation systems in a short period of time.

[0008] Therefore, there is an urgent need to provide a method for generating UAV test cases based on a two-layer quality diversity optimization algorithm to solve the above-mentioned technical problems. Summary of the Invention

[0009] To address the problems existing in the prior art, this invention provides a method for generating test cases for unmanned aerial vehicles (UAVs) based on a two-layer quality diversity optimization algorithm. Under limited budget conditions, it can effectively generate a small number of key test cases that cover a wide range of issues, thereby enabling efficient fault testing of UAV automated systems in a short period of time.

[0010] The technical solution of this invention is implemented as follows:

[0011] A method for generating test cases for unmanned aerial vehicles (UAVs) based on a two-layer quality diversity optimization algorithm, wherein the test cases are used to generate obstacles and simulate the environment to interfere with and test UAV flight missions; the method includes the following steps:

[0012] T1. The attributes of the obstacle are digitally encoded, and the encoding is represented by an individual.

[0013] T2. For the UAV flight mission, analyze and obtain mission characteristics; construct a behavior space based on the mission characteristics, the behavior space including multiple discrete and evenly spaced cells;

[0014] T3. Randomly initialize and generate multiple individuals, and fill the cells with the individuals, so that each cell includes at least one individual;

[0015] T4. Sample and retain one individual for each cell; perform simulation testing on the individual and generate an evaluation value for the individual;

[0016] T5. Sample the individuals based on the evaluation values; perturb the sampled individuals to generate offspring individuals; generate evaluation values ​​for the offspring individuals and store them in the behavior space; the higher the probability of sampling a key test case based on the evaluation value, the more likely it is to be a key test case. Key test cases can efficiently trigger drone mission failure scenarios and improve defect detection rates. Offspring individuals obtained based on key test case perturbations are also more likely to behave similarly to key test cases.

[0017] T6. Repeat step T5 until the iteration termination condition is met; output the individuals in the behavior space as test cases.

[0018] Constructing a behavior space and dividing it into cells, ensuring that each cell has at least one individual code, is to ensure that the final test cases more comprehensively cover the scenario conditions encountered when performing drone flight missions. Simultaneously, the behavior space is constructed based on task characteristics to guide the search direction for test cases.

[0019] The evaluation of each test case, i.e., the simulation process, consumes time and computing resources. However, resource budgets are limited. Therefore, under resource constraints, generating new individuals based on key test cases helps improve the discovery of key test cases, resulting in a final test case set containing a higher proportion of key samples, ultimately significantly improving the detection efficiency of failure scenarios.

[0020] It should be noted that the present invention and the existing genetic algorithms have some intuitive similarities. However, the individuals generated by the genetic algorithm are random and have no range coverage constraints. Although the quality of the solution is improved by evaluating individuals and performing genetics, it cannot guarantee that all generated solutions (test cases) can both meet the diversity requirements and cover all ranges of key failure scenarios.

[0021] Furthermore, the attributes of the obstacle include length, width, height, three-dimensional coordinates, and rotation angle; the numerical combination corresponding to each attribute is extracted to form the code of the obstacle; the individual is formed by the combination of codes of a specified number of obstacles.

[0022] For the test scenario, if the current scenario includes two obstacles, the generated individual code is represented as follows: The first 6 dimensions and the last 6 dimensions are the attribute codes for the two obstacles, with length, width, height, and rotation angle corresponding to... , w, h, r, corresponding to the three-dimensional coordinates x, y, z.

[0023] Furthermore, the individual includes at least two of the obstacles; in step T3, when generating the individual, a distance constraint is also applied to the individual.

[0024] The distance constraint is that the distance between the center points of any two obstacles is limited to a specified range; if the distance exceeds the specified range, the distance between the obstacles is readjusted; or the current individual is discarded.

[0025] Specifically, the range of distance adjustment adopts a repulsive rebound mechanism or an attractive traction mechanism. That is, one obstacle 1 is fixed, and when another obstacle 2 overlaps with obstacle 1 or is too far away, obstacle 2 is bounced away or pulled in the opposite direction along the line connecting the center points. This is used as a constraint to adjust the attribute value of obstacle 2.

[0026] Furthermore, the task characteristics include behavioral characteristics and obstacle characteristics;

[0027] In T2, multiple numerical intervals are constructed based on the task characteristics; the numerical intervals of different task characteristics are combined one by one to form the cell.

[0028] Specifically, obstacle features are the characteristic relationships between obstacles, while behavioral features are the characteristic relationships between tasks and obstacles.

[0029] For example, the angle between obstacles is considered as an obstacle feature; if the angle covers a span of 90°, then it is divided into 6 intervals, with each interval being 15°.

[0030] For example, the proportion of the cross-sectional length of an obstacle to the cross-sectional length of the task area can be used as a behavioral feature. If the total proportion is 100% and each numerical interval is 20%, then it is divided into 5 intervals.

[0031] Based on this, the two task features can be combined to form 30 cells, making the behavior space a two-dimensional spatial structure.

[0032] Due to limited resources, an excessively large task space (i.e., too many cells) leads to higher initialization complexity for individual codes, thus squeezing the exploration space for subsequent critical test cases. Therefore, the task space does not need to be set too large; that is, the number of numerical intervals does not need to be too large. For example, the number of cells can be limited to less than or equal to half the number of simulation tests.

[0033] Furthermore, in T3 and T5, the individual is mapped and stored in the cell according to the task characteristics and their numerical range.

[0034] Furthermore, in T4, the sampling involves selecting and retaining the individual with the smallest behavioral distance; the behavioral distance is the Euclidean distance between a specified point in the cell and the individual.

[0035] Specifically, in a two-dimensional action space, the bottom left corner of each cell can be specified as the designated point. Alternatively, other identical locations can be selected across all cells, such as the top right corner.

[0036] Specifically, each individual can calculate corresponding values ​​based on task characteristics, which serve as the individual's coordinates in the behavior space. For example, in an individual, if the angle between obstacles is 20° and the cross-sectional length of the obstacles occupies 30% of the cross-sectional length of the task area, then the coordinates are (20°, 30%).

[0037] Furthermore, the evaluation value is the true fitness obtained by the individual after the simulation test; the true fitness is the minimum Euclidean distance between the UAV's flight path and the obstacle; the flight path is obtained by the UAV using the individual as a test case in a simulation test.

[0038] Specifically, a distance of less than 1.5m is considered a soft failure, while a distance of 0m is considered a hard failure, i.e., a crash. Test cases that cause the drone to experience these scenarios are called critical test cases. The actual fitness values ​​decrease sequentially for distances of 0, 1.5m, or other larger distances.

[0039] Furthermore, in T5, a weighted probability is generated for the individual based on the evaluation value, a specified number of individuals are selected based on the weighted probability and using a roulette wheel method, and the sampling is completed for the specified individual and other individuals located in the same cell as the specified individual.

[0040] A higher evaluation value indicates a closer likelihood of being a critical test case, resulting in a higher assigned weight probability, or a higher probability of being sampled. Specifically, the weight probability is obtained by regularizing the evaluation value. Individuals within the same cell have high similarity; when one individual is evaluated as a critical test case, the others also have a high probability of being critical test cases.

[0041] Furthermore, the perturbation is achieved by adding random numbers that follow a Gaussian distribution to the encoded values ​​corresponding to the attributes of obstacles in the individual, based on the task characteristics.

[0042] Specifically, disturbances include location disturbances and behavioral disturbances.

[0043] Position perturbation involves adding random numbers to the three-dimensional coordinates of the obstacle.

[0044] Behavioral perturbations are perturbations targeting behavioral characteristics. For example, for the angle between obstacles, the rotation angle of one obstacle is selected and a random number is added to complete the perturbation. For the proportion of the cross-sectional length of an obstacle to the cross-sectional length of the task area, random numbers are added to the length, width, and height of both obstacles to complete the perturbation, and the rotation angle is perturbed by the same magnitude, that is, the same random number is added.

[0045] Furthermore, the iteration ends when the number of times the evaluation value is generated reaches a preset number.

[0046] Compared with the prior art, the present invention achieves the following beneficial effects:

[0047] This invention effectively guides the generation of UAV test cases by constructing and dividing a behavior space. It fills the behavior space and maintains solution diversity without requiring extensive simulations. This method significantly reduces the number of evaluations, concentrating limited resources on discovering key test cases, thereby improving the efficiency of failure scenario detection and ultimately generating a comprehensive, high-quality, and diverse set of test cases. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating a method for generating UAV test cases based on a two-layer quality diversity optimization algorithm, as provided in an embodiment of the present invention.

[0049] Figure 2 This is a schematic diagram of the properties of an obstacle provided in an embodiment of the present invention;

[0050] Figure 3 This is a visual schematic diagram of the distance constraint provided in an embodiment of the present invention;

[0051] Figure 4 This is a visual schematic diagram of obstacle features provided in an embodiment of the present invention;

[0052] Figure 5 This is a visual schematic diagram of the behavioral features provided in the embodiments of the present invention;

[0053] Figure 6 This is a schematic diagram of individual sampling in T4 provided in an embodiment of the present invention;

[0054] Figure 7 This is a schematic diagram of cell sampling in T5 provided in an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0056] like Figures 1 to 7 As shown, this embodiment provides a method for generating UAV test cases based on a two-layer quality diversity optimization algorithm. The test cases are used to generate obstacles and simulate the environment to interfere with and test UAV flight missions; the method includes the following steps:

[0057] T1. The attributes of obstacles are digitally encoded, with each individual representing a code. In this embodiment, obstacle attributes include length, width, height, three-dimensional coordinates, and rotation angle. The numerical combinations corresponding to each attribute are extracted to form the obstacle's code. An individual is formed by combining the codes of a specified number of obstacles. In this embodiment, each individual includes two obstacles, and the simulated test environment contains two obstacles that interfere with flight. The generated individual code is represented as follows: The first 6 dimensions and the last 6 dimensions are the attribute codes for the two obstacles, with length, width, height, and rotation angle corresponding to... , w, h, r, corresponding to the three-dimensional coordinates x, y, z.

[0058] In addition, when generating individuals, a distance constraint is imposed on them; the distance constraint is that the distance between the center points of any two obstacles is limited to a specified range. If the distance exceeds the specified range, the distance between the obstacles is readjusted.

[0059] Specifically, the range of distance adjustment adopts a repulsive rebound mechanism or an attractive traction mechanism. That is, one obstacle 1 is fixed, and when another obstacle 2 overlaps with obstacle 1 or is too far away, obstacle 2 is bounced away or pulled in the opposite direction along the line connecting the center points. This is used as a constraint to adjust the attribute value of obstacle 2.

[0060] In other embodiments, the current individual can be discarded directly, and a new individual that meets the constraints can be generated.

[0061] T2. For UAV flight missions, analyze and obtain mission characteristics. Specifically, mission characteristics include behavioral characteristics and obstacle characteristics; obstacle characteristics are the characteristic relationships between obstacles, and behavioral characteristics are the characteristic relationships between the mission and obstacles. Multiple numerical intervals are constructed based on the mission characteristics.

[0062] In this embodiment, the task features include a behavioral feature and an obstacle feature.

[0063] Among them, the obstacle feature is the angle between two obstacles (i.e., Figure 4 In the equation R = r1 - r2, the coverage span of the included angle is 90°, so it is divided into 6 intervals of 15° each. The behavioral characteristic is the proportion of the cross-sectional length of the obstacle to the cross-sectional length of the task area (i.e.,...). Figure 5 (L1+L2) / L3). The total proportion is 100%, and each numerical interval is 20%, so it is divided into 5 intervals.

[0064] A behavior space is constructed based on task features. The behavior space consists of multiple discrete and evenly spaced cells; that is, the numerical ranges of different task features are combined one by one to form cells. For example, combining the ranges of the first two features can form a two-dimensional behavior space containing 30 cells.

[0065] Due to limited resources, an excessively large task space (i.e., too many cells) leads to higher initialization complexity for individual codes, thus squeezing the exploration space for subsequent critical test cases. Therefore, the task space does not need to be set too large; that is, the number of numerical intervals does not need to be too large. For example, the number of cells can be limited to less than or equal to half the number of simulation tests.

[0066] In this embodiment, the number of simulation tests (i.e., the number of evaluations / resource budget) is set to 100. The iteration termination condition described below is that the number of times the evaluation value is generated reaches the preset number, i.e., 100 times.

[0067] T3. Randomly initialize and generate multiple individuals, and fill the cells with these individuals, ensuring that each cell contains at least one individual. Specifically, based on the task characteristics and their numerical ranges, map and store the individuals in the cells. Each individual's code can be calculated based on the task characteristics to obtain a corresponding numerical value, which serves as the individual's coordinate value in the behavior space. For example, in an individual where the angle between obstacles is 20° and the cross-sectional length of the obstacles occupies 30% of the cross-sectional length of the task area, the coordinates would be (20°, 30%). The corresponding cell would then be [15°-30°, 20%-40%).

[0068] T4. Sample each cell and retain one individual; specifically, select the individual with the smallest behavioral distance; the behavioral distance is the Euclidean distance between the specified point of the cell and the individual. In this embodiment, in the two-dimensional behavioral space, the lower left corner of each cell is specified as the specified point of the cell.

[0069] Simulation tests are performed on the sampled individuals to generate evaluation values ​​for them. In this embodiment, the evaluation value is the true fitness of the individual after simulation testing. The true fitness is the minimum Euclidean distance between the UAV's flight path and obstacles. The flight path is obtained by simulating the UAV with individuals as test cases.

[0070] Specifically, a distance less than 1.5m is considered a soft fault, and a distance of 0m is considered a hard fault, i.e., a crash. Test cases that cause the drone to experience these scenarios are called critical test cases. Assuming the distance is 0m, the true fitness value is 3; the distance is 0-1.5m, the true fitness value is 2; and the distance is greater than 1.5m, the true fitness value is 0.5. In other words, the true fitness value decreases as the distance increases.

[0071] T5. Based on the evaluation value, generate weighted probabilities for each individual. Then, select a specified number of individuals using a roulette wheel method based on these weighted probabilities. Sampling is then performed on the specified individual and other individuals located in the same cell as it; that is, sampling is performed on the cell corresponding to the specified individual. Specifically, the weighted probabilities are obtained by regularizing the evaluation value.

[0072] A higher evaluation value indicates a closer likelihood of being a critical test case, resulting in a higher assigned weight probability and thus a higher probability of being sampled. Individuals within the same cell exhibit high similarity; when one individual is evaluated as a critical test case, the others also have a high probability of being critical test cases.

[0073] The sampled individuals are perturbed to generate offspring individuals; in this embodiment, the perturbation is to add random numbers that follow a Gaussian distribution to the encoded values ​​of the obstacle attributes in the individuals, based on the task characteristics.

[0074] Specifically, the perturbations include location perturbations and behavioral perturbations, that is, local searches for new individuals are achieved based on location perturbations and behavioral perturbations.

[0075] Position perturbation involves adding random numbers to the three-dimensional coordinates of the obstacle.

[0076] Behavioral perturbations are perturbations targeting behavioral characteristics. For example, for the angle between obstacles, the rotation angle of one obstacle is selected and a random number is added to complete the perturbation. For the proportion of the cross-sectional length of an obstacle to the cross-sectional length of the task area, random numbers are added to the length, width, and height of both obstacles to complete the perturbation, and the rotation angle is perturbed by the same magnitude, that is, the same random number is added.

[0077] Through simulation testing, evaluation values ​​are generated for offspring individuals, which are then stored in the behavior space. The higher the evaluation value, the greater the probability of sampling a key test case. Key test cases can efficiently trigger drone mission failure scenarios, improving the defect detection rate. Offspring individuals obtained based on key test case perturbations are also more likely to behave similarly to key test cases.

[0078] T6. Repeat step T5 until the iteration termination condition is met; output the individuals in the behavior space as test cases.

[0079] Constructing a behavior space and dividing it into cells, ensuring that each cell has at least one individual code, is to ensure that the final test cases more comprehensively cover the scenario conditions encountered when performing drone flight missions. Simultaneously, the behavior space is constructed based on task characteristics to guide the search direction for test cases.

[0080] The evaluation of each test case, i.e., the simulation process, consumes time and computing resources. However, resource budgets are limited. Therefore, under resource constraints, generating new individuals based on key test cases helps improve the discovery of key test cases, resulting in a final test case set containing a higher proportion of key samples, ultimately significantly improving the detection efficiency of failure scenarios.

[0081] It should be noted that the present invention and the existing genetic algorithms have some intuitive similarities. However, the individuals generated by the genetic algorithm are random and have no range coverage constraints. Although the quality of the solution is improved by evaluating individuals and performing genetics, it cannot guarantee that all generated solutions (test cases) can both meet the diversity requirements and cover all ranges of key failure scenarios.

[0082] Based on the disclosure and teachings of the foregoing specification, those skilled in the art can make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and some modifications and changes to the present invention should also fall within the protection scope of the claims of the present invention. Furthermore, although some specific terms are used in this specification, these terms are only for convenience of explanation and do not constitute any limitation on the present invention.

Claims

1. A method for generating test cases of unmanned aerial vehicle based on double-layer quality diversity optimization algorithm, characterized in that, The test case is used to generate obstacles and simulate an environment to interfere with and test a UAV flight task; comprising the following steps: T1, encoding the attributes of the obstacles to digitize, representing the encoding as individuals; T2, analyzing the task characteristics for the UAV flight task; The task characteristics include behavior characteristics and obstacle characteristics; Based on the task characteristics, a flight space is constructed, which includes a plurality of cells that are discrete and uniformly spaced; wherein, according to the task characteristics, a plurality of numerical intervals are constructed; the numerical intervals of different task characteristics are combined one by one to form the cells; T3, randomly initializing to generate a plurality of individuals, and filling the individuals into the cells, so that each cell includes at least one individual; T4, sampling and retaining one individual for each cell respectively; simulating the individuals to generate evaluation values for the individuals; T5, sampling the individuals based on the evaluation values; wherein, according to the evaluation values, a weight probability is generated for the individuals, a specified number of individuals are selected according to the weight probability and using the roulette method, and the sampling is completed for the specified individuals and other individuals located in the same cell as the specified individuals; the sampled individuals are disturbed to generate offspring individuals; evaluation values are generated for the offspring individuals, and the offspring individuals are stored in the flight space; T6, repeat step T5 until the iteration end condition is reached; output the individuals in the flight space as test cases.

2. The unmanned aerial vehicle test case generation method based on a double-layer quality diversity optimization algorithm according to claim 1, characterized in that, The attributes of the obstacles include length, width, height, three-dimensional coordinates and rotation angle; the numerical combination corresponding to each attribute is extracted to form the encoding of the obstacles; the individual is formed by combining a specified number of encodings of obstacles.

3. The method of claim 1, wherein the method is based on a double-layer quality diversity optimization algorithm. The individual contains at least two obstacles; in T3, when generating the individual, a distance constraint is also applied to the individual; The distance constraint is that the distance between the center points of any two obstacles is limited to a specified range; if it exceeds the specified range, the distance between the obstacles is adjusted; or the current individual is discarded.

4. The unmanned aerial vehicle test case generation method based on a double-layer quality diversity optimization algorithm according to claim 1, characterized in that, In T3 and T5, according to the task characteristics and their numerical intervals, the individuals are mapped and stored in the cells.

5. The method of claim 1, wherein the method is based on a double-layer quality diversity optimization algorithm. In T4, the sampling is to select and retain the individual with the smallest behavior distance; the behavior distance is the Euclidean distance between the specified point of the cell and the individual.

6. The unmanned aerial vehicle test case generation method based on a double-layer quality diversity optimization algorithm according to claim 1, characterized in that, The evaluation value is the real fitness obtained after the simulation test of the individual; The real fitness is the minimum Euclidean distance between the UAV's flight path and the obstacles; the flight path is obtained by simulating the UAV with the individual as a test case.

7. The method of claim 1, wherein the method is based on a double-layer quality diversity optimization algorithm. The disturbance is to add a random number subject to a Gaussian distribution to the encoding numerical value corresponding to the attributes of the obstacles in the individual according to the task characteristics.

8. The method of claim 1, wherein the method is based on a double-layer quality diversity optimization algorithm. The iteration end condition is that the number of generated evaluation values reaches a preset number.

Images