An Adaptive Generation Method for Spacecraft Test Cases Based on Domain Knowledge

Abstract

This invention discloses an adaptive test case generation method for spacecraft based on domain knowledge, comprising: 1. Obtaining the parameter space of the spacecraft under test, and determining the probability distribution model, normalization, and distance metric method for each parameter; 2. Randomly generating an initial test case in the parameter space according to the probability distribution model and placing it into the test case set E; 3. Randomly generating a candidate test case set C in the parameter space according to the probability distribution model; 4. Representing the test cases as vectors, selecting the candidate test case with the largest shortest distance to the test case set E from the candidate test case set C, and placing it into the test case set E; 5. Continuously executing steps 3 and 4 above until enough test cases are generated in the test case set E. This invention makes the generated test case set closer to the actual testing requirements of spacecraft, enhances the effectiveness of the test case set, and helps to improve testing efficiency.

Description

Technical Field

[0001] This invention belongs to the field of software technology of algorithm design, and in particular relates to an adaptive generation method for spacecraft test cases based on domain knowledge. Background Technology

[0002] Spacecraft systems are large in scale and complex. When launched into space from the ground, they undergo multiple states, including waiting mode, spacecraft separation, initial offset correction, solar array deployment, re-rotation correction, solar search, Earth search offset, Earth search, Earth pointing, and normal mode. Therefore, the number of system operating states is large, and the relationships between these states are complex, leading to an explosive growth in the test space. Traditional test case design relies entirely on the experience of testers, resulting in high testing costs and a degree of uncertainty in the testing process, making it difficult to ensure that the designed test scenarios effectively cover the functional requirements of the spacecraft. Therefore, a method for autonomously generating test cases for spacecraft systems is of significant research value.

[0003] Software failure modes refer to the shape, size, and other information of the data that causes program failures distributed within the input domain. Failures include three typical failure modes: point failure modes, strip failure modes, and block failure modes. Inputs that cause failures tend to form continuous regions; when test cases are evenly distributed, the ability to detect faults is stronger. Traditional adaptive random testing methods can generate test cases that are distributed as evenly as possible within the input domain by measuring the similarity between test cases. However, spacecraft systems have high-dimensional input spaces and complex data types, resulting in extremely high-dimensional input spaces. For example, the controller of a spacecraft system has important input parameters such as ground control commands, satellite position coordinates, satellite velocity coordinates, 3+1S gyroscope attitude angular velocity, fiber optic gyroscope attitude angular velocity, star-sensitive optical axis right ascension and declination, ground-sensitive attitude angle, sun-sensitive solar angle, solar array appearance marker, solar array solar angle, momentum wheel speed, and momentum wheel expected torque. With such a vast input space, the effectiveness of traditional distance-based adaptive random testing methods decreases sharply.

[0004] To address the shortcomings of traditional adaptive random testing methods in testing spacecraft systems, this invention improves upon traditional algorithms using domain knowledge. In designing test cases for spacecraft, test engineers often use empirical data, known as typical values. Typical values ​​and their surrounding data are more likely to trigger corresponding defects, leading to failures. This invention aims to incorporate this "experience" into the automatic generation of test cases for spacecraft. Therefore, it proposes an adaptive test case generation method suitable for the characteristics of spacecraft. This method describes test parameters using a probability distribution model and randomly generates test cases based on this model. While ensuring sufficient randomness of parameters within their value range, this method integrates the domain experience of spacecraft experts, automating test case generation. This results in a test case set that more closely reflects the actual testing needs of spacecraft, enhancing the effectiveness of the test case set, improving testing efficiency, and reducing testing overhead. Summary of the Invention

[0005] The purpose of this invention is to propose an adaptive test case generation algorithm for spacecraft based on domain knowledge. This algorithm integrates the domain experience of spacecraft experts to improve the effectiveness of the generated test case set, making the generated test case set closer to the actual testing needs of spacecraft, thereby improving testing efficiency and reducing testing costs.

[0006] The technical solution of this invention is: an adaptive generation method for spacecraft test cases based on domain knowledge, comprising the following steps:

[0007] Step 1) Obtain the parameter space of the spacecraft system under test. Based on the type and range of different parameters and related domain knowledge, determine the probability distribution model, normalization and distance measurement method of each parameter respectively.

[0008] Step 2) Based on the probability distribution model determined in Step 1), randomly generate an initial test case in the parameter space and put it into the test case set E;

[0009] Step 3) Based on the probability distribution model determined in Step 1), randomly generate m candidate test cases in the parameter space to form a candidate test case set C;

[0010] Step 4) Represent the test cases as vectors, and normalize each parameter component of the vector according to the normalization and distance metric method determined in Step 1). Then calculate the distance between each test case in the candidate set C and all test cases in the test case set E, and select the candidate test case with the largest shortest distance to put into the test case set E.

[0011] Step 5) Continue executing steps 3) and 4) until enough test cases are generated in test case set E.

[0012] Preferably, the probability distribution model for each of the parameters determined in step 1) is described by a normal distribution model.

[0013] Preferably, the normalization method described in step 1) includes deviation standardization, etc., to standardize the data of the multiple parameters.

[0014] Preferably, the probability distribution model for determining the respective parameters in step 1) uses a uniform distribution to describe the probability distribution of the parameters.

[0015] This invention is an adaptive test case generation method for spacecraft based on domain knowledge. Compared with traditional adaptive random testing methods, this method has the following advantages: This invention integrates domain experience in spacecraft, designs corresponding probability distribution models for some parameters, and scientifically generates and adaptively selects test cases based on the distribution models. Thus, while ensuring sufficient randomness of parameters within their value range, it can generate more test cases near typical values, making the generated test case set closer to the actual testing needs of spacecraft, enhancing the effectiveness of the test case set, helping to improve testing efficiency, and reducing testing costs. Attached Figure Description

[0016] Figure 1 Flowchart of the domain knowledge-based adaptive generation algorithm for spacecraft test cases Figure 2 Launch time normal distribution probability density curve

[0017] Figure 3 Initial angular velocity (attitude_angle_velocity_z) normal distribution probability density curve Figure 4 Comparison of defect detection efficiency results between RT, ART, and E-ART methods based on the F-measure evaluation index. Detailed Implementation

[0018] The invention will now be described in further detail with reference to the accompanying drawings. Figure 1 The specific implementation of the technical solution of the present invention will be described in detail below, and the specific steps are as follows:

[0019] Step 1) Obtain the parameter space of the spacecraft system under test. Based on the different types and ranges of parameters and related domain knowledge, determine the probability distribution model, normalization and distance measurement method for each parameter.

[0020] This invention categorizes spacecraft test parameters into two types: parameters with typical values ​​and parameters without typical values. For the first type of parameters, a normal distribution model is used; for the second type, a uniform distribution is used. The normalization method is deviation standardization (described below), the main purpose of which is to standardize the data for all parameters. Euclidean distance is used as the distance metric. An example is provided below to illustrate this.

[0021] When testing spacecraft systems, there are multiple test objects, including propulsion systems, control computers, star sensors, Earth sensors, digital sun sensors, liquid-floating gyroscopes, and star models. These test objects all involve complex inputs, outputs, and initial states. We use the spacecraft's central control computer as the experimental object, selecting two important initial parameters during the satellite launch phase: launch time and initial attitude angular velocity. The launch time (launch_time) is the satellite launch time preset by the ground control center. It determines the positional relationship between the Sun, Earth, and the satellite during the subsequent launch process, significantly affecting the satellite's positioning and alignment actions in space. `launch_time` is a continuous real number containing six components: year, month, day, hour, minute, and second, all belonging to the same dimension. The launch time range is taken as 00:00:00 on January 1, 2000 to 00:00:00 on January 1, 2001. Based on the experience of spacecraft experts, the optimal launch time for this satellite was approximately one month before or after 03:00:00 on April 5, 2000. As the launch time deviates from this optimal time, the probability of successful launch gradually decreases. Taking the relative value of the start time as 0 seconds, the end time is 23,760,000 seconds. Therefore, a probability distribution model for `launch_time` can be established as a normal distribution X ~ N(μ,σ) within the range [0, 23,760,000]. 2 ), where μ is 8218800 and σ is 1290600, and its probability density curve is as follows. Figure 2 As shown; the initial attitude angular velocity, attitude_angle_velocity, is the attitude angular velocity acquired by the satellite after separation from the launch vehicle. It is also a continuous real number. We take the Z-axis component, attitude_angle_velocity_z, as an example for analysis, taking the initial angular velocity range from -1 rad / s to 1 rad / s. Experts in the field often choose to test near the typical value of 0.1 rad / s. Therefore, we can also establish a probability distribution model for attitude_angle_velocity_z as a normal distribution X ~ N(μ, σ) within the range of [-1, 1]. 2), where μ is 0.1 and σ is 0.05, and its probability density curve is as follows. Figure 3 As shown. To ensure that parameters with different dimensions are on the same order of magnitude and to reduce the influence of parameters with large variances, a deviation standardization method is used to normalize the input parameters. The formula is as follows: Where max is the maximum value of the parameter, min is the minimum value of the parameter, and z is the original parameter. ＊ The parameter is a normalized parameter, and its value range is [0, 1]. For continuous parameters, we choose Euclidean distance as the distance metric for test cases, and its formula is: Where, x i Let y represent the value of the i-th parameter in test case x. i Let represent the i-th parameter value of test case y, n be the number of parameters in the test case, and dist(x, y) be the distance between test cases x and y.

[0022] The above example describes step 1) of generating test cases involving two parameters. If other parameters are involved, and domain experts have no typical or empirical values ​​for these parameters, a uniform distribution is used to describe the probability distribution of the parameters. The following steps will continue to use this example for description;

[0023] Step 2) Based on the probability distribution model determined in Step 1), randomly generate an initial test case in the parameter space and put it into the test case set E.

[0024] When randomly generating test cases for the satellite control computer during the launch phase, the launch time (launch_time) is distributed according to a normal distribution X ~ N(8218800, 1290600) within the input range of [0, 23760000]. 2 The attitude angular velocity (attitude_angle_velocity_z) is generated according to a normal distribution X ~ N(0.1, 0.05) within the input range of [-1, 1]. 2 The generated test cases are then created by combining the values ​​of these two parameters. These combined test cases are used as the initial test cases and placed into the test case set E.

[0025] This step demonstrates the difference between the invention's method and adaptive random testing, replacing "random" generation with a method that uses a probabilistic model to generate test cases.

[0026] Step 3) Based on the probability distribution model determined in Step 1), randomly generate m candidate test cases in the parameter space, forming a candidate test case set C. Randomly generate parameter values ​​for launch_time and attitude_angle_velocity_z according to their normal distribution models, and combine them into candidate test cases. Repeat this process m times, generating a candidate test case set C = {C1, C2, ..., C...}. k The number of candidate test cases, m, generated each time can be adjusted based on experience; the default value for m is 10.

[0027] Step 4) Represent the test cases as vectors, and normalize each parameter component of the vector according to the normalization and distance metric method determined in Step 1). Then calculate the distance between each test case in the candidate set C and all test cases in the test case set E, and select the candidate test case with the largest shortest distance to put into the test case set E.

[0028] During the testing of the satellite control computer, the generated test cases are represented as a vector form t = (launch_time, attitude_angle_velocity_z). The test case vector is normalized according to the deviation normalization method determined in step 1), and the normalized vector is... Calculate the candidate test case set C = {c1, c2, ..., c...} m Each test case c in} j (1≤j≤m) and the test case set E={e1,e2,...,e n Find the shortest distance among all test cases within the range, and select the candidate test case c with the largest shortest distance. max Add it to test case set E, and its formula is: Where dist(c, e) represents the distance between two test cases c and e, calculated according to the Euclidean distance metric formula determined in step 1).

[0029] Step 5) Continuously execute steps 3) and 4) until a sufficient number of test cases are generated in the test case set E. In this invention, this number is 6.

[0030] In the process of generating and testing test cases using adaptive random testing methods, F-measure is often used as an evaluation metric. It represents the number of test cases required to detect the first failure. The smaller the F-measure value, the higher the effectiveness of the test case set generated by the test case generation algorithm. Since the adaptive random testing process is significantly affected by the first randomly generated test case, multiple independent repeated experiments are needed to take the average value. In this invention, the default number of repeated experiments is 100. The F-measure values ​​of the repeated experiments are plotted using box plots. The domain knowledge-based adaptive test case generation algorithm for spacecraft (E-ART) proposed in this invention is compared with the random generation algorithm (RT) and the adaptive testing algorithm (ART). The results are as follows: Figure 4 As shown, the x-axis represents the three testing strategies, the y-axis represents the F-measure value, the lower and upper bounds of the box plot represent the first and third quartiles, and the horizontal line in the middle represents the median. Clearly, the F-measure of E-ART is generally smaller than the other two methods, strongly demonstrating that the E-ART method proposed in this invention is more effective in generating test case sets.

[0031] This invention allows testers to change the number of test cases to be generated, and this step can continue to generate test cases until the maximum acceptable test overhead is reached.

Claims

1. An adaptive generation method for spacecraft test cases based on domain knowledge, characterized in that: The method includes the following steps: Step 1) Obtain the parameter space of the spacecraft system under test. Based on the different types, ranges, and domain knowledge of the parameters, determine the probability distribution model, normalization, and distance measurement method for each parameter. The test parameters of the spacecraft are divided into two categories: the first type is parameters with typical values, and the second type is parameters without typical values. For the first type of parameter, a normal distribution model is used to describe it, and for the second type of parameter, a uniform distribution model is used to describe it. Euclidean distance is used for distance measurement. The normalization method mentioned in Step 1) includes deviation standardization to standardize the data of the various parameters. Step 2) Based on the probability distribution model determined in Step 1), randomly generate an initial test case in the parameter space and put it into the test case set E; Step 3) Based on the probability distribution model determined in Step 1), randomly generate m candidate test cases in the parameter space to form a candidate test case set C; Step 4) Represent the test cases as vectors, and normalize each parameter component of the vector according to the normalization and distance metric method determined in Step 1). Then calculate the distance between each test case in the candidate set C and all test cases in the test case set E, and select the candidate test case with the largest shortest distance to put into the test case set E. Step 5) Continuously execute steps 3) and 4) until enough test cases are generated in test case set E; By integrating experience from the spacecraft field, we designed corresponding probability distribution models for some parameters, and based on the distribution models, we randomly generated and adaptively selected test cases. This ensured that the parameters had sufficient randomness within their range of values, while generating more test cases near typical values.

Images