RESTful API test case generation and optimization method based on cyber-physical system
By analyzing the call relationship of the RESTful API and optimizing the test suite using the NSGA-II algorithm, the test cost and time cost problems caused by the increase in the number of test cases in the information physics system are solved, efficient fault detection and test case design are achieved, and performance is generated. A better test suite.
Patent Information
- Application Number
- CN202210071709.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-21
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2042-01-21
AI Technical Summary
In information physics systems, as the number of RESTful APIs increases, the number of test cases increases, resulting in increased testing costs and time costs, and a minimized test suite may not cover all test functions and reduce fault detection capabilities, making it difficult to be limited Execute valid test cases within time.
By analyzing the call relationship between RESTful APIs, designing test cases, and defining optimization goals, minimizing test suites using NSGA-II algorithm, setting optimization goals including failure detection percentage and test case execution time, optimize test suites to improve failures Detection capability and reduce testing costs.
It realizes efficient test case design in information physics systems, generates test suites with better performance, can effectively detect faults and reduce test costs within a limited time.
Smart Images
Figure CN114610600B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of software testing, and in particular relates to a method for generating and optimizing RESTful API test cases in an information-physical system. Background Art
[0002] Cyber-Physical System (CPS) is an autonomous system that closely integrates computing, communication and process control technologies. It uses the collaboration between physical objects by equipping them with interfaces to information devices and combining them with intelligent mechanisms. Using the RESTful style to define interfaces in cyber-physical systems can easily implement cross-platform calls, allowing users using different programming languages to access resources in cyber-physical systems.
[0003] For testing RESTful APIs in cyber-physical systems, a test suite is usually developed to test them. However, as the number of RESTful APIs in cyber-physical systems increases, the number of test cases will also increase. Considering the time and resource costs, it is actually impossible to execute all test cases. Therefore, it is necessary to find a solution to effectively minimize the test suite before executing the test cases to reduce the testing cost.
[0004] There are two potential problems with test suite minimization (TSM): first, the minimized test suite may not cover all test functions (i.e., test requirements); second, the minimized test suite may have lower fault detection capabilities than the original test suite. Therefore, in actual testing work, it is necessary to screen test cases based on various costs (e.g., test case execution time) and effectiveness criteria (e.g., fault detection capabilities). The overall goal of test case selection is to select test cases that can be executed within a limited time budget while optimally meeting various cost and efficiency goals. Therefore, we also face the challenge of testing RESTful APIs in cyber-physical systems, which is the need to minimize the test suite used to test the product while achieving high fault detection capabilities. Summary of the invention
[0005] Purpose of the invention: In order to test RESTful API in information-physical system and reduce testing costs, the present invention provides a method for generating and optimizing test cases for RESTful API in information-physical system.
[0006] Technical solution: The present invention provides a method for generating and optimizing RESTful API test cases in a cyber-physical system, comprising the following steps:
[0007] (1) Analyze the mutual call relationship between the RESTful APIs in the tested cyber-physical system according to the logical order of function implementation, and draw a call relationship diagram;
[0008] (2) Design test cases based on the call relationship diagram and the attributes of each RESTful API;
[0009] (3) Define the optimization goals of the test suite for RESTful APIs in cyber-physical systems and determine the parameter settings for the optimization goals;
[0010] (4) A test suite minimization method for RESTful APIs in cyber-physical systems is proposed using the NSGA-II algorithm based on the optimization objective.
[0011] Furthermore, the step (1) comprises the following steps:
[0012] (11) Determine the priority level of each RESTful API according to the characteristics of the RESTful API in the cyber-physical system under test; when determining the priority level of each RESTful API, it is necessary to divide each RESTful API into the following levels according to its function: define the basic control function API as the Control layer; define the complex manipulation function API as the Maneuvering layer; define the logical strategy function API as the Strategic layer;
[0013] (12) Manually analyze the calling sequence relationship between various RESTful APIs;
[0014] (13) Draw a call graph based on the calling relationship.
[0015] Furthermore, the step (2) comprises the following steps:
[0016] (21) When designing test cases, the API value range must be considered and the test input values must be determined based on the provided API design instructions;
[0017] (22) When designing test cases, call the required APIs in sequence according to the call relationship diagram in step (1).
[0018] Furthermore, the step (3) comprises the following steps:
[0019] (31) When testing the RESTful API in the cyber-physical system, when the test results are inconsistent with the expected results, faults are detected from the test case results; in the face of these faults, the cause of each fault is found, its impact on the operation of the cyber-physical system is analyzed, and then these faults are classified to determine the severity of the fault;
[0020] (32) The average fault detection percentage is an important parameter used to judge the fault detection capability of the test case. The average fault detection percentage is selected as one of the optimization objectives;
[0021] (33) The test case execution time can be used to evaluate the quality of a test case, and the test case execution time is selected as one of the optimization goals.
[0022] Furthermore, the step (4) comprises the following steps:
[0023] (41) Initialize the population and construct a non-dominated solution set based on the dominance relationship:
[0024] The purpose of classifying and sorting the Pop group is to divide it into several non-intersecting subgroups, and the basis for classifying and sorting individuals is the Pareto dominance relationship;
[0025] Assume that the size of the group Pop is N, and classify and sort the group Pop into m subsets P 1 , P 2 ,…,P m , let two vectors {n p} and {s p}where p∈Pop,n p Record the number of dominant individuals p, s p Record the set of individuals dominated by individual p, that is:
[0026] n p =|{q|q>pp,q∈Pop}|
[0027] s p =|{q|p>qp,q∈Pop}|
[0028] Calculate n for each individual through a double loop p and p , follow the method P k = {all individuals q|n q -k+1=0} to find P 2 , P 3 ;
[0029] The construction process of non-dominated sets is divided into two parts. The first part is used to calculate n i and i , and find P 1 The time required is (rN 2 ), r is the target number, N is the size of the evolutionary population; the second part is used to find P 2 , P 3 ...P mIn the worst case, an evolutionary population of size N has N layers of Pareto Front sets, and the time complexity is O(N 2 );
[0030] For each individual i in the population, there are two parameters n i and i ,n i is the number of individuals in the population that dominate the solution of individual i, s i is the set of individuals whose solutions are dominated by individual i;
[0031] Find all n in the population i = 0, store them in the current non-dominated set rank1; for each individual j in the current non-dominated set rank1, traverse the individual set s dominated by it j , the set s j For each individual t in n t are all subtracted by 1, that is, the number of individuals that dominate the solution of individual t is reduced by 1; if n t -1=0, then store individual t in another set H; take rank1 as the set of first-level non-dominated individuals; rank1 only dominates individuals, and is not dominated by any other individuals, and assign the same non-dominated order to all individuals in the set, and then continue to perform the above-mentioned grading operation on set H, and also assign the corresponding non-dominated order, until all individuals are graded, that is, they are all assigned the corresponding non-dominated order, and finally obtain the non-dominated solution set;
[0032] (42) According to the three optimization objectives, namely, the fitness function, set the parameters of the selection operator, the crossover operator, and the mutation operator; and select the data source;
[0033] (43) Run the operator, output the results, and merge the streamlined test cases into a minimized test suite.
[0034] Furthermore, the faults in step (31) are divided into four levels, namely fatal faults, serious faults, general faults and minor faults.
[0035] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are as follows: the design of the test case fully considers the characteristics of the information-physical system and is more suitable for the testing of the information-physical system. Compared with the original test suite, the method defines three optimization goals, and minimizes the test suite through the optimization goals to obtain a test suite with better performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 Flowchart of the present invention;
[0037] Figure 2To determine the optimization goal, a minimized test suite flow chart is obtained;
[0038] Figure 3 This is a RESTful API hierarchy diagram of the present invention. DETAILED DESCRIPTION
[0039] The present invention is further described in detail below with reference to the accompanying drawings.
[0040] The present invention proposes a method for generating and optimizing RESTful API test cases based on cyber-physical systems. Figure 1 As shown, the following steps are included:
[0041] Step 1: Manually analyze the mutual calling relationships between the RESTful APIs in the cyber-physical system under test and draw a calling relationship diagram.
[0042] RESTful API classification. First, analyze the functions of RESTful API in the cyber-physical system. The functions of RESTful API are the key to supporting the normal operation of the cyber-physical system, and different functions have different corresponding implementation methods. Analyze the working method of RESTful API to mark the function of the RESTful API.
[0043] When determining the priority level of each RESTful API, it is necessary to divide each RESTful API into three levels according to its function: the basic control function corresponds to the Control layer; the complex manipulation function API corresponds to the Maneuvering layer; the logical strategy function API corresponds to the Strategic layer. According to the functions of each RESTful API obtained by analysis, the corresponding RESTful API is divided into the corresponding level. Manually analyze the call sequence relationship between each RESTful API; draw a call graph based on the call relationship.
[0044] Step 2: Design test cases based on the call relationship diagram and the properties of each RESTful API.
[0045] (21) When designing test cases, you need to consider the API's value range and determine the test input value based on the provided API design instructions.
[0046] (22) When designing test cases, call the required APIs in sequence according to the call relationship diagram in step (1).
[0047] Create a Python project in Pycharm. Do not include Chinese characters, spaces, or other special symbols in the project directory. Import all test cases.
[0048] Import the following classes:
[0049] Flask class: used to call the Flask framework to write RESTful API.
[0050] Request class: used to obtain data in the request RESTful API, where the acquisition methods include post, get, etc.
[0051] json class: used to parse JSON objects from strings.
[0052] pprint class: The pprint module provides methods for printing arbitrary Python data structures.
[0053] time class: used to obtain the current time status
[0054] The numerical collection of the optimization target fault severity level is labeled as Fault Level; the numerical collection of the optimization target average fault detection percentage is labeled as Average Percentage of Detected; the numerical collection of the optimization target test case execution time is labeled as Time Execution; the data of each test case is collected according to the definition of the optimization target and stored in the database.
[0055] Step 3: Define the optimization goals of the test suite for the RESTful API in the cyber-physical system and determine the parameter settings for the optimization goals. Figure 2 As shown, the following steps are included
[0056] (31) When testing the RESTful API in a cyber-physical system, when the test results are inconsistent with the expected results, faults are detected from the test case results. Faced with these faults, it is necessary to try to find the cause of each fault, analyze its impact on the operation of the cyber-physical system, and then classify these faults to determine the severity of the fault. The fault severity level is selected as one of the optimization goals.
[0057] The faults in cyber-physical systems are roughly divided into four levels, namely fatal faults, serious faults, general faults and minor faults.
[0058] (32) The average fault detection percentage is an important parameter used to judge the fault detection capability of a test case. The average fault detection percentage is selected as one of the optimization objectives.
[0059] (33) The test case execution time can be used to evaluate the quality of a test case, and the test case execution time is selected as one of the optimization goals.
[0060] The numerical collection of the optimization target fault severity level is labeled as Fault Level; the numerical collection of the optimization target average fault detection percentage is labeled as Average Percentage of Detected; the numerical collection of the optimization target test case execution time is labeled as Time Execution; the data of each test case is collected according to the definition of the optimization target and stored in the database.
[0061] The faults that occur in the information-physical system are roughly divided into four levels, namely fatal faults, serious faults, general faults and minor faults. Minor faults: Faults that have no effect on the function of the information-physical system or are caused by external interference are considered minor faults. General faults: Faults that have no effect on the overall operation of the information-physical system and only affect some functions are considered general faults. Serious faults: Faults that have a certain impact on the overall operation of the information-physical system and affect some functions are considered serious faults. Fatal faults: Faults that have a significant impact on the overall operation of the information-physical system and cause the collapse of the entire information-physical system are considered fatal faults.
[0062] The average percentage of failure detection (APFD) of a test case can be measured as follows:
[0063]
[0064] Where APFD is the average percentage of fault detection; NumSuc tci is the number of successful executions of test case i within a given number of executions; NumFail tci It is the number of execution failures of test case i within a given number of executions; the test case execution time is based on the actual execution time.
[0065] Step 4: Use the NSGA-II algorithm to minimize the test suite of RESTful API in cyber-physical systems according to the optimization goal.
[0066] (4.1) The NSGA-Ⅱ algorithm is used to screen and optimize the three optimization objectives of the test case. First, the population is initialized and the non-dominated solution set is constructed according to the dominance relationship;
[0067] The purpose of classifying and sorting the Pop group is to divide it into several non-intersecting subgroups, and the basis for classifying and sorting individuals is the Pareto dominance relationship.
[0068] Assume that the size of the group Pop is N, and classify and sort the group Pop into m subsets P 1 , P 2 ,…,P m , let two vectors {np} and {s p}where p∈Pop,n p Record the number of dominant individuals p, s p Record the set of individuals dominated by individual p, that is,
[0069] n p =|{q|q>pp,q∈Pop}|
[0070] s p =|{q|p>qp,q∈Pop}|
[0071] First, a double loop is used to calculate the n of each individual p and p , then press method P k = {all individuals q|n q -k+1=0} to find P 2 , P 3 .
[0072] The construction process of non-dominated sets is divided into two parts. The first part is used to calculate n i and i , and find P 1 The time required is (rN 2 ), where r is the target number and N is the size of the evolutionary population; the second part is used to find P 2 , P 3 ...P m In the worst case, an evolutionary population of size N has N layers of Pareto Front sets, and the time complexity is O(N 2 ).
[0073] For each individual i in the population, there are two parameters n i and i ,n i is the number of individuals in the population that dominate the solution of individual i, s i is the set of individuals whose solutions are dominated by individual i.
[0074] Find all n in the population i = 0, store them in the current non-dominated set rank1; for each individual j in the current non-dominated set rank1, traverse the individual set s dominated by it j , the set s j For each individual t in n t All are subtracted by 1, that is, the number of individuals in the solution that dominates individual t is reduced by 1 (because the individual j that dominates the individual has been stored in the current non-dominated set). If n t-1=0, then store individual t in another set H; take rank1 as the set of first-level non-dominated individuals, so the solution individual in rank1 is the best. It only dominates individuals, but is not dominated by any other individuals. All individuals in the set are given the same non-dominated order, and then continue to perform the above-mentioned classification operation on set H, and also assign the corresponding non-dominated order, until all individuals are classified, that is, they are all assigned the corresponding non-dominated order, and finally obtain the non-dominated solution set, such as Figure 3 shown.
[0075] (4.2) According to the three optimization objectives (i.e., fitness function) proposed in step 3, set the parameters of the selection operator, crossover operator, and mutation operator, and select the data source.
[0076] (4.3) Run the operator and output the results; merge the streamlined test cases into a minimized test suite.
[0077] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0078] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0079] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0080] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A method for generating and optimizing RESTful API test cases in cyber-physical systems, characterized in that: The steps include: (1) Analyze the mutual call relationship between the RESTful APIs in the tested cyber-physical system according to the logical order of function implementation, and draw a call relationship diagram; (2) Design test cases based on the call relationship diagram and the properties of each RESTful API; (3) Define the optimization goals of the test suite for RESTful APIs in cyber-physical systems and determine the parameter settings for the optimization goals; (4) Using the NSGA-II algorithm to minimize the test suite of RESTful APIs in cyber-physical systems according to the optimization objectives; The step (1) comprises the following steps: (11) Determine the priority level of each RESTful API according to the characteristics of the RESTful API in the tested cyber-physical system; when determining the priority level of each RESTful API, it is necessary to divide each RESTful API into the following levels according to its function: define the basic control function API as the Control layer; define the complex manipulation function API as the Maneuvering layer; define the logical strategy function API as the Strategic layer; (12) Manually analyze the calling sequence relationship between various RESTful APIs; (13) Draw a call graph based on the call relationship; The step (3) comprises the following steps: (31) When testing the RESTful API in the cyber-physical system, when the test results are inconsistent with the expected results, faults will be detected from the test case results; in the face of these faults, the cause of each fault is found, its impact on the operation of the cyber-physical system is analyzed, and then these faults are classified to determine the severity of the fault; (32) The average fault detection percentage is an important parameter used to judge the fault detection capability of the test case. The average fault detection percentage is selected as one of the optimization objectives; (33) The test case execution time can be used to evaluate the quality of a test case, and the test case execution time is selected as one of the optimization objectives; The step (4) comprises the following steps: (41) Initialize the population and construct a non-dominated solution set based on the dominance relationship: The purpose of classifying and sorting the Pop group is to divide it into several non-intersecting subgroups, and the basis for classifying and sorting individuals is the Pareto dominance relationship; Assume that the size of the group Pop is N, and classify and sort the group Pop into m subsets P1, P2, ..., P m , let two vectors {n p } and {s p }where p∈Pop,n p Record the number of dominant individuals p, s p Record the set of individuals dominated by individual p, that is: n p =|{q|q>p p,q∈Pop}| s p =|{q|p>q p,q∈Pop}| Calculate n for each individual through a double loop p and p , follow the method P k = {all individuals q|n q -k+1=0} to find P2 and P3; The construction process of non-dominated sets is divided into two parts. The first part is used to calculate n i and i , and find P1, the time required is (rN 2 ), r is the target number, N is the size of the evolutionary population; the second part is used to find P2, P3...P m In the worst case, an evolving population of size N has N layers of Pareto Front sets, and the time complexity is O(N 2 ); For each individual i in the population, there are two parameters n i and i ,n i is the number of individuals in the population that dominate the solution of individual i, s i is the set of individuals whose solutions are dominated by individual i; Find all n in the population i = 0, store them in the current non-dominated set rank1; for each individual j in the current non-dominated set rank1, traverse the individual set s dominated by it j , the set s j For each individual t in n t are all subtracted by 1, that is, the number of individuals that dominate the solution of individual t is reduced by 1; if n t -1=0, then store individual t in another set H; take rank1 as the set of first-level non-dominated individuals; rank1 only dominates individuals, and is not dominated by any other individuals, and assign the same non-dominated order to all individuals in the set, and then continue to perform the above-mentioned grading operation on set H, and also assign the corresponding non-dominated order, until all individuals are graded, that is, they are all assigned the corresponding non-dominated order, and finally obtain the non-dominated solution set; (42) According to the three optimization objectives, namely, the fitness function, set the parameters of the selection operator, the crossover operator, and the mutation operator; and select the data source; (43) Run the operator, output the results, and merge the streamlined test cases into a minimized test suite.
2. The method for generating and optimizing RESTful API test cases in an information-physical system according to claim 1 is characterized in that: The step (2) comprises the following steps: (21) When designing test cases, the API value range must be considered and the test input values must be determined based on the provided API design instructions; (22) When designing test cases, call the required APIs in sequence according to the call relationship diagram in step (1).
3. The method for generating and optimizing RESTful API test cases in an information-physical system according to claim 1, characterized in that: The faults in step (31) are classified into four levels, namely fatal fault, serious fault, general fault and minor fault.