Software test case generation method and related device

CN119053952APending Publication Date: 2024-11-29HUAWEI TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380013499.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-02-27
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

When the existing technology generates software test cases, manual construction is inefficient and difficult to cover complex models and boundary conditions, resulting in insufficient testing and difficulty in meeting vehicle-level functional safety certification requirements. There is a path explosion problem in complex scenarios based on symbolic execution technology. , cannot be applied to nonlinear or high-order logic scenarios.

Method used

By generating an intermediate representation and combining symbolic execution and fuzz testing technology, the initial test seed is automatically generated, and the target test seed is obtained through seed mutation processing until the coverage requirements are met, and test cases that meet the coverage requirements are generated.

Benefits of technology

Generating test cases that meet coverage requirements in complex scenarios improves test efficiency and coverage, can meet the requirements of vehicle-level functional safety certification, and is suitable for complex and non-linear logic scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119053952A_ABST
    Figure CN119053952A_ABST
Patent Text Reader

Abstract

The invention discloses a software test case generation method and related device.The method comprises the steps that according to functional components in software and the logic relation between the functional components, intermediate representation of the software is obtained, and the intermediate representation comprises software information and code information bound with the software information; performing symbolic execution processing on code information included in the intermediate representation to obtain an initial test seed of the software; under the condition that the coverage rate corresponding to the initial test seeds does not meet the coverage rate requirement, target test seeds are obtained based on the initial test seeds, and the coverage rate corresponding to the target test seeds is higher than the coverage rate of the initial test seeds; according to the coverage rate corresponding to the target test seed, seed variation analysis processing is conducted on the target test seed, a test case is obtained, and the coverage rate corresponding to the test case meets the coverage rate requirement. Through the method, the test case meeting the coverage rate requirement can be generated in a complex scene.
Need to check novelty before this filing date? Find Prior Art

Description

A software test case generation method and related device Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a software test case generation method and related devices. Background Art

[0002] For software with high safety requirements, such as vehicle control behavior models and drone behavior models, a model-based software development approach (MBD) is generally adopted. The core of MBD is behavioral modeling, which involves expressing software logic through modeling. Simultaneously, simulation, testing, analysis, and verification are performed based on the model to ensure the correctness of the model logic. Then, based on the verified model, code is automatically generated to produce the software code.

[0003] In the MDB process, model testing and simulation are important means of model function verification and reliability verification. Developers generally manually construct some test cases based on the model logic for model verification and coverage analysis. The main drawbacks of manually constructed test cases are: 1) Due to the low efficiency of manually constructed model test cases, especially for large and logically complex models, the workload of manually constructing test cases is relatively large; 2) Manually constructed test cases are difficult to cover the entire model and complex boundary conditions, resulting in insufficient testing and problems left behind in the backend; 3) Manually constructed test cases are difficult to achieve high coverage testing for complex models and are difficult to meet automotive-grade functional safety certification requirements.

[0004] To address these issues caused by manually constructed test cases, those skilled in the art have proposed a test case generation method based on symbolic execution. Automatically generating model test cases based on symbolic execution is a static analysis technique. Because symbolic execution analyzes program execution paths, it can cause path explosion in complex scenarios for vehicle control software. Therefore, symbolic execution is only suitable for linear logic expression scenarios and is inappropriate for nonlinear vehicle control software (such as exponentiation operations) or high-level, complex logic modeling scenarios.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a software test case generation method and related devices, which can generate test cases that meet coverage requirements in complex scenarios.

[0007] In a first aspect, the present application provides a method for generating software test cases, the method comprising: obtaining an intermediate representation of the software according to functional components in the software and logical relationships between functional components, the intermediate representation comprising software information and code information bound to the software information; performing symbolic execution processing on the code information included in the intermediate representation to obtain an initial test seed for the software; obtaining a target test seed based on the initial test seed when the coverage corresponding to the initial test seed does not meet the coverage requirement, wherein the coverage corresponding to the target test seed is higher than the coverage of the initial test seed; performing seed mutation on the target test seed according to the coverage corresponding to the target test seed. Analyze and process to obtain test cases, the coverage corresponding to the test cases meeting the coverage requirements; wherein the coverage includes one or more of software coverage, branch coverage, or revised condition / determination MC / DC coverage. The coverage is determined after the test seed is input into the code information included in the intermediate representation and the test is run. The software coverage is the ratio of the number of software modules executed during the test run to the total number of software modules. The branch coverage is the ratio of the number of program branches executed during the test run to the total number of program branches. The MC / DC coverage is the ratio of the number of branches executed during the test run that meet the MC / DC standard to the total number of program branches. Through this method, test cases that meet coverage requirements can be generated in complex scenarios.

[0008] In combination with the first aspect, in a possible implementation method, the intermediate representation of the software is obtained based on the functional components in the software and the logical relationships between the functional components, including: determining the component information and connection information of the functional components and the control flow information of the software based on the functional components in the software and the logical relationships between the functional components; constructing a computing node of the intermediate representation based on the component information, and the computing node is used to indicate the behavioral characteristics of the functional components; storing the connection information in the port attributes of the computing node, and the computing node performs data interaction through the port; storing the computing node in the control flow node based on the control flow information, and the control flow node is used to indicate the behavior of the functional component; inserting equivalent code information of the functional component in the computing node; and obtaining the intermediate representation in combination with the semantic node and the control flow node.

[0009] In conjunction with the first aspect, in one possible implementation, based on the coverage corresponding to the target test seed, a seed mutation analysis process is performed on the target test seed to obtain a test case, including: if the coverage corresponding to the target test seed meets the coverage requirement, determining that the test case includes the initial test seed and the target test seed; if the coverage corresponding to the target test seed does not meet the coverage requirement, adding the target test seed to the test case, and performing mutation processing based on the target test seed to obtain an updated target test seed; based on the coverage corresponding to the updated target test seed, performing the seed mutation analysis process on the updated target test seed. In this way, a test case that meets the coverage requirement can be generated.

[0010] With reference to the first aspect, in a possible implementation, obtaining the target test seed based on the initial test seed includes: performing mutation processing based on the initial test seed to obtain the target test seed.

[0011] In combination with the first aspect, in a possible implementation, the initial test seed is time series data; the mutation processing based on the initial test seed includes: inputting the initial test seed into the intermediate representation and performing a test run to obtain the value of the operand corresponding to the initial test seed, the value of the operand being obtained by inserting a stub in a branch of the code information included in the intermediate representation; performing mutation on the data at each time series position of the initial test seed in turn; inputting the mutated test seed into the intermediate representation and performing a test run to obtain the value of the operand corresponding to the mutated test seed; if the mutation of the data at the first time series position causes the value of the operand to change, then continuing to perform mutation on the data at the first time series position, the first time series position being one or more of the various time series positions of the initial test seed.

[0012] In conjunction with the first aspect, in one possible implementation, if the data mutation at the first time sequence position causes the difference between the operand value and the target comparison value to decrease, the previous mutation strategy is maintained and the change in the previous mutation step size is strengthened. If the data mutation at the first time sequence position causes the difference between the operand value and the target comparison value to increase, the previous mutation is rolled back and the mutation step size is reduced. In this way, the mutation method can be guided by the impact of the mutation.

[0013] In conjunction with the first aspect, in one possible implementation, the initial test seed is time series data; and performing mutation processing based on the initial test seed includes setting data at some time series positions in the time series data to the same value. In this way, test cases can be generated that are closer to the actual signals of the vehicle control behavior model.

[0014] In conjunction with the first aspect, in one possible implementation, performing mutation processing based on the initial test seed includes: randomly generating at least one sine function and at least one cosine function, wherein the sine function and the cosine function have random offsets; forming a curve signal based on the at least one sine function and the at least one cosine function; and replacing part of the signal in the initial test seed with part of the signal in the curve signal. In this way, test cases that are closer to the actual signals of the vehicle control behavior model can be generated.

[0015] In combination with the first aspect, in a possible implementation, the intermediate representation is structured data.

[0016] In the second aspect, the present application provides a test case generator, which includes a software processing unit and a test case generation unit, wherein: the software processing unit is used to obtain an intermediate representation of the software based on functional components in the software and the logical relationship between functional components, and the intermediate representation includes software information and code information bound to the software information; the test case generation unit is used to perform the following operations: perform symbolic execution processing on the code information included in the intermediate representation to obtain an initial test seed for the software; if the coverage rate corresponding to the initial test seed does not meet the coverage rate requirement, obtain a target test seed based on the initial test seed, wherein the coverage rate corresponding to the target test seed is higher than the coverage rate of the initial test seed; according to the target The coverage rate corresponding to the test seed is performed, and the target test seed is subjected to seed mutation analysis to obtain a test case, and the coverage rate corresponding to the test case meets the coverage rate requirement; wherein, the coverage rate includes one or more of software coverage rate, branch coverage rate or revision condition / determination MC / DC coverage rate, and the coverage rate is determined after the test seed is input into the code information included in the intermediate representation and the test is run; the software coverage rate is the ratio of the number of software modules executed during the test run to the total number of software modules; the branch coverage rate is the ratio of the number of program branches executed during the test run to the total number of program branches; the MC / DC coverage rate is the ratio of the number of branches executed during the test run that meet the MC / DC standard to the total number of program branches.

[0017] In combination with the second aspect, in a possible implementation, the software processing unit is specifically used to: determine the component information and connection information of the functional components and the control flow information of the software based on the functional components in the software and the logical relationship between the functional components; construct a computing node of the intermediate representation based on the component information, and the computing node is used to indicate the behavioral characteristics of the functional components; store the connection information in the port attributes of the computing node, and the computing node performs data interaction through the port; based on the control flow information, store the computing node in the control flow node, and the control flow node is used to indicate the behavior of the functional component; insert equivalent code information of the functional component into the computing node; and obtain the intermediate representation in combination with the semantic node and the control flow node.

[0018] In combination with the second aspect, in a possible implementation method, the test case generation unit is specifically used to: if the coverage corresponding to the target test seed meets the coverage requirement, determine that the test case includes the initial test seed and the target test seed; if the coverage corresponding to the target test seed does not meet the coverage requirement, add the target test seed to the test case, and perform mutation processing based on the target test seed to obtain an updated target test seed; according to the coverage corresponding to the updated target test seed, perform the seed mutation analysis processing operation on the updated target test seed.

[0019] In combination with the second aspect, in a possible implementation, when the test case generation unit is used to obtain the target test seed based on the initial test seed, it specifically includes: performing mutation processing based on the initial test seed to obtain the target test seed.

[0020] In combination with the second aspect, in a possible implementation, the initial test seed is time series data; when the test case generation unit is used to perform mutation processing based on the initial test seed, it specifically includes: inputting the initial test seed into the intermediate representation and performing a test run to obtain the value of the operand corresponding to the initial test seed, and the value of the operand is obtained by inserting a stub in the branch of the code information included in the intermediate representation; performing mutation on the data at each time series position of the initial test seed in turn; inputting the mutated test seed into the intermediate representation and performing a test run to obtain the value of the operand corresponding to the mutated test seed; if the mutation of the data at the first time series position causes the value of the operand to change, then continuing to perform mutation on the data at the first time series position, and the first time series position is one or more of the various time series positions of the initial test seed.

[0021] In combination with the second aspect, in one possible implementation, if the mutation of the data at the first time series position causes the difference between the value of the operand and the target comparison value to become smaller, the previous mutation strategy is maintained unchanged, and the change of the previous mutation step is strengthened; if the mutation of the data at the first time series position causes the difference between the value of the operand and the target comparison value to become larger, the previous mutation is rolled back, and the mutation step is reduced.

[0022] In combination with the second aspect, in a possible implementation, the initial test seed is time series data; when the test case generation unit is used to perform mutation processing based on the initial test seed, it specifically includes: setting the data at some time series positions in the time series data to the same value.

[0023] In combination with the second aspect, in a possible implementation method, the test case generation unit is used to perform mutation processing based on the initial test seed, specifically including: randomly generating at least one sine function and at least one cosine function, wherein the sine function and the cosine function have random offsets; forming a curve signal based on the at least one sine function and the at least one cosine function; and replacing part of the signal in the initial test seed with part of the signal in the curve signal.

[0024] In conjunction with the second aspect, in a possible implementation, the intermediate representation is structured data.

[0025] In a third aspect, the present application provides a computer-readable storage medium, which stores a computer program, and the computer program includes program instructions. When the program instructions are executed, the method described in the first aspect or any possible implementation of the first aspect is executed.

[0026] In a fourth aspect, the present application provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are run on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art.

[0028] FIG1 is a schematic diagram of an application scenario of software testing provided by an embodiment of the present application;

[0029] FIG2 is a flow chart of a method for generating software test cases according to an embodiment of the present application;

[0030] FIG3 is a schematic diagram of an intermediate representation provided by an embodiment of the present application;

[0031] FIG4 is a flow chart of a method for generating an intermediate representation of software provided in an embodiment of the present application;

[0032] FIG5 is a flow chart of a method for performing mutation processing provided in an embodiment of the present application;

[0033] FIG6 is a schematic diagram of a mutation process provided in an embodiment of the present application;

[0034] FIG7 is a data interaction diagram of a test case generation process provided by an embodiment of the present application;

[0035] FIG8 is a schematic diagram of an extreme signal input scenario provided by an embodiment of the present application;

[0036] FIG9 is a schematic diagram of an application scenario of a division by zero test provided in an embodiment of the present application;

[0037] FIG10 is a schematic diagram of an application scenario of a violation condition test provided in an embodiment of the present application;

[0038] FIG11 is a schematic diagram of a result of random tiling mutation provided by an embodiment of the present application;

[0039] FIG12 is a schematic diagram of a result of a curve signal mutation provided by an embodiment of the present application;

[0040] FIG13 is a schematic diagram of the structure of a test case generator provided in an embodiment of the present application;

[0041] FIG14 is a schematic diagram of the structure of another test case generator provided in an embodiment of the present application;

[0042] FIG15 is a schematic structural diagram of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application are described in more detail below.

[0044] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of this application, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include plural expressions unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in this application refers to and includes any or all possible combinations of one or more listed items. The term "plurality" used in this application refers to two or more.

[0045] It should be noted that the terms "first", "second", "third", etc. in the specification and claims of the present application and in the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the term "comprises" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0046] In order to better understand the embodiments of the present application, some concepts involved in the embodiments of the present application are first introduced.

[0047] 1. Software Behavior Modeling

[0048] Software behavioral modeling is a method for expressing software behavioral logic using componentized models based on the software's specific logic. In this method, the software logic is implemented based on the model (rather than traditional code implementation). Generally, the software behavior is modeled within a modeling tool, and execution simulation can be performed directly at the model level based on the behavioral model.

[0049] 2. Functional Components

[0050] A functional component is a unit with data transmission and processing capabilities. A software behavioral model consists of multiple functional components, which are connected via ports for data transfer. Multiple functional components can execute concurrently and transfer data through ports.

[0051] 3. Symbolic Execution

[0052] Symbolic execution is a program analysis technique that analyzes a program to determine the inputs that trigger the execution of specific code regions. When analyzing a program using symbolic execution, symbolic values ​​are used as inputs, rather than the concrete values ​​typically used when executing the program. When the target code is reached, the analyzer obtains the corresponding path constraints, which are then used by a constraint solver to determine the concrete values ​​that trigger the target code.

[0053] Since the automatic generation of model test cases based on symbolic execution technology is a static analysis technology, and since symbolic execution technology analyzes according to the program execution path, it will cause path explosion problems for some complex software scenarios. Therefore, symbolic execution technology is only applicable to linear logic expression scenarios, and is not applicable to some software nonlinearities (such as exponentiation operations, etc.) or high-order complex logic modeling scenarios.

[0054] 4. Constraint Solving

[0055] Constraint solving is a method for solving constraint satisfaction problems. The principle is to find a solution to a problem based on constraints that satisfies all constraints and is within its domain. If such a solution exists, the problem is said to be satisfiable; otherwise, it is said to be unsatisfiable.

[0056] 5. Constraint Solver

[0057] A constraint solver is a functional program that can solve constraints. Based on the constraint solving rules, the constraint solver can solve the constraints of the input program and output the solution results.

[0058] 6. Software coverage

[0059] Software coverage is the ratio of the number of software modules executed by the program to the total number of software modules after software testing. Software coverage in this application can be understood as model coverage, that is, the ratio of the number of basic component blocks of the model executed by the program to the total number of basic blocks of the model after software testing.

[0060] 7. Branch Coverage

[0061] After the software test is run, branch paths appear wherever there are decision branches. The ratio of the branches traversed by the entire program execution to the total number of branches is the branch coverage rate. In other words, branch coverage is the ratio of the number of program branches executed during the test run to the total number of program branches.

[0062] 8. Modified condition / decision coverage (MC / DC)

[0063] MC / DC coverage is a special type of branch coverage, which is the ratio of the number of branches that meet the MC / DC standard executed during the test run to the total number of program branches. The MC / DC standard requires that each input and output appear at least once in a program, each condition in each judgment must produce all output results at least once, each judgment must produce all possible output results at least once, and each condition in each judgment must be able to independently affect the result of a judgment. A condition can independently affect the result of a judgment in the following ways: (1) changing the condition while keeping other conditions unchanged; (2) changing the condition while keeping other conditions that affect the result unchanged.

[0064] 9. Fuzz Testing

[0065] Fuzz testing is a software testing technique. Its core concept is to input automatically or semi-automatically generated random data into a program and monitor program anomalies, such as crashes, to identify possible program errors, such as memory leaks. Fuzz testing is often used to detect security vulnerabilities in software or computer systems.

[0066] 10. Seed mutation

[0067] Seed mutation is a method of generating test data in fuzz testing. The test data is the seed. To ensure that the test data better covers boundaries or abnormal scenarios, the test data needs to be randomized or semi-randomized based on the previous data. This process is called seed mutation.

[0068] 11. Program Instrumentation

[0069] Program instrumentation is a method of achieving testing objectives by inserting operations into the program under test. The basic principle of program instrumentation is to insert probes at appropriate locations in the program without disrupting the original logical integrity of the program under test. These probes are essentially code segments that collect information, such as assignment statements or function calls that collect coverage information. The probes execute and output program performance characteristics.

[0070] Please refer to Figure 1, which is a schematic diagram of an application scenario of software testing provided by an embodiment of the present application. In the software testing process shown in Figure 1, the test case generator provided by an embodiment of the present application is first used to generate test cases corresponding to the software, and then the software verification and coverage analysis are performed based on the test cases.

[0071] The test case generator includes a software processing unit and a test case generation unit, wherein:

[0072] The software processing unit can generate an intermediate representation corresponding to the software based on the input modeled software information. In one possible implementation, the modeled software information is a behavioral model of the software, and the intermediate representation can be obtained based on the behavioral model of the software. The behavioral modeling of the software can be completed in a modeling tool corresponding to the software, and the behavioral model corresponding to the software can be output based on the modeling tool. After that, the software behavioral model is input into the software processing unit (or called a "model interpreter"), and the software processing unit can obtain the intermediate representation of the software based on the functional components in the software behavioral model and the logical relationships between the functional components.

[0073] The test case generation module is used to generate test cases corresponding to the software based on the intermediate representation using a solution that combines symbolic execution and fuzz testing technology. The intermediate representation obtained by the software processing unit serves as the input of the test case generation module. In an embodiment of the present application, the test case generation module first generates an initial test seed based on symbolic execution. If the coverage corresponding to the initial test seed does not meet the coverage requirement, the fuzz testing technology is used to search for mutations based on the initial test seed to generate a target test seed with higher coverage, and finally a test case that meets the coverage requirement can be obtained.

[0074] Since the software behavior model is based on a semi-formal representation (expressed as a graphical logical model), before test case generation, the software behavior model needs to be converted and interpreted into a labeled representation (i.e., an intermediate representation) that can be used by the subsequent test case generation module for symbolic execution and fuzzy search, thereby facilitating the test case generation module to automatically generate the initial test seeds for the software.

[0075] After the test case generator generates the test cases corresponding to the software, the data feedback of the generated test cases is injected into the software to test and verify the software, obtain the test results, and perform coverage statistics and analysis based on the test results.

[0076] Please refer to Figure 2, which is a flow chart of a software test case generation method provided in an embodiment of the present application. The method can be applied to the application scenario shown in Figure 1, and the method includes the following steps.

[0077] Step 201: Obtain an intermediate representation of the software based on the functional components in the software and the logical relationships between the functional components.

[0078] Optionally, step 201 may be performed by a software processing unit in a test case generator.

[0079] The intermediate representation is structured data that includes modeled software information and code information bound to the software information. The intermediate representation combines common features of high-level programming languages ​​with various properties of software behavior models. For example, the intermediate representation can be represented using a tree-like node representation similar to the extensible markup language (XML). The component information and code segment description information (describing the logical relationships between functional components) of each functional component can be represented as a calculation node (Calculate). Each calculation node can include a port node and an expression node (Expression). The port node can be used for data exchange, and the expression node can be used to describe the execution logic of the code segment corresponding to the calculation component. Optionally, the intermediate representation can also include code-related nodes, such as file nodes (File), header nodes (Header), data structure nodes (Structure), function nodes (Function), reference nodes (Ref), and schedule nodes (Schedule).

[0080] For example, see Figure 3, which is a schematic diagram of an intermediate representation provided by an embodiment of the present application. As shown in Figure 3, the intermediate representation contains software information and code information bound to the software information. The intermediate representation includes a file node File, a header information node Header, a data structure node Structure, a function node Function, a reference node Ref, a scheduling node Schedule, and multiple calculation nodes Calculate. Among them, the file node File may include a header information node Header, a data structure node Structure, and a function node Function, and the function node Function may include a reference node Ref and a scheduling node Schedule, and the scheduling node Schedule may include the multiple calculation nodes Calculate. Each of the multiple calculation nodes Calculate may include a reference node Ref, an input port node (Input), an output port node (Output), and an expression node Expression. Figure 3 only takes the first calculation node Calculate as an example to illustrate the nodes included in the calculation node Calculate.

[0081] Please refer to Figure 4, which is a flowchart of a method for generating an intermediate representation of software provided in an embodiment of the present application. The method includes the following steps.

[0082] 01. Based on the functional components in the software and the logical relationships between them, determine the component information and connection information of the functional components, as well as the control flow information of the software. The functional components of the software can be understood as basic component blocks in the software behavior model or composite component blocks composed of basic component blocks. For example, the "functional components of the software" can be seen in the "Scale functional component" and "AddSubtract functional component" in Figure 3, where Scale is a function used to standardize data and AddSubtract is an addition and subtraction function.

[0083] 02. Construct a calculation node of the intermediate representation based on the component information. The calculation node is used to indicate the behavioral characteristics of the functional component.

[0084] 03. Store the connection information in the port attributes of the computing node, and the computing node exchanges data through the port.

[0085] By storing the connection information in the port attributes of the computing nodes, the data flow information of the behavioral model can be expressed.

[0086] 04. Based on the control flow information, the computing nodes are stored in the control flow nodes, which are used to indicate the behavior of the functional components.

[0087] By storing the computation nodes in the control flow nodes, the control flow information of the behavior model can be expressed.

[0088] 05. Insert equivalent code information of functional components into computing nodes.

[0089] The equivalent code information can express code semantic information.

[0090] 06. Combine semantic nodes and control flow nodes to obtain intermediate representation.

[0091] For example, semantic nodes such as header file references express code reference information.

[0092] In the intermediate representation, reference nodes are used to trace back to model components, and computation nodes are used to represent component behavior. Computation nodes exchange data through ports, and component behavior is defined through control flow nodes. This enables the binding of modeled software information and code information, and clearly expresses the relationship between components and code.

[0093] Step 202: Perform symbolic execution processing on the code information included in the intermediate representation to obtain an initial test seed of the software.

[0094] This step uses symbolic execution technology to perform a logical solution on the code information. Typically, a constraint solver from the industry is used. Examples include the Z3 solver, the open-source bounded model checker for C and C++ programs (CBMC), and others. These solvers analyze the program's logical branches and boundary conditions and directly output seed data covering the relevant branches and boundaries. This seed data serves as the initial test seed. It should be noted that the initial test seeds generated using symbolic execution technology are specific to linear logic expression scenarios.

[0095] In one possible implementation, the steps of symbolic execution may include:

[0096] 1) Traverse the control logic in the code information and convert the control logic into a constraint expression (usually a Boolean expression). For example, a=b+1;if(a<10)… will be converted into the Boolean expression b+1<10.

[0097] 2) Constraint solvers are performed based on all constraint expressions in the code information, outputting an initial test seed. For example, the constraint expression b+1<10 has many values ​​that make it positive and negative. Randomly select two values ​​that just make it positive and negative, such as b=5 and b=11, which respectively make this Boolean expression positive and negative, and use them as the solution result.

[0098] For the initial test seed obtained, the coverage rate corresponding to the initial test seed will be calculated. The coverage rate includes one or more of model coverage, branch coverage, or MC / DC coverage. The coverage rate is calculated based on the coverage situation after the test seed is input into the code information included in the intermediate representation and the test is run. The model coverage rate is the ratio of the number of model basic component blocks executed during the test run to the total number of model basic blocks. The branch coverage rate is the ratio of the number of program branches executed during the test run to the total number of program branches. The MC / DC coverage rate is the ratio of the number of branches executed during the test run that meet the MC / DC standard to the total number of program branches.

[0099] Afterwards, the coverage corresponding to the initial test seed is compared with the coverage requirement. If the coverage requirement is met, step 203 is executed. If the coverage requirement is not met, step 204 is executed. The coverage requirement can be set according to the test requirements. For example, the coverage requirement can be set to require that each of the model coverage, branch coverage, and MC / DC coverage reaches its corresponding ratio threshold. For another example, the coverage requirement can be set to require that at least one of the model coverage, branch coverage, and MC / DC coverage reaches its corresponding ratio threshold, and so on.

[0100] Step 203: When the coverage rate corresponding to the initial test seed meets the coverage rate requirement, the initial test seed is used as a test case.

[0101] After step 203 is executed, the process ends.

[0102] Step 204: When the coverage corresponding to the initial test seed does not meet the coverage requirement, a target test seed is obtained based on the initial test seed.

[0103] The target test seed is obtained from the initial test seed by using fuzz testing techniques to perform seed mutation processing on the initial test seed to obtain the target test seed. The test seed generated after seed mutation can target complex nonlinear scenarios, such as complex nonlinear calculation scenarios related to vehicle control software.

[0104] This step mutates the seed based on the initial test seed generated in step 202 to obtain a mutated test seed. The mutated test seed is input into the intermediate representation and then tested. Based on the program branches and software module execution during the test run, the coverage rate corresponding to the mutated test seed is determined. If the coverage rate corresponding to the mutated test seed is higher than the coverage rate of the initial test seed, the mutated test seed is used as the target test seed. If the coverage rate corresponding to the mutated test seed is less than or equal to the coverage rate of the initial test seed, the mutated test seed is discarded. In other words, the coverage rate corresponding to the target test seed is higher than the coverage rate of the initial test seed.

[0105] In a possible implementation, both the initial test seed and the target test seed are added to the seed pool.

[0106] Step 205: Perform seed mutation analysis on the target test seed according to the coverage rate corresponding to the target test seed to obtain a test case, and the coverage rate corresponding to the test case meets the coverage rate requirement.

[0107] In this step, if the coverage rate corresponding to the target test seed meets the coverage rate requirement, it is determined that the test case includes the initial test seed and the target test seed. If the coverage rate corresponding to the target test seed does not meet the coverage rate requirement, the target test seed is added to the test case, and mutation processing is performed based on the target test seed to obtain an updated target test seed; according to the coverage rate corresponding to the updated target test seed, the seed mutation analysis processing operation is performed on the updated target test seed. In other words, if the coverage rate corresponding to the target test seed meets the coverage rate requirement, the initial test seed and the target test seed are determined as test cases. If the coverage rate corresponding to the target test seed does not meet the coverage rate requirement, seed mutation is continued to be performed based on the target test seed, and the seed in the mutated test seed that can improve the coverage rate is used as the updated target test seed, and it is determined again whether the coverage rate corresponding to the updated target test seed meets the coverage rate requirement, and the seed mutation operation is repeated until the coverage rate corresponding to the final target test seed meets the coverage rate requirement.

[0108] It should be noted that the target test seeds obtained during each seed mutation process will be added to the test case. It can also be understood that the initial seed pool contains the initial test seeds, and during each seed mutation process, the target test seeds obtained will be added to the seed pool. The test case in the embodiment of this application is the final seed pool.

[0109] In the method embodiment shown in FIG2 , an intermediate representation corresponding to the software (including modeled software information and code information bound to the software information) is first obtained. Then, with the intermediate representation of the model as input, a solution combining symbolic execution and fuzz testing technology is used to automatically generate test cases. In the process of generating test cases, symbolic execution is first used to generate initial test seeds for linear scenarios that meet most use cases. Then, based on these initial test seeds, fuzz testing technology is used to search for mutations to generate target test seeds with higher coverage for nonlinear and complex scenarios, and finally test cases that meet coverage requirements are generated. The method of FIG2 not only meets the requirements of quickly generating targeted high-coverage test cases in linear scenarios, but also meets the coverage requirements of test cases in complex scenarios.

[0110] The following introduces some seed mutation methods involved in step 204.

[0111] This step uses the initial test seed as the base data for seed mutation. Initially, the initial test seed is added to the seed pool. A seed is randomly mutated from the seed pool to obtain the randomly mutated test seed. The program execution of the two seeds before and after the mutation is compared. Time series data impact analysis techniques are used to determine the impact relationship between data at different time series positions in the seed and software branches. Based on this impact relationship, guided mutation is performed. At the same time, unguided mutation is also performed on the seed to further increase seed diversity, improve test case coverage, and enhance defect detection capabilities.

[0112] In a possible implementation, when the initial test seed is time series data, a method for performing mutation processing based on the initial test seed can be referred to the flowchart shown in FIG5 . The method includes the following steps.

[0113] Step 501: Input the initial test seed into the intermediate representation and then perform a test run to obtain the value of the operand corresponding to the initial test seed.

[0114] The value of the operand is obtained by inserting a stub in the branch of the code information included in the intermediate representation. The value of the operand of each branch of the program can be obtained by inserting a stub. The operand is the entity on which the operator acts and is a component of the expression. It specifies the amount of digital operations performed in the program. For example, the operand can be a conditional operand, which is an operand in the conditional judgment, usually referring to a variable in the code. For example, a branch L is as follows:

[0115] In this branch L, a is the conditional operand of the branch. Through the instrumentation, the value of a can be obtained. Specifically, in this branch L, a is compared with "10", and "10" is the target comparison value.

[0116] Step 502: Mutate the data at each time sequence position of the initial test seed in sequence.

[0117] For example, an initial test seed is [2, 4, 6, 1, 3, 9], which includes data at the time sequence position from the 0th position (the corresponding data value is 2) to the 5th position (the corresponding data value is 9). First, a mutation is performed on the data from the 0th position. The mutation can be a random mutation, that is, randomly changing the value of the data. For example, the value of the data at the 0th position can be modified to 4. Then, the test seed obtained after this mutation is [4, 4, 6, 1, 3, 9].

[0118] Step 503: Input the mutated test seed into the intermediate representation and perform a test run to obtain the value of the operand corresponding to the mutated test seed.

[0119] In step 503, the operand values ​​corresponding to the mutated test seed are obtained. The values ​​of the operands corresponding to the mutated test seed are then compared to see if they have changed. If so, refer to step 504 for execution; if not, refer to step 505 for execution.

[0120] Step 504: If the mutation of the data at the first time sequence position causes the value of the operand to change, continue to perform the mutation on the data at the first time sequence position.

[0121] The first sequential position is one or more of the sequential positions of the initial test seed. That is, after determining that a change in data at a sequential position will affect the value of an operand, the mutation is continued to be performed on the data at the sequential position.

[0122] Optionally, in an embodiment of the present application, the method of mutation can be guided based on the impact of the mutation. For example, the direction and / or step size of the mutation can be guided. In one possible implementation, if the mutation of the data at the first time sequence position causes the difference between the value of the operand and the target comparison value to become smaller, the last mutation strategy is maintained unchanged, and the change in the last mutation step size is strengthened; if the mutation of the data at the first time sequence position causes the difference between the value of the operand and the target comparison value to become larger, the last mutation is rolled back, and the mutation step size is reduced. Optionally, the difference here is the absolute value of the difference.

[0123] Please refer to Figure 6, which is a schematic diagram of a mutation process provided by an embodiment of the present application. The mutation illustrated in Figure 6 is a mutation of the data at the second time sequence position starting from 0. It should be noted that the example in Figure 6 is introduced using the branch L illustrated above as an example. If the value of the data at the second time sequence position corresponds to the value of the operand a. When mutating the data at the second time sequence position, the mutation step size is initially specified as 1, and the specific process is as follows:

[0124] (1) Initially, the value of the data at the second timing position is 6. The value of a is set to 6. At this time, the branch does not meet the requirement (because 6 < 10, that is, a ≠ 10). The difference (gap) between the value of the branch operand and the target comparison value is 4.

[0125] (2) Take the first round of mutation: The mutation step size of the first round of mutation is the initial step size (i.e., 1), and the initial mutation strategy is to increase the value. After executing the first round of mutation, the value of the data at the second time sequence position changes from 6 to 7, and the value of a is set to 7. At this time, the branch is not satisfied (because 7 < 10, i.e., a ≠ 10), and the difference between the value of the branch operand and the target comparison value is 3.

[0126] (3) Take the second round of mutation: Since the first round of mutation makes the difference between the value of the branch operand and the target comparison value smaller, then continue to maintain the previous mutation strategy (that is, continue to increase the data at the second time sequence position), and strengthen the change of the last mutation step. Here is the initial change of the mutation step. The initial change of the mutation step can be set to increase. For example, the mutation step increases from 1 to 2. After executing the second round of mutation, the value of the data at the second time sequence position changes from 7 to 9, and the value of a is set to 9. At this time, the branch does not meet the requirements (because 9 < 10, that is, a ≠ 10), and the difference between the value of the branch operand and the target comparison value is 1.

[0127] (4) The third round of mutation is performed: Since the difference between the branch operand value and the target comparison value has become smaller due to the second round of mutation, the previous mutation strategy is maintained (i.e., the data at the second time sequence position is further increased), and the change in the mutation step size is strengthened, i.e., the mutation step size is further increased. For example, the mutation step size is increased from 2 to 4. After the third round of mutation, the value of the data at the second time sequence position is changed from 9 to 13, and the value of a is set to 13. At this time, the branch does not meet the requirements (since 13>10, i.e., a≠10), and the difference between the branch operand value and the target comparison value is 3.

[0128] (5) Take the fourth round of mutation: Since the third round of mutation has increased the difference between the value of the branch operand and the target comparison value, the previous mutation is rolled back (i.e., the data at the second time position is changed to the value before the third round of mutation), and the mutation step is reduced from 4 to 2. After executing the fourth round of mutation, the value of the data at the second time position is first rolled back from 13 to 9, and then increased to 11. The value of a is set to 11. At this time, the branch does not meet the requirements (since 11>10, i.e., a≠10), and the difference between the value of the branch operand and the target comparison value is 1.

[0129] (6) Take the fifth round of mutation: Since the fourth round of mutation has reduced the difference between the value of the branch operand and the target comparison value, continue to maintain the previous mutation strategy (that is, roll back the data at the second time sequence position again), and strengthen the change of the last mutation step, that is, continue to reduce the mutation step, and reduce the mutation step from 2 to 1. After executing the fifth round of mutation, the value of the data at the second time sequence position first rolls back from 11 to 9, and then increases to 10. Set the value of a to 10, and now the branch satisfies (because 10=10, that is, a=10). At this point, the mutation process of the data at the second time sequence position ends.

[0130] Step 505: If the mutation of the data at the second time sequence position does not change the value of the operand, then continue to perform mutations on the data at other time sequence positions. If there are no other time sequence positions that have not been mutated, then end this process.

[0131] The second sequential position is one or more of the sequential positions of the initial test seed. That is, after determining that a change to the data at a sequential position will not affect the operand value, the data at that sequential position is no longer mutated. Instead, the process continues to determine whether changes to data at other sequential positions will affect the operand value. If no other sequential positions have yet to be mutated, the process ends.

[0132] For example, please refer to Figure 7, which is a data interaction diagram of a test case generation process provided by an embodiment of the present application. The test case generator illustrated in Figure 7 includes a software processing unit and a test case generation unit. The functions of the software processing unit and the test case generation unit can refer to the introduction of the relevant embodiment of Figure 2 above. In the process introduced in Figure 7, the mutation method used for seed mutation can refer to the introduction of the relevant embodiment of Figure 5.

[0133] Specifically, the test case generation unit of the test case generator includes a symbolic execution unit and a fuzzy testing unit.

[0134] First, the behavioral model of the software is input into the software processing unit, and the software processing unit can process the graphical logical behavioral model of the software into a structured, analyzable and executable intermediate representation of the software.

[0135] Next, the symbolic execution unit uses symbolic execution technology to perform symbolic execution on the intermediate representation of the software, which can quickly generate an initial test seed and add the initial test seed to the seed pool, which is used to store test seeds (i.e., test cases). Next, the coverage corresponding to the initial test seed is calculated. Generally, for simple software scenarios, the initial test seed can meet the coverage requirements, so there is no need for the fuzzy testing unit to further process the initial test seed, and the initial test seed stored in the seed pool is the final test case. For complex software scenarios, such as models related to nonlinear operations or complex vehicle control models with deep loop nesting, the initial test seed cannot meet the coverage requirements, so the fuzzy testing unit is required to further process the initial test seed.

[0136] If the initial test seed fails to meet coverage requirements, the fuzz testing unit selects a seed from the seed pool and performs a mutation operation, obtaining a mutated test seed. The mutated test seed is then fed into the intermediate representation and run through the test, obtaining execution trace information and numerical data from the test run. Based on this execution trace information and numerical data, an impact analysis of the mutation is performed, which guides the next mutation operation. For example, this can guide the direction and / or step size of the mutation.

[0137] At the same time, based on the execution tracking information and numerical information, a coverage analysis is performed on the mutated test seed, that is, the coverage corresponding to the mutated seed is calculated. The coverage includes one or more of model coverage, branch coverage, or MC / DC coverage. Based on the coverage, it is determined whether to add the mutated test seed to the seed pool. Specifically, if the coverage corresponding to the mutated test seed is higher than the coverage corresponding to the initial test seed, the mutated test seed is added to the seed pool.

[0138] Repeat the mutation operation on the test seed until the test seed meets the coverage requirement or the mutation time of the test seed reaches the set time threshold. The test seeds included in the final seed pool are the test cases finally generated.

[0139] In other possible implementations, some non-directional mutation methods can also be used. Examples are as follows:

[0140] (1) Bitwise flip

[0141] Bit flipping randomly flips the bits of a portion of the test seed sequence. For example, randomly flip the subsequence from index 2 to index 4: 010101110001 => 011011110001.

[0142] (2) Arithmetic operations

[0143] Arithmetic operations are to randomly test some positions in the seed and uniformly perform arithmetic operations on them, for example:

[0144] Addition operation: +1, +2, +3, +4…, +MAX, where MAX is the predetermined maximum value;

[0145] Subtraction: -1, -2, -3, -4, ..., -MAX;

[0146] Multiplication operation: *2, *3, *4, …, *MAX;

[0147] Division operations: / 2, / 3, / 4, …, / MAX.

[0148] (3) Destruction

[0149] The purpose of corruption is to make the data as chaotic as possible. For example, for a test seed, the values ​​at different positions are randomly swapped; at the same time, some parts of the test seed are randomly set to some random values ​​or extreme values.

[0150] Through the seed mutation method described above, extreme values ​​can be injected into the software's behavioral model to discover defects in the behavioral model. For example, see the following scenario.

[0151] Please refer to Figure 8, which is a schematic diagram of an extreme signal input scenario provided by an embodiment of the present application. In the application scenario shown in Figure 5, the extreme signal is input into the software's behavioral model. This approach can cause the model to behave incorrectly, thereby detecting whether there is a relevant processing mechanism in the behavioral model.

[0152] Please refer to Figure 9, which is a schematic diagram of an application scenario of a division by zero test provided by an embodiment of the present application. Optionally, it is possible to detect whether there are arithmetic errors and logical errors in the software behavior model, such as division by zero, array out-of-bounds, and memory problems.

[0153] Please refer to Figure 10, which is a schematic diagram of an application scenario for a condition violation test provided in an embodiment of the present application. The test in Figure 8 can check whether a user-written condition has been violated. The condition violation test shown in Figure 8 is applied to a distributed master-slave system, which includes multiple machines, one of which is designated as the master. In other words, the system must meet the condition that there can only ever be one master.

[0154] The process of verifying the correctness of the system can include the following steps: (1) Modeling: Users can build a model to establish the entire system; (2) Testing: Users can randomly kill the host, or randomly turn on or off certain machines, and repeat these operations; (3) Verification: Convert the constraint of "there is always only one host" into a model condition, and check that this condition remains true in various test cases.

[0155] In another possible implementation, there are some mutation operations for the behavioral model of the vehicle control software. Optional mutation operations for time series data include two mutation strategies: random tiling mutation and curve signal mutation.

[0156] 1) Random tile mutation

[0157] Random tiling mutation means setting the data at some time series positions in the time series data to the same value. For example, the three middle values ​​in the time series data can be tiled to 5. For example, the original time series data [0.1, 0.8, 1.5, 2.3, 3.3] can be mutated to [0.1, 5, 5, 5, 3.3].

[0158] Since time delay (TimeDelay) components often appear in vehicle control behavior modeling, the TimeDelay component requires the signal to remain the same for a period of time (the signal must be flattened). Traditional fuzz testing methods will generate a large amount of fluctuating data. By randomly tiling and mutating, the signal can be made closer to the actual signal, ultimately improving coverage. The TimeDelay component and its related extended components require the signal to maintain a value for a period of time. Only when this condition is met can the relevant branch be entered. Please refer to Figure 11, which is a schematic diagram of the result of a random tiling mutation provided in an embodiment of the present application.

[0159] 2) Abnormal curve signal

[0160] The behavioral models of vehicle control software sometimes use curved signals as input. For example, in vehicle model recognition applications, a curve induction coil signal is required. If the input signal is not a curve, some of the logic model functions will not be covered and executed, and high coverage will not be achieved.

[0161] The execution process of curve signal mutation can be referred to as follows:

[0162] First, at least one sine function (sin function) and at least one cosine function (cos function) are randomly generated, where the sine and cosine functions have random offsets. For example, the original sin(x) function is added with a random offset to become the sin(x+off) function, where off is the offset, which is randomly generated.

[0163] Then, a curve signal is formed based on at least one sine function and at least one cosine function. For example, all sin functions and cos functions are added together to form the curve signal.

[0164] Next, part of the curve signal is used to replace part of the initial test seed signal. For example, a segment of the curve signal is randomly intercepted and directly replaced with the original signal. See Figure 12, which is a schematic diagram of the results of a curve signal mutation provided by an embodiment of the present application.

[0165] For several typical test models, the test case generation method proposed in the embodiment of the present application and the test case generation method of the industry's Simulink Design Verifier (SLDV) are used to obtain the test case coverage results shown in Table 1.

[0166] Table 1

[0167] In Table 1, tests were conducted based on four behavioral models: NLGuidance, RHB, Euler321, and MHI1209. Based on Table 1, it can be seen that the coverage of the test cases corresponding to the embodiment of the present application is improved compared with the coverage of the test cases corresponding to the industry's SLDV, with the coverage increased by 17% to 96%. The advantages of the embodiment of the present application are quite obvious.

[0168] It is understood that in order to implement the functions in the above embodiments, the test case generator includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily appreciate that, in conjunction with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0169] Please refer to Figure 13, which is a schematic diagram of the structure of a test case generator provided in an embodiment of the present application. The test case generator 130 includes a software processing unit 1301 and a test case generation unit 1302. These two units are described in detail below.

[0170] The software processing unit 1301 is configured to obtain an intermediate representation of the software according to the functional components in the software and the logical relationships between the functional components. The intermediate representation includes software information and code information bound to the software information.

[0171] The test case generation unit 1302 is configured to perform the following operations: perform symbolic execution processing on the code information included in the intermediate representation to obtain an initial test seed for the software; if the coverage corresponding to the initial test seed does not meet the coverage requirement, obtain a target test seed based on the initial test seed, wherein the coverage corresponding to the target test seed is higher than the coverage of the initial test seed; perform seed mutation analysis on the target test seed according to the coverage corresponding to the target test seed to obtain a test case, wherein the coverage corresponding to the test case meets the coverage requirement; wherein the coverage includes one or more of software coverage, branch coverage, or revision condition / determination MC / DC coverage, and the coverage is determined after the test seed is input into the code information included in the intermediate representation and a test run is performed; the software coverage is the ratio of the number of software modules executed during the test run to the total number of software modules, the branch coverage is the ratio of the number of program branches executed during the test run to the total number of program branches, and the MC / DC coverage is the ratio of the number of branches executed during the test run that meet the MC / DC standard to the total number of program branches.

[0172] In one possible implementation, the software processing unit 1301 is specifically used to: determine the component information and connection information of the functional components and the control flow information of the software based on the functional components in the software and the logical relationship between the functional components; construct a computing node of the intermediate representation based on the component information, and the computing node is used to indicate the behavioral characteristics of the functional components; store the connection information in the port attributes of the computing node, and the computing node performs data interaction through the port; based on the control flow information, store the computing node in the control flow node, and the control flow node is used to indicate the behavior of the functional component; insert equivalent code information of the functional component into the computing node; and obtain the intermediate representation in combination with the semantic node and the control flow node.

[0173] In one possible implementation, the test case generation unit 1302 is specifically used to: if the coverage corresponding to the target test seed meets the coverage requirement, determine that the test case includes the initial test seed and the target test seed; if the coverage corresponding to the target test seed does not meet the coverage requirement, add the target test seed to the test case, and perform mutation processing based on the target test seed to obtain an updated target test seed; according to the coverage corresponding to the updated target test seed, perform the seed mutation analysis processing operation on the updated target test seed.

[0174] In a possible implementation, when the test case generating unit 1302 is used to obtain the target test seed based on the initial test seed, the method specifically includes: performing mutation processing based on the initial test seed to obtain the target test seed.

[0175] In one possible implementation, the initial test seed is time series data; the test case generation unit 1302 is used to perform mutation processing based on the initial test seed, specifically including: inputting the initial test seed into the intermediate representation and performing a test run to obtain the value of the operand corresponding to the initial test seed, the value of the operand being obtained by inserting a stub in a branch of the code information included in the intermediate representation; performing mutation on the data at each time series position of the initial test seed in turn; inputting the mutated test seed into the intermediate representation and performing a test run to obtain the value of the operand corresponding to the mutated test seed; if the mutation of the data at the first time series position causes the value of the operand to change, then continuing to perform mutation on the data at the first time series position, the first time series position being one or more of the various time series positions of the initial test seed.

[0176] In one possible implementation, if the mutation of the data at the first time sequence position causes the difference between the value of the operand and the target comparison value to decrease, the previous mutation strategy is maintained unchanged, and the change of the previous mutation step is strengthened; if the mutation of the data at the first time sequence position causes the difference between the value of the operand and the target comparison value to increase, the previous mutation is rolled back, and the mutation step is reduced.

[0177] In a possible implementation, the initial test seed is time series data; when the test case generation unit 1302 is used to perform mutation processing based on the initial test seed, it specifically includes: setting the data at some time series positions in the time series data to the same value.

[0178] In one possible implementation, the test case generation unit 1302 is used to perform mutation processing based on the initial test seed, specifically including: randomly generating at least one sine function and at least one cosine function, wherein the sine function and the cosine function have random offsets; forming a curve signal based on the at least one sine function and the at least one cosine function; and replacing part of the signal in the initial test seed with part of the signal in the curve signal.

[0179] In a possible implementation, the intermediate representation is structured data.

[0180] The operations performed by the various modules of the test case generator 130 shown in FIG13 can be referred to in the above-mentioned method embodiment regarding the test case generator, and will not be described in detail here. The above-mentioned modules can be implemented in hardware, software, or a combination of hardware and software. In one embodiment, the functions of the software processing unit 1301 and the test case generation unit 1302 described above can be implemented by one or more processors in the test case generator 130.

[0181] Please refer to Figure 14, which is a schematic diagram of the structure of another test case generator provided in an embodiment of the present application. The test case generator 140 can be used to implement the method described in the above method embodiment. For details, please refer to the description of the above method embodiment.

[0182] The test case generator 140 may include one or more processors 1401. The processor 1401 may be a general-purpose processor or a dedicated processor, etc. The processor 1401 may be used to control the test case generator 140, execute software programs, and process data of the software programs.

[0183] Optionally, the test case generator 140 may include one or more memories 1402, on which program code 1403 may be stored. The program code can be executed on the processor 1401, so that the test case generator 140 performs the method described in the above method embodiment. Optionally, data may also be stored in the memory 1402. The processor 1401 and the memory 1402 may be provided separately or integrated together. Optionally, the memory 1402 may also be located outside the test case generator 140 and coupled to the test case generator 140 in some manner.

[0184] Optionally, the test case generator 140 may further include a transceiver 1404. The transceiver 1404 may be referred to as a transceiver unit, a transceiver, or a transceiver circuit, and is configured to implement transceiver functions. The transceiver 1404 may include a receiver and a transmitter. The receiver may be referred to as a receiver or a receiving circuit, and is configured to implement a receiving function; the transmitter may be referred to as a transmitter or a transmitting circuit, and is configured to implement a transmitting function.

[0185] The processor 1401 calls the program code or instruction stored in the memory, causing the test case generator to perform the following operations:

[0186] Obtaining an intermediate representation of the software based on functional components in the software and logical relationships between the functional components, the intermediate representation including software information and code information bound to the software information;

[0187] Performing symbolic execution processing on the code information included in the intermediate representation to obtain an initial test seed of the software;

[0188] When the coverage rate corresponding to the initial test seed does not meet the coverage rate requirement, obtaining a target test seed based on the initial test seed, wherein the coverage rate corresponding to the target test seed is higher than the coverage rate of the initial test seed;

[0189] Performing seed mutation analysis on the target test seed according to the coverage rate corresponding to the target test seed to obtain a test case, wherein the coverage rate corresponding to the test case meets the coverage rate requirement;

[0190] Among them, the coverage rate includes one or more of software coverage rate, branch coverage rate or revised condition / determination MC / DC coverage rate. The coverage rate is determined after inputting the test seed into the code information included in the intermediate representation and performing a test run. The software coverage rate is the ratio of the number of software modules executed during the test run to the total number of software modules. The branch coverage rate is the ratio of the number of program branches executed during the test run to the total number of program branches. The MC / DC coverage rate is the ratio of the number of branches executed during the test run that meet the MC / DC standard to the total number of program branches.

[0191] The operations performed by the test case generator 140 may refer to the relevant contents of the test case generator in the above method embodiment, and will not be described in detail here.

[0192] The structure of the test case generator 140 may not be limited by FIG. 14 . The test case generator may be an independent device or may be part of a larger device. For example, the test case generator may be: (1) an independent integrated circuit IC, or a chip, or a chip system or subsystem; (2) a collection of one or more ICs, optionally, the IC collection may also include a storage component for storing data and program code; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, an intelligent terminal, a wireless device, a handheld device, a mobile unit, an in-vehicle device, a cloud device, an artificial intelligence device, etc.; (6) others, etc.

[0193] In the case where the test case generator can be a chip or a chip system, please refer to the chip structure diagram shown in Figure 15. The chip 150 shown in Figure 15 includes a logic circuit 1501 and an input / output interface 1502. The number of logic circuits 1501 can be one or more, and the number of input / output interfaces 1502 can be multiple.

[0194] The logic circuit 1501 is configured to perform the following operations:

[0195] Obtaining an intermediate representation of the software based on functional components in the software and logical relationships between the functional components, the intermediate representation including software information and code information bound to the software information;

[0196] Performing symbolic execution processing on the code information included in the intermediate representation to obtain an initial test seed of the software;

[0197] When the coverage rate corresponding to the initial test seed does not meet the coverage rate requirement, obtaining a target test seed based on the initial test seed, wherein the coverage rate corresponding to the target test seed is higher than the coverage rate of the initial test seed;

[0198] According to the coverage rate corresponding to the target test seed, the target test seed is subjected to seed mutation analysis to obtain a test case, wherein the coverage rate corresponding to the test case meets the coverage rate requirement;

[0199] Among them, the coverage rate includes one or more of software coverage rate, branch coverage rate or revised condition / determination MC / DC coverage rate. The coverage rate is determined after inputting the test seed into the code information included in the intermediate representation and performing a test run. The software coverage rate is the ratio of the number of software modules executed during the test run to the total number of software modules. The branch coverage rate is the ratio of the number of program branches executed during the test run to the total number of program branches. The MC / DC coverage rate is the ratio of the number of branches executed during the test run that meet the MC / DC standard to the total number of program branches.

[0200] The operations performed by the logic circuit 1501 may refer to the introduction of the test case generator in the above method embodiment.

[0201] Those skilled in the art will also appreciate that the various illustrative logical blocks and steps listed in the embodiments of the present application can be implemented by electronic hardware, computer software, or a combination of both. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present application.

[0202] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer-readable storage medium is executed by a computer, the functions of any of the above method embodiments are realized.

[0203] The present application also provides a computer program product, which implements the functions of any of the above method embodiments when executed by a computer.

[0204] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state drive (SSD)).

[0205] The predefined in this application may be understood as defined, predefined, stored, pre-stored, pre-negotiated, pre-configured, solidified, or pre-burned.

[0206] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0207] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

Claims

1. A method for generating software test cases, characterized in that: include: According to the functional components in the software and the logical relationships between the functional components, an intermediate representation of the software is obtained, wherein the intermediate representation includes software information and code information bound to the software information; Performing symbolic execution processing on the code information included in the intermediate representation to obtain an initial test seed of the software; In the case where the coverage rate corresponding to the initial test seed does not meet the coverage rate requirement, obtaining a target test seed based on the initial test seed, wherein the coverage rate corresponding to the target test seed is higher than the coverage rate of the initial test seed; According to the coverage rate corresponding to the target test seed, seed mutation analysis processing is performed on the target test seed to obtain a test case, and the coverage rate corresponding to the test case meets the coverage rate requirement; Among them, the coverage rate includes one or more of software coverage rate, branch coverage rate or revised condition / determination MC / DC coverage rate. The coverage rate is determined after the test seed is input into the code information included in the intermediate representation and the test is run. The software coverage rate is the ratio of the number of software modules executed during the test run to the total number of software modules. The branch coverage rate is the ratio of the number of program branches executed during the test run to the total number of program branches. The MC / DC coverage rate is the ratio of the number of branches that meet the MC / DC standard executed during the test run to the total number of program branches.

2. The method according to claim 1, characterized in that: The step of obtaining the intermediate representation of the software according to the functional components in the software and the logical relationships between the functional components includes: Determine component information and connection information of the functional components and control flow information of the software according to the functional components in the software and the logical relationships between the functional components; constructing a computing node of the intermediate representation based on the component information, wherein the computing node is used to indicate a behavioral feature of the functional component; The connection information is stored in the port attribute of the computing node, and the computing node performs data exchange through the port; Based on the control flow information, storing the computing node in the control flow node, the control flow node being used to indicate the behavior of the functional component; Inserting equivalent code information of the functional component into the computing node; The intermediate representation is obtained by combining the semantic node and the control flow node.

3. The method according to claim 1 or 2, characterized in that: According to the coverage rate corresponding to the target test seed, the target test seed is subjected to seed mutation analysis to obtain a test case, including: If the coverage rate corresponding to the target test seed meets the coverage rate requirement, determining that the test case includes the initial test seed and the target test seed; If the coverage rate corresponding to the target test seed does not meet the coverage rate requirement, the target test seed is added to the test case, and mutation processing is performed based on the target test seed to obtain an updated target test seed; according to the coverage rate corresponding to the updated target test seed, the seed mutation analysis processing operation is performed on the updated target test seed.

4. The method according to any one of claims 1 to 3, characterized in that: The step of obtaining a target test seed based on the initial test seed comprises: Mutation processing is performed based on the initial test seed to obtain the target test seed.

5. The method according to claim 4, characterized in that The initial test seed is time series data; The performing mutation processing based on the initial test seed comprises: Input the initial test seed into the intermediate representation and then perform a test run, and obtain a value of an operand corresponding to the initial test seed, wherein the value of the operand is obtained by inserting a stub in a branch of code information included in the intermediate representation; Mutating the data at each time sequence position of the initial test seed in sequence; Input the mutated test seed into the intermediate representation and then perform a test run to obtain the value of the operand corresponding to the mutated test seed; If the mutation of the data at the first time sequence position causes the value of the operand to change, the mutation is continued to be performed on the data at the first time sequence position, where the first time sequence position is one or more of the various time sequence positions of the initial test seed.

6. The method according to claim 5, characterized in that If the mutation of the data at the first time sequence position makes the difference between the value of the operand and the target comparison value smaller, the last mutation strategy is kept unchanged, and the change of the last mutation step length is strengthened; If the mutation of the data at the first time sequence position causes the difference between the value of the operand and the target comparison value to increase, the last mutation is rolled back and the mutation step length is reduced.

7. The method according to claim 4, characterized in that The initial test seed is time series data; The performing mutation processing based on the initial test seed comprises: The data at some time series positions in the time series data are set to the same value.

8. The method according to claim 4, characterized in that The performing mutation processing based on the initial test seed comprises: randomly generating at least one sine function and at least one cosine function, wherein the sine function and the cosine function have a random offset; forming a curve signal based on the at least one sine function and the at least one cosine function; Part of the signal in the initial test seed is replaced by part of the signal in the curve signal.

9. The method according to any one of claims 1 to 8, characterized in that: The intermediate representation is structured data.

10. A test case generator, characterized in that: The test case generator includes a software processing unit and a test case generating unit, wherein: The software processing unit is used to obtain, according to the functional components in the software and the logical relationship between the functional components An intermediate representation of the software, the intermediate representation comprising software information and code information bound to the software information; The test case generation unit is used to perform the following operations: Performing symbolic execution processing on the code information included in the intermediate representation to obtain an initial test seed of the software; In the case where the coverage rate corresponding to the initial test seed does not meet the coverage rate requirement, obtaining a target test seed based on the initial test seed, wherein the coverage rate corresponding to the target test seed is higher than the coverage rate of the initial test seed; According to the coverage rate corresponding to the target test seed, seed mutation analysis processing is performed on the target test seed to obtain a test case, and the coverage rate corresponding to the test case meets the coverage rate requirement; Among them, the coverage rate includes one or more of software coverage rate, branch coverage rate or revised condition / determination MC / DC coverage rate. The coverage rate is determined after the test seed is input into the code information included in the intermediate representation and the test is run. The software coverage rate is the ratio of the number of software modules executed during the test run to the total number of software modules. The branch coverage rate is the ratio of the number of program branches executed during the test run to the total number of program branches. The MC / DC coverage rate is the ratio of the number of branches that meet the MC / DC standard executed during the test run to the total number of program branches.

11. The test case generator according to claim 10, characterized in that: The software processing unit is specifically used for: Determine component information and connection information of the functional components and control flow information of the software according to the functional components in the software and the logical relationships between the functional components; constructing a computing node of the intermediate representation based on the component information, wherein the computing node is used to indicate a behavioral feature of the functional component; The connection information is stored in the port attribute of the computing node, and the computing node performs data exchange through the port; Based on the control flow information, storing the computing node in the control flow node, the control flow node being used to indicate the behavior of the functional component; Inserting equivalent code information of the functional component into the computing node; The intermediate representation is obtained by combining the semantic node and the control flow node.

12. The test case generator according to claim 10 or 11, characterized in that: The test case generation unit is specifically used for: If the coverage rate corresponding to the target test seed meets the coverage rate requirement, determining that the test case includes the initial test seed and the target test seed; If the coverage rate corresponding to the target test seed does not meet the coverage rate requirement, the target test seed is added to the test case, and mutation processing is performed based on the target test seed to obtain an updated target test seed; According to the coverage rate corresponding to the updated target test seed, the seed mutation analysis processing operation is performed on the updated target test seed.

13. The test case generator according to any one of claims 10 to 12, characterized in that: When the test case generation unit is used to obtain the target test seed based on the initial test seed, it specifically includes: Mutation processing is performed based on the initial test seed to obtain the target test seed.

14. The test case generator according to claim 13, characterized in that: The initial test seed is time series data; When the test case generation unit is used to perform mutation processing based on the initial test seed, it specifically includes: Input the initial test seed into the intermediate representation and then perform a test run, and obtain a value of an operand corresponding to the initial test seed, wherein the value of the operand is obtained by inserting a stub in a branch of code information included in the intermediate representation; Mutating the data at each time sequence position of the initial test seed in sequence; Input the mutated test seed into the intermediate representation and then perform a test run to obtain the value of the operand corresponding to the mutated test seed; If the mutation of the data at the first time sequence position causes the value of the operand to change, the mutation is continued to be performed on the data at the first time sequence position, where the first time sequence position is one or more of the various time sequence positions of the initial test seed.

15. The test case generator according to claim 14, characterized in that: If the mutation of the data at the first time sequence position makes the difference between the value of the operand and the target comparison value smaller, the last mutation strategy is kept unchanged, and the change of the last mutation step length is strengthened; If the mutation of the data at the first time sequence position causes the difference between the value of the operand and the target comparison value to increase, the last mutation is rolled back and the mutation step length is reduced.

16. The test case generator according to claim 13, characterized in that: The initial test seed is time series data; When the test case generation unit is used to perform mutation processing based on the initial test seed, it specifically includes: The data at some time series positions in the time series data are set to the same value.

17. The test case generator according to claim 13, characterized in that: When the test case generation unit is used to perform mutation processing based on the initial test seed, it specifically includes: randomly generating at least one sine function and at least one cosine function, wherein the sine function and the cosine function have a random offset; forming a curve signal based on the at least one sine function and the at least one cosine function; Part of the signal in the initial test seed is replaced by part of the signal in the curve signal.

18. The test case generator according to any one of claims 10 to 17, characterized in that: The intermediate representation is structured data.

19. A test case generator, characterized in that: It comprises a processor, wherein the processor is used to call the program code or instruction stored in the memory so that the test case generator executes the method as described in any one of claims 1-9.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed, the method according to any one of claims 1 to 9 is executed.