Information processing device, information processing method, and computer program

The information processing device automatically generates high-coverage test code by analyzing source code to determine type information and object initialization functions, addressing the challenge of manual input requirements and improving test effectiveness for dynamically typed languages and void pointers.

WO2024232193A1PCT designated stage expired Publication Date: 2024-11-14SONY GROUP CORP

Patent Information

Application Number
PCT/JP2024/013516
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-09
Filing Date
2024-04-01
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

In software development, generating high-coverage test code is challenging due to the need for manual input of type information and object initialization functions, especially for dynamically typed languages and void pointers, which is tedious and burdensome for developers, leading to potential null pointer dereference exceptions and meaningless test cases.

Method used

An information processing device and method that automatically generates type information and object initialization functions by analyzing source code, using static and dynamic analysis to narrow down variable types and collect method sequences, thereby generating symbolic execution drivers and test cases without requiring user input.

Benefits of technology

Automatically generates meaningful test code with high coverage that reflects the user's design intent, reducing the burden on developers and ensuring correct initialization of objects, thus preventing null pointer issues and improving test effectiveness.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024013516_14112024_PF_FP_ABST
    Figure JP2024013516_14112024_PF_FP_ABST
Patent Text Reader

Abstract

Provided is an information processing device that generates a test code having high coverage. This information processing device is equipped with: an analysis unit that analyzes a source code so as to collect type information for the source code and generate an object initialization function; a symbol execution driver generation unit that uses the type information and the object initialization function to generate a symbol execution driver; a symbol execution unit that executes the symbol execution driver generated by the symbol execution driver generation unit so as to generate a test case; and a test code generation unit that generates test code for feeding the test case generated by the symbol execution unit into a program to be tested and executing the same.
Need to check novelty before this filing date? Find Prior Art

Description

Information processing device, information processing method, and computer program

[0001] The technology disclosed in this specification (hereinafter referred to as "the present disclosure") relates to an information processing device, an information processing method, and a computer program that perform processing related to program development.

[0002] In software development, program behavior that differs from the specifications or is not expected by the developer is a defect, and all defects must be corrected and eliminated before release. Testing is performed to detect program defects and ensure operation. For example, testing is performed by creating test cases with inputs and outputs described in the specifications or expected by the developer, and checking whether the program returns the correct output for the input. If the correct output is not returned for the input, or if an exception (fault) occurs and the program does not work, it is deemed to be a defect. The developer then analyzes the program to identify the cause and correct the program logic. To ensure the high quality and reliability of the program, it is necessary to perform tests with high coverage.

[0003] JP 2018-45325 A

[0004] An object of the present disclosure is to provide an information processing device, an information processing method, and a computer program that perform processing for generating high-coverage test code.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and a first aspect thereof is an information processing device including: an analysis unit that analyzes source code to collect type information of the source code and generate an object initialization function; a symbolic execution driver generation unit that generates a symbolic execution driver using the type information and the object initialization function; a symbolic execution unit that executes the symbolic execution driver generated by the symbolic execution driver generation unit to generate test cases; and a test code generation unit that generates test code for providing the test cases generated by the symbolic execution unit to a program to be tested and executing the program.

[0006] The analysis unit performs static analysis on the program to be tested in a statically typed language, investigates the types of variables created by functions that call the function to be tested and the functions that pass them, and narrows down the types of arguments to be passed to the function to be tested.

[0007] The analysis unit also instruments the program to be tested in a dynamically typed language so that the types of variables to be passed to the function to be tested are output as an execution trace immediately before the function to be tested is called, passes input to the instrumented program, executes it, collects the execution trace, and analyzes the type information and objects of the variables passed to the function to be tested from the collected execution trace.

[0008] In addition, the analysis unit performs instrumentation to output the methods called and the order in which they were called from the time an object is created until it is deleted as an execution trace, so that the method sequence for initializing each object can be collected, and analyzes the execution trace to generate an object initialization function.

[0009] The symbolic execution driver generation unit initializes and symbolizes variables to be passed to the function under test, and generates a symbolic execution driver consisting of code that simply calls the function under test.

[0010] The test code generation unit generates test code for a function to be tested using test cases generated by executing the symbolic execution driver generated by the symbolic execution driver generation unit on a symbolic execution engine as input values.

[0011] A second aspect of the present disclosure is an information processing method having: an analysis step of analyzing source code and generating type information and an object initialization function of the source code; a symbolic execution driver generation step of generating a symbolic execution driver using the type information and the object initialization function; a symbolic execution step of executing the symbolic execution driver generated in the symbolic execution driver generation step to generate test cases; and a test code generation step of generating test code for providing the test cases generated in the symbolic execution step to a program to be tested and executing the program.

[0012] Furthermore, a third aspect of the present disclosure is a computer program written in a computer-readable format to cause a computer to function as: an analysis unit that analyzes source code and generates type information and an object initialization function for the source code; a symbolic execution driver generation unit that generates a symbolic execution driver using the type information and the object initialization function; a symbolic execution unit that executes the symbolic execution driver generated by the symbolic execution driver generation unit to generate test cases; and a test code generation unit that generates test code for providing the test cases generated by the symbolic execution unit to a program to be tested and executing the program.

[0013] A computer program according to a third aspect of the present disclosure defines a computer program written in a computer-readable format to perform predetermined processing on a computer. The computer program can be provided to a computer capable of executing various program codes in a computer-readable format via a storage medium or communication medium, such as an optical disk, a magnetic disk, or a semiconductor memory, or a communication medium such as a network. By installing the computer program according to the third aspect of the present disclosure on a computer via any of these media, a cooperative effect is exerted on the computer, and the same effects as those of the information processing device according to the first aspect of the present disclosure can be obtained.

[0014] According to the present disclosure, it is possible to provide an information processing device, an information processing method, and a computer program that, when testing a program, analyzes source code to automatically generate argument type information and object initialization functions, thereby creating meaningful test cases.

[0015] It should be noted that the effects described in this specification are merely examples, and the effects brought about by the present disclosure are not limited to these. Furthermore, the present disclosure may also bring about additional effects in addition to the effects described above.

[0016] Further objects, features, and advantages of the present disclosure will become apparent from the following detailed description based on the embodiments and accompanying drawings.

[0017] Fig. 1 is a diagram showing the functional configuration of an automatic test generation device 100 to which the present disclosure is applied. Fig. 2 is a flowchart showing an example of a processing procedure for automatically generating test code for source code. Fig. 3 is a diagram showing an example of the hardware configuration of an information processing device 2000.

[0018] Hereinafter, embodiments of the present disclosure will be described in the following order with reference to the drawings.

[0019] A. Overview B. Configuration of automatic test generation tool B-1. Generation of type information B-1-1. Static analysis B-1-2. Dynamic analysis B-2. Generation of object initialization functions B-2-1. Dynamic analysis B-2-2. Generation of new method sequences B-2-2-1. Random generation B-2-2-2. Mutation B-2-2-3. Use of machine learning B-2-2-4. Evaluation of generated method sequences B-2-3. Generation of object initialization functions B-3. Generation of symbolic execution drivers B-4. Symbolic execution and test code generation C. Test code generation processing procedure D. Configuration of information processing device

[0020] A. Overview In statically typed languages ​​such as C / C++, it is possible to express ambiguous types, such as void pointers. In dynamically typed languages, there is no need to specify the type of arguments passed to functions, so any type of value or object can be passed. The exact type of variable passed to a function cannot be determined until the program is executed.

[0021] To automatically generate effective test code with high coverage, the user (such as a program developer) must add type annotations to each function that uses a void pointer or each function in a dynamically typed language. Without these annotations, meaningless tests that simply pass a null pointer to a function that uses a void pointer, tests that cause null pointer dereference exceptions, or huge test cases that use all types used in the program as inputs will be generated.

[0022] In object-oriented languages, if the object of the class being tested or the object of the class being passed to the function being tested is not properly initialized, the processing of the class or function being tested may not execute correctly. For example, consider the linked list (double-linked list) class. The linked list class provides methods such as add (adding and inserting an element), size (obtaining the number of elements), get (obtaining the object stored in an element), set (replacing an element), and remove (deleting an element). When generating a test for the remove method (a method that deletes a node from a linked list), the add method must be executed before executing the remove method to add a node to the linked list and make the linked list non-empty; otherwise, the processing that is actually being tested (deleting a node) in the remove method will not execute. However, it is impossible to accurately determine how to initialize an object before executing a test of a certain method without actually running and analyzing the program. In other words, in the case of object-oriented languages, the behavior you want to test is not executed, and meaningless test cases are generated. For this reason, users need to create functions used to initialize objects and pass them to the test code automatic generation tool.

[0023] In short, to automatically generate meaningful tests with high coverage that reflect the user's design intent, the user needs to input annotations for type information and object initialization functions. However, providing type information and object initialization functions for all functions to be tested by the user is a very cumbersome task and places a heavy burden on the user.

[0024] For example, a test support device has been proposed that automatically generates drivers and stubs for test target functions in a dynamically typed language using annotations, while generating correct test drivers and stubs even when incorrect annotations (comments in the source code) are added (see Patent Document 1). Here, a test driver is a program that calls and executes the test target functions and classes. A stub is a dummy function and class that the test target functions and classes call (stubs may be created to enable testing even for functions and classes that are difficult to test because they access networks, databases, etc.). This test support device repeatedly modifies the driver to eliminate errors when an undefined method of an object provided as a test case is called in the test target function, ultimately generating a correct test driver. However, adding annotations to all test target functions is a very cumbersome task and places a heavy burden on the user.

[0025] Therefore, this disclosure proposes a technology for automatically generating high-coverage test code by analyzing source code and automatically generating type information and object initialization functions. According to this disclosure, meaningful test code with high coverage that reflects the user's design intent can be automatically generated without the user having to provide type information of the source code or object initialization functions.

[0026] An automatic test code generation tool to which the present disclosure is applied does not require the user to input type annotations or object initialization functions. Furthermore, an automatic test code generation tool to which the present disclosure is applied can generate test code with restricted types for void pointers and function arguments in dynamically typed languages, and can generate test code that uses object initialization functions.

[0027] In this specification, the terms "test code," "test case," and "coverage" are used with the following meanings.

[0028] Test code: Source code for providing various test cases to a program and executing them. Test case: Input provided to the program under test. Coverage: In this specification, "coverage" refers to line coverage. Line coverage is a value expressed as a percentage that indicates which lines of source code are executable and which lines are not.

[0029] "Source code" and "program" both refer to computer programs, but the former refers to a data format that is easy for humans to understand and write, such as a programming language, while the latter refers to a format built into an executable format. Also, "function" and "method" are synonymous, but a function that belongs to an object is called a "method."

[0030] B. Configuration of Automatic Test Generation Tool FIG. 1 schematically illustrates the functional configuration of an automatic test generation device 100 to which the present disclosure is applied. The automatic test generation device 100 inputs source code and outputs test code for executing various test cases for the source code. A downstream test execution device (not shown) can use the test code generated by the automatic test generation device 100 to perform high-coverage testing of the source code. The automatic test generation device 100 can be implemented using a general information processing device such as a personal computer (PC). Furthermore, the automatic test generation device 100 may be a device physically integrated with the test execution device.

[0031] The automatic test generation device 100 shown in FIG. 1 includes a type information generation unit 101 , an object initialization function generation unit 102 , a symbolic execution driver generation unit 103 , a symbolic execution unit 104 , and a test code generation unit 105 .

[0032] The type information generation unit 101 is configured to analyze the source code of the test target and generate type information. Similarly, the object initialization function generation unit 102 is configured to analyze the source code of the test target and generate an object initialization function. Therefore, since the automatic test generation device 100 automatically generates type information and object initialization functions internally, it can automatically generate meaningful test code with high coverage even if the user does not provide type information of the source code or object initialization functions.

[0033] B-1. Generation of Type Information In this section B-1, we will explain how the type information generation unit 101 generates type information necessary to generate tests with high coverage and little waste.

[0034] B-1-1. Static Analysis When the source code to be tested is written in a statically typed language, the type information generation unit 101 first performs static analysis to find out what types of variables are created in the function that calls the function to be tested and what types of variables are passed as arguments to the function to be tested, thereby narrowing down the types of variables passed to a certain number of functions.

[0035] For example, consider the function swap consisting of the following code: This function swap takes a void pointer as an argument.

[0036]

[0037] Although any type is allowed for a void pointer, the type information generation unit 101 refers to the line in the source code where the function under test is called and checks what type of variable is passed by the function that called the function under test. In the above source code example, the type information generation unit 101 checks line 7 and determines that the function under test, swap, is called by function function, and then checks line 5 and determines that function function declares the types of variables x and y that it passes to function swap under test. Therefore, the type information generation unit 101 can narrow down the type passed to function swap under test to type int.

[0038] B-1-2. Dynamic Analysis The simple static analysis described in Section B-1-1 above is not enough to accurately narrow down the types for all cases. Furthermore, if the source code being tested is written in a dynamically typed language, any type of value or object can be stored in the same variable, making it impossible to determine what type of value or object will be passed to a function without actually running the program. For such functions, type information can be collected by performing the dynamic analysis described in Section B-1-2.

[0039] When performing dynamic analysis, the type information generation unit 101 first instruments the program under test so that the types of variables passed to the function under test are output as an execution trace immediately before calling each function under test. Next, the type information generation unit 101 passes input to the instrumented program and executes it. At this time, the type information generation unit 101 can generate input to the program using test case generation techniques such as random generation, fuzzing, and symbolic execution. When generating input to a program using fuzzing and symbolic execution, there are techniques for efficiently searching to reach a specific line of the program. By using such techniques, it is possible to find input that executes the function under test and efficiently obtain an execution trace.

[0040] The type information generation unit 101 then analyzes what types of variables and objects are being passed to the function under test from the execution trace collected by generating such test cases and executing the program under test using the generated test cases as input for a certain period of time.

[0041] For example, suppose the function to be tested is a function add consisting of the following code: In the fourth line of this code, the arguments x and y of the function to be tested add are of type int, and in the fifth line, the arguments x and y of the function to be tested add are of type float.

[0042]

[0043] The type information generation unit 101 performs instrumentation on the add function so that the types of variables passed to the function are output as an execution trace, and inputs and executes a test case generated by a predetermined method, thereby obtaining the following execution trace. This execution trace includes a trace result in which int type is passed to the arguments x and y of the test target function add, which corresponds to the fourth line of the source code, and a trace result in which float type is passed to the arguments x and y of the test target function add, which corresponds to the fifth line of the source code. The output of such an execution trace as an intermediate product by an automatic test generation tool is one of the features of the present disclosure.

[0044]

[0045] Therefore, the type information generating unit 101 can obtain type information from the execution trace that indicates that the arguments x and y of the test target function add may be of int type or of float type.

[0046] The symbolic execution driver generation unit 103 in the latter stage generates a symbolic execution driver using the type information collected through static and dynamic analysis of the source code by the type information generation unit 101. However, details of the symbolic execution driver generation will be explained in Section B-3 below.

[0047] Fuzzing is a technique for testing program behavior by repeatedly running a program while generating new input values ​​by modifying randomly generated input values ​​or input values ​​received from a user. Fuzzing can be done by modifying some of the input values ​​to increase program coverage or to modify some of the input values ​​to reach a specific line in the program.

[0048] Symbolic execution is a method for comprehensively exploring program behavior by treating program inputs as symbolic values ​​(symbols) rather than specific values. The program is executed while collecting branching conditions, and at the end of each execution path search, the collected branching conditions are solved using a constraint solver to generate input values ​​that will take that execution path.

[0049] B-2. Generation of Object Initialization Functions This section B-2 describes a method by which the object initialization function generator 102 automatically generates object initialization functions that are required to generate high-coverage tests for code to be tested in an object-oriented language.

[0050] B-2-1. Dynamic Analysis As explained in Section B-1-2 above, the object initialization function generation unit 102 instruments the program to be tested, inputs input values ​​generated using test case generation techniques such as random generation, fuzzing, and symbolic execution to the instrumented program, executes the program, and obtains an execution trace.

[0051] However, the object initialization function generation unit 102 records, as an execution trace, which methods of an object were called and in what order from the time the object was created to the time it was deleted, so that a sequence of methods for initializing each object can be collected. To this end, the object initialization function generation unit 102 instruments the program under test so that, immediately before a method is called, the object type name, object identification information, and the name of the called method are output as an execution trace. The object initialization function generation unit 102 then passes input to the instrumented program, executes it, collects execution traces, extracts method sequences for initializing the object from the collected execution traces, and generates an object initialization function from the extracted method sequences.

[0052] As an example, a case will be described in which an execution trace is collected for the following code. In this code, an object s1 of the SampleClass class is generated on line 12, and an object s2 of the SampleClass class is generated on line 13. Furthermore, the init, setLocation, calc, and finalize methods of object s1 are called in sequence on lines 14, 16, 17, and 19, respectively, and the init, calc, and finalize methods of object s2 are called in sequence on lines 15, 18, and 20, respectively. The object initialization function generator 102 performs instrumentation on this code immediately before calling a method so as to output the object type name, object identification information, and the name of the called method as an execution trace.

[0053]

[0054] The object initialization function generation unit 102 performs instrumentation on the above code, inputs the generated test cases into the instrumented program, and executes it, thereby collecting the object type name (Class), object identification information (id), and called method name (function) as an execution trace, as follows: where the object type name is SampleClass. Also, suppose that "1" and "2" are automatically assigned as object identification information to objects s1 and s2 generated from SampleClass, respectively.

[0055]

[0056] From the information acquired by the execution trace, the object initialization function generation unit 102 can extract a method sequence that indicates what methods were called for the object and in what order from the time the object to be passed to the function under test was generated until the function under test was called. By grouping the execution trace for each object's identification information, the object initialization function generation unit 102 can obtain the following two method sequences: method sequence 1 corresponding to object s1 and method sequence 2 corresponding to object s2.

[0057] Method sequence 1: init → setLocation → calc → finalize Method sequence 2: init → calc → finalize

[0058] The process of acquiring an execution trace by the object initialization function generation unit 102 described in Section B-2-1 can be performed together with the process performed by the type information generation unit 101 described in Section B-1-2 above. One of the features of the present disclosure is that the automatic test generation tool outputs such an execution trace as an intermediate product (ibid.).

[0059] B-2-2. Generating a New Method Sequence There are cases where the information from the execution trace described in B-2-1 above is insufficient. For this reason, the object initialization function generation unit 102 generates a new method sequence using the method described in B-2-2, and adds it to generate the object initialization function.

[0060] There are three methods for generating a method sequence: random generation, mutation, and machine learning. Each generation method will be explained below.

[0061] B-2-2-1. Random Generation In this method, a sequence is randomly generated from the public methods of the object passed to the function under test.

[0062] B-2-2-2. Mutation This method generates a new method sequence by modifying part of the method sequence extracted by the dynamic analysis described in Section B-2-1 above. There are three ways to modify part of a method sequence: adding a method, deleting a method, and modifying a method.

[0063] Add Method: Randomly select one of the object's public methods and insert it at a random position in the method sequence. Delete Method: Randomly select one of the object's public methods and delete it. Modify Method: Randomly select one of the object's methods and replace it with another public method.

[0064] B-2-2-3. Use of Machine Learning This method learns from the execution trace collected by the dynamic analysis described in Section B-2-1 above what methods of the object passed to the function under test are called before the function under test is called. For example, by learning from naming conventions that if the function under test is remove**, functions such as add***, init***, and insert*** tend to be called before calling such a method, it is possible to generate a sequence of methods that need to be executed before calling the function under test, even if the function under test does not appear in the execution trace.

[0065] B-2-2-4. Evaluation of Generated Method Sequences The object initialization function generation unit 102 evaluates new method sequences that have been generated by applying changes using at least one of the above-described methods of random generation, mutation, and machine learning, and deletes unnecessary method sequences that do not contribute to coverage, leaving only method sequences that can increase coverage.

[0066] Specifically, the object initialization function generator 102 generates a test driver using the newly generated method sequence, and performs fuzzing and symbolic execution on the test driver to measure coverage. Only method sequences that cover lines that cannot be reached by existing method sequences are retained, and those that do not are deleted. The object initialization function generator 102 repeats this process for a certain period of time or until coverage no longer improves.

[0067] For example, consider the case where a method change is made to the method sequence 1: init → setLocation → calc → finalize shown in section B-2-1 above. As an example, a mutation (method change) can be used to change the method setLocation included in method sequence 1 to another method, finalize, to generate a new method sequence 1': init → finalize → calc → finalize as shown below.

[0068] Method sequence 1': init → finalize → calc → finalize

[0069] When the calc method is a function for which a test is to be generated, the following test driver is generated, and fuzzing and symbolic execution are performed on this test driver.

[0070]

[0071] At this time, the coverage of calc is measured. If the unreached lines of the calc method can be covered, a new method sequence 1' is added as a method sequence of SampleClass. On the other hand, if the unreached lines of the calc method cannot be covered, this new method sequence 1' is deleted, and when creating a new method sequence in the future, no changes will be made to make it the same as method sequence 1'.

[0072] B-2-3. Generating Object Initialization Functions The object initialization function generation unit 102 generates an object initialization function that initializes an object using a newly generated method sequence according to the explanation in Section B-2-2 above. If the calc method in the above SampleClass example is the function for which tests are to be generated, the object initialization function generation unit 102 generates an object initialization function for each method sequence generated as explained in Section B-2-2 above. Specifically, the object initialization function generation unit 102 generates the following object initialization function for the above method sequence 1 (init → setLocation → calc → finalize):

[0073]

[0074] Furthermore, the object initialization function generating unit 102 generates the following object initialization function for the above method sequence 2 (init→calc→finalize).

[0075]

[0076] B-3. ​​Generation of Symbolic Execution Driver The symbolic execution driver generation unit 103 generates code to be run on the symbolic execution engine, i.e., a symbolic execution driver, from the source code received as input, the type information generated by the type information generation unit 101 (see section B-1 above), and the object initialization function generated by the object initialization function generation unit 102 (see section B-2 above).

[0077] The symbolic execution driver is code that simply initializes and symbolizes variables to be passed to the function under test and calls the function under test. When generating a symbolic execution driver, the symbolic execution driver generation unit 103 uses the type information generated by the type information generation unit 101 and the object initialization function generated by the object initialization function generation unit 102. If the function under test receives a void pointer as an argument or is a function in a dynamically typed language, the symbolic execution driver generation unit 103 initializes variables with types that may be passed to the function, based on the type information generated by the type information generation unit 101. If multiple types may be passed, the symbolic execution driver generation unit 103 generates a separate symbolic execution driver for each type.

[0078] In the example of the add function shown in Section B-1-2 above, type information indicating that variables of type int and type float may be passed to the function under test can be obtained from the type information generation unit 101. In this case, the symbol execution driver generation unit 103 generates separate symbol execution drivers for the int and float types. That is, the symbol execution driver generation unit 103 generates the following symbol execution driver that passes an int type as an argument:

[0079]

[0080] Furthermore, the symbol execution driver generation unit 103 generates the following symbol execution driver that passes a float type as an argument.

[0081]

[0082] Furthermore, when an object is passed as an argument to the function to be tested, the symbolic execution driver generation unit 103 checks whether there is a function capable of initializing the object among the object initialization functions generated as described above by the object initialization function generation unit 102. If such a function exists, the symbolic execution driver generation unit 103 calls the function to initialize the object, and inserts code into the symbolic execution driver that initializes and symbolizes variables passed as arguments to the object initialization function.

[0083] In the case of the above-mentioned SampleClass example, the symbol execution driver generation unit 103 generates, for example, the following code for the object initialization function of the above-mentioned method sequence 1 (init->setLocation->calc->finalize): The symbol execution driver generation unit 103 inserts into the symbol execution driver code that initializes and symbolizes variables to be passed as arguments to the object initialization function.

[0084]

[0085] B-4. Symbolic Execution and Test Code Generation The symbolic execution unit 104 generates test cases by executing the symbolic execution driver generated by the symbolic execution driver generation unit 103 on a symbolic execution engine. Then, the test code generation unit 105 generates test code for the function to be tested using the test cases generated by the symbolic execution unit 105 as input values.

[0086] At this time, if necessary, the test code generation unit 105 generates test code using the object initialization function generated by the object initialization function generation unit 102. In the case of the above-mentioned example of SampleClass, the test code generation unit 105 generates the following test code using the object initialization function generated by the object initialization function generation unit 102.

[0087]

[0088] C. Test Code Generation Processing Procedure Figure 2 shows, in the form of a flowchart, an example of a processing procedure for automatically generating test code from source code in the automatic test generation device 100 according to this embodiment. The processing procedure for generating test code will be described below with reference to Figure 2.

[0089] When the source code to be tested is written in a statically typed language and static analysis is to be performed on the source code to be tested (Yes in step S201), the type information generation unit 101 first performs static analysis to examine what types of variables are created in the function that calls the function to be tested and what types of variables are passed to the function to be tested as arguments, thereby narrowing down the types of variables passed to a certain number of functions and collecting type information (step S202).

[0090] When performing dynamic analysis on the source code to be tested (Yes in step S203), such as when simple static analysis alone is not sufficient to accurately narrow down the types for all cases or when the source code to be tested is written in a dynamically typed language, the type information generation unit 101 performs first instrumentation on the program to be tested so that the types of variables to be passed to the functions to be tested are output as an execution trace immediately before calling each function to be tested (step S204).

[0091] In addition, the object initialization function unit 102 performs second instrumentation on the program to be tested so as to output, as an execution trace, which methods of an object were called and in what order from the time the object was created to the time it was deleted, so that the method sequence for initializing each object can be collected (step S205).

[0092] Next, inputs to the program are generated using a test case generation technique such as random generation, fuzzing, or symbolic execution (step S206). The generated inputs are then passed to the instrumented program in steps S204 and S205, and the program is executed to obtain an execution trace as an intermediate product. In practice, the execution trace is collected by generating test cases in steps S206 and S207 and executing the program to be tested using the generated test cases as input for a certain period of time.

[0093] The type information generation unit 101 acquires type information regarding the types of variables to be passed to the function under test from the execution trace (see, for example, Table 1) that describes the types of variables to be passed to the function under test (step S208).

[0094] The object initialization function generator 102 also collects a method sequence for initializing each object from an execution trace that records the type name of the called object, the object's identification information, and the called method name in that order (step S209).

[0095] Furthermore, the object initialization function generation unit 102 generates a new method sequence using at least one of random generation, mutation, and use of a machine learning model (step S210). In step S210, among the newly generated method sequences, only those that can increase coverage more than the method sequence generated in step S209 may be kept, and the rest may be deleted.

[0096] Next, the object initialization function generating unit 102 generates an object initialization function for each of the method series acquired in the preceding steps S209 and S210 (step S211).

[0097] Next, the symbolic execution driver generation unit 103 generates a symbolic execution driver using the type information collected by the type information generation unit 101 through static and dynamic analysis of the source code (step S212). If multiple types may be passed, the symbolic execution driver generation unit 103 generates a symbolic execution driver for each type. In addition, the symbolic execution driver generation unit 103 checks whether the object initialization functions generated by the object initialization function generation unit 102 include a function that can initialize the object. If such a function exists, the symbolic execution driver generation unit 103 calls the function to initialize the object and inserts code into the symbolic execution driver that initializes and symbolizes variables passed as arguments to the object initialization function.

[0098] Next, the symbol execution unit 104 executes the symbol execution driver generated by the symbol execution driver generation unit 103 on the symbol execution engine to generate test cases (step S213).

[0099] The test code generation unit 105 generates test code for the function to be tested using the test case generated by the symbol execution unit 105 as input values ​​(step S214), and outputs this to the downstream test execution device, thereby completing this processing.

[0100] D. Configuration of Information Processing Device In this section D, the hardware configuration of an information processing device that can operate as the automatic test generation device 100 will be described.

[0101] 3 shows an example of the hardware configuration of an information processing device 2000. This information processing device 2000 includes a CPU (Central Processing Unit) 2001, a ROM (Read Only Memory) 2002, a RAM (Random Access Memory) 2003, a host bus 2004, a bridge 2005, an expansion bus 2006, an interface unit 2007, an input unit 2008, an output unit 2009, a storage unit 2010, a drive 2011, and a communication unit 2013.

[0102] The CPU 2001 controls the overall operation of the information processing device 2000 in accordance with various programs. The ROM 2002 stores programs (such as a basic input / output system) and calculation parameters used by the CPU 2001 in a nonvolatile manner. The RAM 2003 is used to load programs used in the execution of the CPU 2001 and to temporarily store parameters such as work data that change as appropriate during program execution. Programs loaded into the RAM 2003 and executed by the CPU 2001 include, for example, various application programs and an operating system (OS).

[0103] The CPU 2001, ROM 2002, and RAM 2003 are interconnected by a host bus 2004, which includes a CPU bus and other components. The CPU 2001 executes various application programs in an execution environment provided by the OS through the cooperative operation of the ROM 2002 and RAM 2003, thereby enabling various functions and services to be realized. If the information processing device 2000 is a personal computer, the OS may be, for example, Microsoft Windows (registered trademark), Unix (registered trademark), or a successor OS. The application programs may also include test code automatic generation tools and test support programs that support program testing using the generated test code.

[0104] The host bus 2004 is connected to an expansion bus 2006 via a bridge 2005. The expansion bus 2006 is, for example, a PCI (Peripheral Component Interconnect) bus or PCI Express, and the bridge 2005 is based on the PCI standard. However, the information processing device 2000 does not need to be configured so that the circuit components are separated by the host bus 2004, bridge 2005, and expansion bus 2006, and may be implemented so that almost all circuit components are interconnected by a single bus (not shown).

[0105] The interface unit 2007 connects peripheral devices such as an input unit 2008, an output unit 2009, a storage unit 2010, a drive 2011, and a communication unit 2013 in accordance with the standards of the expansion bus 2006. However, not all of the peripheral devices shown in FIG. 3 are necessarily required, and the information processing device 2000 may further include peripheral devices not shown. Furthermore, the peripheral devices may be built into the main body of the information processing device 2000, or some of the peripheral devices may be externally connected to the main body of the information processing device 2000.

[0106] The input unit 2008 is composed of an input control circuit that generates an input signal based on an input from a user and outputs the signal to the CPU 2001. If the information processing device 2000 is a personal computer, the input unit 2008 may include a keyboard, a mouse, a touch panel, and may further include a camera and a microphone. The output unit 2009 includes a display device such as a liquid crystal display (LCD) device, an organic electroluminescence (EL) display device, and an LED (light emitting diode).

[0107] The storage unit 2010 stores files such as programs (applications, OS, etc.) executed by the CPU 2001 and various data. The storage unit 2010 is configured with a large-capacity storage device such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive), but may also include an external storage device.

[0108] The removable storage medium 2012 is a storage medium configured as a cartridge, such as a microSD card. The drive 2011 performs read and write operations on the loaded removable storage medium 113. The drive 2011 outputs data read from the removable storage medium 2012 to the RAM 2003 or the storage unit 2010, and writes data on the RAM 2003 or the storage unit 2010 to the removable storage medium 2012.

[0109] The communication unit 2013 is a device that performs wireless communication such as Wi-Fi (registered trademark), Bluetooth (registered trademark), or a cellular communication network such as 4G or 5G. The communication unit 2013 may also include terminals such as a Universal Serial Bus (USB) or a High-Definition Multimedia Interface (HDMI) (registered trademark), and may further include a function for performing HDMI (registered trademark) communication with a USB device such as a scanner or printer, or a display.

[0110] The information processing device 2000 can also operate as a test execution device that tests a test target program using test code generated by the automatic test generation device 100. Furthermore, one information processing device 2000 can operate as both the automatic test generation device 100 and the test execution device. The information processing device 2000 is configured using, for example, a PC, and operates as the automatic test generation device 100 and the test execution device by executing a predetermined computer program.

[0111] The present disclosure has been described in detail above with reference to specific embodiments. However, the present disclosure should not be construed as being limited to the above-described embodiments, and it is obvious that those skilled in the art can modify or substitute the embodiments without departing from the spirit of the present disclosure. Furthermore, the effects described in this specification are merely examples, and the effects brought about by the present disclosure are not limited thereto, and additional effects not described in this specification may exist.

[0112] The present disclosure can be applied to an automatic test generation tool for programs whose source code is written using various programming languages, including statically typed languages, dynamically typed languages, and object-oriented languages, to relieve users of burdensome and extremely cumbersome tasks such as interpreting the types of variables used in the source code and assigning object initialization functions.

[0113] In short, the present disclosure has been described in the present specification by way of example only, and the contents of the specification should not be interpreted in a limiting manner. To determine the gist of the present disclosure, the claims should be taken into consideration.

[0114] The series of processes described in this specification can be executed by hardware, software, or a configuration that combines hardware and software. When executing processes by software, a program recording a processing sequence related to realizing the present disclosure is installed in memory in a computer incorporated in dedicated hardware and executed. It is also possible to install the program in a general-purpose computer capable of executing various processes and execute the processes related to realizing the present disclosure.

[0115] The program can be stored in advance on a recording medium installed in the computer, such as a HDD, SSD, or ROM. Alternatively, the program can be temporarily or permanently stored on a removable recording medium such as a flexible disk, CD-ROM (Compact Disc Read Only Memory), MO (Magneto Optical) disk, DVD (Digital Versatile Disc), BD (Blu-Ray Disc (registered trademark)), magnetic disk, or USB (Universal Serial Bus) memory. Using such a removable recording medium, a program related to the realization of the present disclosure can be provided as so-called package software.

[0116] The program may also be transferred wirelessly or via a wire from a download site to a computer via a network such as a wide area network (WAN) typified by cellular, a local area network (LAN), the Internet, etc. The computer can receive the program transferred in this manner and install it in a large-capacity storage device such as an HDD or SSD within the computer.

[0117] The present disclosure may also be configured as follows.

[0118] (1) An information processing device comprising: an analysis unit that analyzes source code, collects type information of the source code, and generates an object initialization function; a symbolic execution driver generation unit that generates a symbolic execution driver using the type information and the object initialization function; a symbolic execution unit that executes the symbolic execution driver generated by the symbolic execution driver generation unit to generate a test case; and a test code generation unit that generates test code for providing the test case generated by the symbolic execution unit to a program to be tested and executing it.

[0119] (2) The information processing device described in (1) above, wherein the analysis unit performs static analysis on a program to be tested in a statically typed language, investigates the types of variables created by functions that call the function to be tested and the functions that pass them, and narrows down the types of arguments to be passed to the function to be tested.

[0120] (3) The information processing device described in any one of (1) or (2) above, wherein the analysis unit instruments a program to be tested in a dynamically typed language so that the types of variables to be passed to the function to be tested are output as an execution trace immediately before calling the function to be tested, passes input to the instrumented program, executes it, and collects the execution trace, and analyzes the type information and objects of the variables passed to the function to be tested from the collected execution trace.

[0121] (3-1) The information processing device according to (3), wherein the analysis unit generates inputs to the program under test using at least one of random generation, fuzzing, and symbolic execution.

[0122] (3-2) The information processing device described in (3-1) above, wherein the analysis unit, when using at least one of fuzzing and symbolic execution, utilizes a technique for efficiently searching to reach a specific line of the program, discovers an input that executes the function under test, and obtains an execution trace.

[0123] (4) The information processing device described in any one of (1) to (3) above, wherein the analysis unit performs instrumentation to output the methods called and the order of the calls between the time an object is created and the time it is deleted as an execution trace so that a method sequence for initializing each object can be collected, and analyzes the execution trace to generate an object initialization function.

[0124] (4-1) The information processing device described in (4) above, wherein the analysis unit instruments a test target program in an object-oriented language so as to output an object type name, object identification information, and the called method name as an execution trace immediately before calling a method, passes input to the instrumented program and executes it to collect an execution trace, extracts a method sequence for initializing the object from the collected execution trace, and generates an object initialization function from the extracted method sequence.

[0125] (4-2) The information processing device described in (4) above, wherein the analysis unit extracts from the execution trace a method sequence that indicates what methods were called on the object and in what order from the time the object to be passed to the function under test was generated until the function under test was called.

[0126] (5) The information processing device according to any one of (3) or (4) above, wherein an input to the instrumented program is generated using at least one of random generation, fuzzing, and symbolic execution.

[0127] (6) The information processing device according to (4), wherein the analysis unit further generates a new method sequence and generates an object initialization function in addition to the method sequence extracted from the execution trace.

[0128] (6-1) The information processing device described in (6) above, wherein the analysis unit executes a test driver generated using a new method sequence to measure coverage, and keeps only the method sequence that covers lines that cannot be reached by the existing method sequence, and deletes the other method sequence.

[0129] (7) The information processing device according to (6), wherein the analysis unit randomly generates a new method sequence from among public methods of an object to be passed to a function to be tested.

[0130] (8) The information processing device according to (6), wherein the analysis unit modifies a part of the method sequence extracted from the execution trace to generate a new method sequence.

[0131] (9) The information processing device according to (8), wherein the analysis unit adds some methods randomly selected from public methods of the object to change the method sequence extracted from the execution trace.

[0132] (10) The information processing device according to (8), wherein the analysis unit deletes some randomly selected methods to change the method sequence extracted from the execution trace.

[0133] (11) The information processing device according to (8), wherein the analysis unit replaces some randomly selected methods with other public methods to change the method sequence extracted from the execution trace.

[0134] (12) The information processing device described in (6) above, wherein the analysis unit uses machine learning to learn from the execution trace which method of an object passed to the function under test will be called, and generates a sequence of methods that need to be executed before calling the function under test.

[0135] (13) The information processing device described in any one of (8) to (12) above, wherein the analysis unit measures coverage by performing fuzzing or symbolic execution on the test driver generated using the new method sequence created by the changes, and leaves only the method sequence that covers lines that cannot be reached by the existing method sequence.

[0136] (14) The information processing device according to any one of (1) to (13), wherein the symbolic execution driver generation unit initializes and symbolizes variables to be passed to the function to be tested, and generates a symbolic execution driver consisting of code that simply calls the function to be tested.

[0137] (15) The information processing device described in (14) above, wherein the symbolic execution driver generation unit generates a symbolic execution driver by initializing and symbolizing variables with types that may be passed to the function being tested based on the collected type information in the case of a function that receives a void pointer as an argument and a function of a dynamically typed language.

[0138] (16) The information processing device described in (14) above, wherein when an object is passed as an argument to a function to be tested, the symbol execution driver generation unit initializes the object by calling a function in the generated object initialization function that can initialize the object, and inserts code into the symbol execution driver that initializes and symbolizes variables passed as arguments to the object initialization function.

[0139] (17) The information processing device according to any one of (1) to (16), wherein the test code generation unit generates test code for a function to be tested using test cases generated by executing the symbol execution driver generated by the symbol execution driver generation unit on a symbol execution engine as input values.

[0140] (17-1) The information processing device according to (17), wherein the test code generation unit generates test code using the object initialization function generated by the analysis unit, as necessary.

[0141] (18) An information processing method comprising: an analysis step of analyzing source code and generating type information and an object initialization function of the source code; a symbolic execution driver generation step of generating a symbolic execution driver using the type information and the object initialization function; a symbolic execution step of executing the symbolic execution driver generated in the symbolic execution driver generation step to generate test cases; and a test code generation step of generating test code for providing the test cases generated in the symbolic execution step to a program to be tested and executing the program.

[0142] (19) A computer program written in a computer-readable format to cause a computer to function as: an analysis unit that analyzes source code and generates type information and object initialization functions for the source code; a symbolic execution driver generation unit that generates a symbolic execution driver using the type information and object initialization functions; a symbolic execution unit that executes the symbolic execution driver generated by the symbolic execution driver generation unit to generate test cases; and a test code generation unit that generates test code for providing the test cases generated by the symbolic execution unit to a program to be tested and executing it.

[0143] 100... Automatic test generation device, 101... Type information generation unit, 102... Object initialization function generation unit, 103... Symbolic execution driver generation unit, 104... Symbolic execution unit, 105... Test code generation unit, 2000... Information processing device, 2001... CPU, 2002... ROM, 2003... RAM, 2004... Host bus, 2005... Bridge, 2006... Expansion bus, 2007... Interface unit, 2008... Input unit, 2009... Output unit, 2010... Storage unit, 2011... Drive, 2012... Removable recording medium, 2013... Communication unit

Claims

1. An information processing device comprising: an analysis unit that analyzes source code to collect type information of the source code and generate an object initialization function; a symbolic execution driver generation unit that generates a symbolic execution driver using the type information and the object initialization function; a symbolic execution unit that executes the symbolic execution driver generated by the symbolic execution driver generation unit to generate test cases; and a test code generation unit that generates test code for providing the test cases generated by the symbolic execution unit to a program to be tested and executing it.

2. The information processing device according to claim 1, wherein the analysis unit performs static analysis on a program to be tested in a statically typed language, investigates the types of variables created by functions calling the function to be tested and the functions that pass them, and narrows down the types of arguments to be passed to the function to be tested.

3. The information processing device of claim 1, wherein the analysis unit instruments the program under test so as to output the types of variables passed to the function under test as an execution trace immediately before calling the function under test, passes input to the instrumented program, executes it, collects the execution trace, and analyzes type information and objects of the variables passed to the function under test from the collected execution trace.

4. The information processing device according to claim 1, wherein the analysis unit performs instrumentation to output the methods called and the order in which they were called from the time an object is created to the time it is deleted as an execution trace so as to collect a sequence of methods for initializing each object, and analyzes the execution trace to generate an object initialization function.

5. The information processing apparatus according to claim 3, wherein an input to the instrumented program is generated using at least one of the techniques of random generation, fuzzing, and symbolic execution.

6. The information processing device according to claim 4, wherein the analysis section further generates a new method sequence and generates an object initialization function in addition to the method sequence extracted from the execution trace.

7. The information processing device according to claim 6, wherein the analysis unit randomly generates a new method sequence from among public methods of an object to be passed to a function under test.

8. The information processing device according to claim 6, wherein the analysis section modifies a part of the method sequence extracted from the execution trace to generate a new method sequence.

9. The information processing device according to claim 8, wherein the analysis section modifies the method sequence extracted from the execution trace by adding some methods randomly selected from public methods of an object.

10. The information processing device according to claim 8, wherein the analysis section modifies the method sequence extracted from the execution trace by deleting some methods selected at random.

11. The information processing device according to claim 8, wherein the analysis unit modifies the method sequence extracted from the execution trace by replacing a randomly selected portion of methods with another public method.

12. The information processing device according to claim 6, wherein the analysis unit uses machine learning to learn from the execution trace which method of an object passed to the function under test is called, and generates a sequence of methods that need to be executed before calling the function under test.

13. The information processing device according to claim 8, wherein the analysis unit measures coverage by performing fuzzing or symbolic execution on a test driver generated using the new method sequence created by applying the changes, and leaves only the method sequence that covers lines that cannot be reached by the existing method sequence.

14. The information processing device according to claim 1, wherein said symbolic execution driver generation unit generates a symbolic execution driver consisting of code that initializes and symbolizes variables to be passed to a function under test, and that simply calls the function under test.

15. The information processing device according to claim 14, wherein the symbolic execution driver generation unit generates a symbolic execution driver by initializing and symbolizing variables with types that may be passed to a function under test based on the collected type information.

16. The information processing device according to claim 14, wherein when an object is passed as an argument to a function to be tested, the symbol execution driver generation unit initializes the object by calling a function in the generated object initialization function that can initialize the object, and inserts code into the symbol execution driver that initializes and symbolizes variables passed as arguments to the object initialization function.

17. The information processing device according to claim 1, wherein the test code generation unit generates test code for a function to be tested by using test cases generated by executing the symbol execution driver generated by the symbol execution driver generation unit on a symbol execution engine as input values.

18. An information processing method comprising: an analysis step of analyzing source code and generating type information and an object initialization function for the source code; a symbolic execution driver generation step of generating a symbolic execution driver using the type information and the object initialization function; a symbolic execution step of executing the symbolic execution driver generated in the symbolic execution driver generation step to generate test cases; and a test code generation step of generating test code for providing the test cases generated in the symbolic execution step to a program to be tested and executing it.

19. A computer program written in a computer-readable format to cause a computer to function as: an analysis unit that analyzes source code and generates type information and an object initialization function for the source code; a symbolic execution driver generation unit that generates a symbolic execution driver using the type information and the object initialization function; a symbolic execution unit that executes the symbolic execution driver generated by the symbolic execution driver generation unit to generate test cases; and a test code generation unit that generates test code for providing the test cases generated by the symbolic executor to a program to be tested and executing it.

Citation Information

Patent Citations

  • Program, test case generation method, and test case generation apparatus

    JP2014006643A

Cited By

  • Generation method and device of initialization code of MCU, medium, program product and equipment

    CN119917158A