Program modification support system and program modification support method

Abstract

To generate test cases effective for verification of a patch candidate without generating useless test cases.SOLUTION: A program correction support system supports correction of a program by a computer including an arithmetic unit and a storage unit. The arithmetic unit analyzes a source code to acquire a condition of an input value of the source code obtained by the analysis and an output and increases test cases on the basis of information at the time of correction, which indicates candidates relating to correction about the condition of the input value and the output, and a test addition reference for increasing test cases where the source code is to be tested by a test code.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a program correction support system and a program correction support method. [Background technology]

[0002] Conventionally, there is a technique for identifying a part containing a defect and generating a patch for the part in a debugging operation in a unit test of a program. For example, Patent Document 1 discloses such a technique. Patent Document 1 describes identifying a defective part in a software program, generating a possible repair at the defective part based on a repair candidate, automatically generating a first test to test the possible repair, generating a first query for a first test oracle based on the first test, obtaining a response to the first query from a human, generating a first human-provided test oracle based on the first query and the obtained response to the first query, adding the first human-provided test oracle to the first test to generate an extended first test, augmenting a test suite to include the extended first test, and testing the possible repair using the augmented test suite. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-29015 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned Patent Document 1, only rearrangement of arguments is disclosed as a method for generating inputs for test cases. With this method, it is not always possible to generate test cases that are effective for verifying patch candidates.

[0005] For example, if the test case for the function median(n1,n2,n3) that finds the median value originally includes "the output of median(2,6,8) is 6," it is not possible to generate median(8,6,2) or median(6,8,2) as a new input. Also, for example, if a bug occurs when a negative number is entered as an argument in the original source code, the location of the bug will be overlooked when identifying the bug. Furthermore, if a similar bug occurs in the generated patch, it will be overlooked when verifying the patch candidate, and the patch with the bug will be output.

[0006] On the other hand, brute-force or random generation of test case inputs generates an unnecessarily large number of test cases and increases the effort required of developers to set the outputs (expectations).

[0007] An object of the present invention is to provide a technique capable of generating test cases that are effective for verifying patch candidates without generating unnecessary test cases. [Means for solving the problem]

[0008] The program modification support system of the present invention is a program modification support system that supports program modification using a computer having an operation unit and a memory unit, wherein the operation unit analyzes source code, obtains input value conditions and outputs of the source code obtained by the analysis, and expands the test cases based on modification time information indicating candidates for modification of the input value conditions and outputs and test addition criteria for increasing the number of test cases that test the source code using test code. Effect of the Invention

[0009] According to the present invention, it is possible to generate test cases that are effective for verifying patch candidates without generating unnecessary test cases. [Brief description of the drawings]

[0010] [Figure 1] FIG. 2 is a diagram illustrating an example of a hardware configuration of the program modification support system according to the present embodiment. [Diagram 2] 2 is a diagram illustrating an example of a functional configuration of the program modification device illustrated in FIG. 1; [Diagram 3] FIG. 13 is a diagram illustrating an example of source code. [Figure 4] FIG. 13 illustrates an example of a test code. [Diagram 5] FIG. 11 is a diagram showing an example of a defective part candidate; [Figure 6] FIG. 13 is a diagram illustrating an example of patch candidates. [Figure 7] FIG. 13 is a diagram showing an example of a test addition criterion. [Figure 8] FIG. 13 is a diagram illustrating an example of generated test code. [Figure 9] FIG. 13 is a diagram showing an example of a correction result. [Figure 10] FIG. 13 is a diagram illustrating an example of a flowchart showing a processing procedure of a program correction process performed in the present embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of a flowchart showing the processing procedure of the test generation process performed in S1003 shown in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The following description and drawings are examples for explaining the present invention, and appropriate omissions and simplifications have been made for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0012] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

[0013] In the following explanation, various information may be described using expressions such as "database," "table," and "list," but the various information may be expressed in other data structures. To indicate independence of data structure, "XX table," "XX list," and the like may be referred to as "XX information." When describing identification information, when expressions such as "identification information," "identifier," "name," "ID," and "number" are used, these are interchangeable.

[0014] When there are multiple components having the same or similar functions, they may be described by using the same reference numerals with different subscripts. However, when there is no need to distinguish between these multiple components, the subscripts may be omitted.

[0015] In addition, in the following description, processing performed by executing a program may be described, but the program is executed by a processor (e.g., a CPU (Central Processing Unit), a GPU (Graphics Processing Unit)) to perform a defined process using storage resources (e.g., a memory) and / or an interface device (e.g., a communication port) as appropriate, so the subject of the processing may be the processor. Similarly, the subject of the processing performed by executing a program may be a controller, device, system, computer, or node having a processor. The subject of the processing performed by executing a program may be a calculation unit, and may include a dedicated circuit that performs specific processing (e.g., an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)).

[0016] A program may be installed in a device such as a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource for storing a program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In the following description, two or more programs may be realized as one program, and one program may be realized as two or more programs.

[0017] The configuration of an embodiment of a program modification support system and a program modification support method according to the present invention will be described below.

[0018] 1 is a diagram showing an example of a hardware configuration of a program modification support system in this embodiment. In the following, an example is shown in which the program modification support system is configured by a program modification device 100, which is a single computer, but all or part of these functions may be distributed among one or more computers, such as a cloud, and may achieve similar functions by communicating with each other via a network. Specific processes performed by each unit constituting the program modification device 100 will be described later using flowcharts. As shown in FIG. 1, the program correction device 100 can be realized by a general computer having a processor 110 such as a CPU, a main storage device 120 such as a memory, an auxiliary storage device 130 such as an HDD (Hard Disk Drive), an input device 140 such as a scanner, keyboard, or mouse, an output device 150 such as a display, and a communication device 160 such as a NIC (Network Interface Card) for connecting to a communication network.

[0019] Fig. 2 is a diagram showing an example of a functional configuration of the program correction device 100 shown in Fig. 1. As shown in Fig. 2, the program correction device 100 according to this embodiment is configured to include a defect location identifying section 210, a patch candidate generating section 220, a test generating section 230, a patch candidate verifying section 240, and an information storage section 290. The information storage section 290 stores source code 291, test code 292, defect location candidates 293, patch candidates 294, test addition criteria 295, generated test codes 296, and correction results 297.

[0020] Various data (e.g., source code 291, test code 292, defect location candidate 293, patch candidate 294, test addition criterion 295, generated test code 296, and correction result 297) stored in the program correction device 100 or used for processing can be realized by the processor 110 reading them from the main storage device 120 or the auxiliary storage device 130 and using them. Also, each functional unit of the program correction device 100 (e.g., defect location identification unit 210, patch candidate generation unit 220, test generation unit 230, and patch candidate verification unit 240) can be realized by the processor 110 loading a predetermined program stored in the auxiliary storage device 130 into the main storage device 120 and executing it.

[0021] The above-mentioned specified program may be stored (downloaded) into the auxiliary external storage device 130 from a network via the communication device 160 or from a storage medium such as a USB (Universal Serial Bus), and then loaded onto the main storage device 120 and executed by the processor 110, or may be loaded directly onto the main storage device 120 and executed by the processor 110.

[0022] The defect location identifying section 210 identifies candidates for a defect location in the source code 291 by using the source code 291 and a test code 292 for testing the source code 291 .

[0023] The patch candidate generating unit 220 generates patch candidates by correcting candidates for defective parts of the source code based on a predetermined correction pattern.

[0024] The test generation unit 230 generates test cases for the source code based on test criteria related to verification of the candidate defect portions and / or the candidate patches in the source code, and test addition criteria described later.

[0025] Fig. 3 is a diagram showing an example of source code 291. Source code 291 is code that includes a defect. Although the specific details will be described later, Fig. 3 shows, for example, that source code 291 includes a defect location 301 in which "true" is written instead of "false" as it should be.

[0026] Fig. 4 is a diagram showing an example of the test code 292. The test code 292 is code for identifying a defective part of the source code 291. Fig. 4 shows that, for example, when a numerical value "5" is inputted into the source code 291, the expected value is "false" (test1), and when a numerical value "20" is inputted into the source code 291, the expected value is "false" (test2).

[0027] Fig. 5 is a diagram showing an example of the defect location candidate 293. The defect location candidate 293 is a table that stores candidates for defect locations included in the source code 291. As shown in Fig. 5, the defect location candidate 293 stores an ID for identifying a defect location and a defect location candidate identified by the ID in association with each other. Fig. 5 shows, for example, that the defect location candidate identified by the ID "1" is "line 2".

[0028] Fig. 6 is a diagram showing an example of a patch candidate 294. The patch candidate 294 is code obtained by modifying the source code 291 with a predetermined modification pattern. As shown in Fig. 6, the patch candidate 294 is stored in association with an ID for identifying the patch candidate and the patch candidate identified by the ID. Fig. 6 shows, for example, that the patch candidate identified by the ID "1" is code obtained by modifying the second line of the source code 291 from "x>1" to "x>=1".

[0029] Fig. 7 is a diagram showing an example of the test addition criteria 295. The test addition criteria 295 is a table that defines criteria for adding test cases. As shown in Fig. 7, the test addition criteria 295 stores an ID for identifying the criteria to be added and the test addition criteria identified by the ID in association with each other. Although not specifically illustrated in this example, it is assumed that the test criteria before being added are defined by various conventionally known methods.

[0030] In FIG. 7, for example, the additional criterion for a test identified by ID "1" is "an input value that increases the coverage of a defective part candidate." Patent Document 1 describes rearranging arguments as a method for generating input for a test case, but does not disclose the criteria for performing the test, and therefore source code defects cannot always be tested appropriately. In this example, in addition to "an input value that increases the coverage of a defective part candidate," "a boundary value of a conditional expression of a patch candidate" is exemplified as an additional criterion for a test. As will be described later in detail, in the former example, as a result of identifying a defective part, test code is generated for a defective (or highly likely) method or instruction block, so that the coverage of the part (the coverage rate of the process executed in the test, the conditional branch, and the execution path) is improved, thereby improving the verification accuracy of the patch candidate. In the latter example, as a result of generating a patch candidate, test code is generated based on the output patch candidate, so that, for example, by generating a boundary value, the verification accuracy of the patch candidate can be improved. As an example, if if(x>0) should be corrected to if(x>1), but if(x≧1) is generated as a patch candidate, the incorrect patch candidate can be eliminated by adding a test with the boundary value x=1 as input.

[0031] Fig. 8 is a diagram showing an example of generated test code 296. The generated test code 296 is code in which test code according to the standard defined in test addition standard 295 is added to the test code 292. As shown in Fig. 8, the generated test code 296 indicates that new codes shown as test3, test4, and test5 have been added to test1 and test2 included in the test code 292 shown in Fig. 4.

[0032] Fig. 9 is a diagram showing an example of a correction result 297. The correction result 297 is code showing the result of correcting the source code 291. As shown in Fig. 9, the correction result 297 shows that the source code 291 shown in Fig. 2 has been corrected with the code with ID "3" of the patch candidate 294 shown in Fig. 6.

[0033] FIG. 10 is a diagram showing an example of a flowchart illustrating the processing procedure of the program correction processing performed in this embodiment.

[0034] In S1010, the defect identifying unit 210 applies the test code 292 shown in Fig. 4 to the source code 291 shown in Fig. 3 to identify a defect in the source code 291. For example, the defect identifying unit 210 calculates the likelihood that each line in the source code contains a defect, and identifies a line for which the calculation result is equal to or greater than a predetermined threshold as a defect. The defect identifying unit 210 outputs the identified defect as a defect candidate 293 shown in Fig. 5.

[0035] In S1020, the patch candidate generation unit 220 outputs source code obtained by correcting the output defect location in the source code 291 using a predetermined correction pattern as a patch candidate 294. In this manner, the patch candidate generation unit 220 rewrites lines that are likely to contain defects to generate multiple correction patch proposals that pass the test. The above-mentioned predetermined correction pattern may be a pattern that describes the correction content, such as inserting missing statements, changing the instance generation method, changing a conditional expression, etc., which may be determined in advance.

[0036] In S1030, the test generation unit 230 generates test code for each of the output patch candidates based on the test addition criteria 295. The specific process of S1003 will be described later with reference to FIG.

[0037] In S1040, the patch candidate verification unit 240 executes the generated test code, and outputs patch candidates that pass various predetermined tests as final patch proposals.

[0038] FIG. 11 is a diagram illustrating an example of a flowchart showing the procedure of the test generation process performed in S1003 shown in FIG.

[0039] In S1110, the test generation unit 230 analyzes the source code 291 before modification, and obtains pairs of input value conditions and output values. For example, the test generation unit 230 analyzes the source code 291 shown in Fig. 3, and obtains input conditions "x>1" and "x≦1", and the corresponding outputs "true" and "false", respectively. This information can be realized using a technique such as symbolic execution.

[0040] In S1120, the test generation unit 230 obtains an input value and an expected value that executes an instruction or condition that is not executed in the test code before it is added. The expected value may be an output value of the source code before correction. For example, in the test code 292 shown in FIG. 4, a test case that satisfies "x≦1" in the source code 291 is not described. Therefore, the test generation unit 230 obtains an input value of "X=0" that satisfies the test case that satisfies "x≦1". As shown in S1120, the test generation unit 230 reads the test addition criteria 295 shown in FIG. 7 and obtains an "input value that increases the coverage of the defective part candidate" that corresponds to ID "1".

[0041] In S1130, the test generation unit 230 analyzes each of the patch candidates and obtains boundary values ​​of the input conditions. The boundary values ​​can be obtained using a technique such as symbolic execution. For example, the test generation unit 230 obtains "x=1" as a boundary value for the patch candidate with ID "1" shown in FIG. 6. The test generation unit 230 also obtains "x=10" as a boundary value for the patch candidates with IDs "2" and "3". As shown in S1130, the test generation unit 230 reads the test addition criteria 295 shown in FIG. 7 and obtains the "boundary value of the conditional expression of the patch candidate" corresponding to ID "2".

[0042] In S1140, the test generation unit 230 presents the acquired test input values ​​to the developer and acquires the correct expected values ​​from the developer. For example, the test generation unit 230 displays the acquired boundary values ​​"x=1" or "x=10" on a display, which is an example of the output device 150. Also, for example, the test generation unit 230 acquires from the developer the expected values ​​"true" or "false" input from a keyboard, which is an example of the input device 140.

[0043] In S1150, the test generation unit 230 generates test code with the input values ​​acquired in S1120 and S1130 and the expected value acquired in S1140 as test cases. For example, the test generation unit 230 adds code 801 including the input value "X=0" and expected value "false" acquired in S1120 to the generated test code 296. The test generation unit 230 also adds code 802 including the input value "X=1" and expected value "false" acquired in S1130, and code 803 including the input value "X=10" and expected value "true" to the generated test code 296.

[0044] In this embodiment, in a program correction support system that supports program correction by a computer having a calculation unit (e.g., a CPU) and a storage unit (e.g., a memory), the calculation unit analyzes a source code (e.g., a source code 291), acquires input value conditions and outputs (e.g., a set of “x>1” and “true”, “x≦1” and “false”) of the source code obtained by the analysis, and expands the test cases based on correction time information (e.g., a defect location candidate 293, a patch candidate 294) indicating candidates for correction of the input value conditions and outputs, and test addition criteria (e.g., a test addition criterion 295) for increasing test cases for testing the source code by a test code (e.g., a test code 292). In this way, by expanding the test cases based on the correction time information and the test addition criterion, it is possible to exclude incorrect patch candidates with high accuracy in patch candidate verification. Also, by expanding only the test cases based on the correction time information and the test addition criterion, it is possible to reduce the developer's effort in setting expected values.

[0045] Furthermore, the as-modified information includes candidates for the defective part of the source code (for example, "line 2" of source code 291 shown in defective part candidate 293), and the calculation unit generates a test case including an input that increases the coverage of the defective part (for example, test case "X=0" that satisfies "x≦1" in source code 291), and outputs test code including the test case. Therefore, in patch candidate verification, incorrect patch candidates can be excluded with high accuracy while taking into account the coverage of the defective part.

[0046] Furthermore, the modification time information includes a patch candidate for the source code (for example, patch candidate 294 obtained by modifying source code 291 with a predetermined modification pattern), and the calculation unit generates a test case including an input of a boundary value for the patch candidate (for example, boundary value "x=1" for patch candidate 294 with ID "1") and outputs test code including the test case. Therefore, in patch candidate verification, incorrect patch candidates can be excluded with high accuracy while taking into account the boundary value for the patch candidate.

[0047] Furthermore, the calculation unit generates a test input value (for example, the input value of "X=0"), acquires an expected value for the generated input value from outside the program correction support system (for example, acquires the expected value "true" input by the developer), generates a test case including the input value and the expected value, and generates a test code including the test case. Therefore, it is possible to generate a test case as expected (designed) by the developer, and as a result, it is possible to improve the quality of the generated test cases.

[0048] In this embodiment, the acquired boundary value is displayed on a display, which is an example of the output device 150, or the expected value is acquired as input from a keyboard, which is an example of the input device 140. However, if the input device 140 includes an audio input device such as a microphone, or the output device 150 includes an audio output device such as a speaker, it is possible to use another type of interface that can be recognized by the user, such as outputting the boundary value by voice through the device, or accepting input of the expected value.

[0049] The present invention is not limited to the above-described embodiments as they are, and in the implementation stage, the components can be modified and embodied within the scope of the gist of the present invention, or multiple components disclosed in the above-described embodiments can be appropriately combined. [Explanation of symbols]

[0050] 100 Program correction device 210 Defective part identification part 220 Patch Candidate Generation Unit 230 Test Generation Unit 240 Patch Candidate Verification Unit 290 Information storage section 291 Source Code 292 Test Code 293 Possible defective areas 294 Patch Candidates 295 Test Additional Standards 296 Post-generation test code 297 Correction results

Claims

1. A program correction support system that supports program correction by a computer having a calculation unit and a storage unit, The calculation unit is Analyzing a source code, and obtaining input value conditions and outputs of the source code obtained by the analysis; In a case where the test cases are expanded based on modification time information indicating candidates for modification of the input value conditions and the output and test addition criteria for increasing the number of test cases for testing the source code by the test code, The modification time information includes candidates for defect locations in the source code, the calculation unit generates a test case including an input that increases coverage of the defective portion, the input improving coverage of a process, a conditional branch, and an execution path executed in the test, and outputs a test code including the test case; A program correction support system comprising:

2. The modification time information includes a patch candidate for the source code, the calculation unit generates a test case including an input of a boundary value in the patch candidate, and outputs a test code including the test case.

2. The program correction support system according to claim 1.

3. the calculation unit generates a test input value, acquires an expected value for the generated input value from outside the program correction support system, generates a test case including the input value and the expected value, and generates a test code including the test case.

2. The program correction support system according to claim 1.

4. 1. A program modification support method performed in a program modification support system that supports program modification using a computer having a calculation unit and a storage unit, comprising: Analyzing a source code, and obtaining input value conditions and outputs of the source code obtained by the analysis; In a case where the test cases are expanded based on modification time information indicating candidates for modification of the input value conditions and the output and test addition criteria for increasing the number of test cases for testing the source code by the test code, The modification time information includes candidates for defect locations in the source code, the calculation unit generates a test case including an input that increases coverage of the defective portion, the input improving coverage of a process, a conditional branch, and an execution path executed in the test, and outputs a test code including the test case; 23. A program modification support method comprising:

5. The modification time information includes a patch candidate for the source code, generating a test case including an input of a boundary value in the patch candidate, and outputting a test code including the test case; 5. The program correction support method according to claim 4.

6. generating input values ​​for a test, acquiring expected values ​​for the generated input values ​​from outside the program correction support system, generating test cases including the input values ​​and the expected values, and generating test code including the test cases; 5. The program correction support method according to claim 4.

Images