Software test device and software test method

Abstract

A software test device (100) comprises a CPU (101), an auxiliary storage device (103) that stores the original of software to be tested, and a main storage device (102) that stores an implementation binary (220) copied from the original. When testing the copied implementation binary, for some or all of the functions called by some or all of the functions or features implemented as features to be tested in the copied implementation binary, the CPU dynamically changes a function return value, or a value passed via an argument, to a specified value, regardless of the processing content, without modifying the original implementation binary (210).

Description

Software testing device and software testing method

[0001] The present invention relates to a software testing device and a software testing method.

[0002] The importance of software testing is self-evident. Recently, in connection with keywords such as legacy code, refactoring, and test-driven development, the importance of fine-grained testing during the development phase has been increasing. This type of testing is generally performed by executing the entire system or a part of it (such as a function or class) against the execution procedures and input data used in the test, and verifying whether the output matches the expected result. Here, the combination of the execution procedures and input data used in the test is called a test case (Non-Patent Document 1). Testers intentionally control and implement test cases. Test cases are set up manually or automatically to exercise the functions of the system under test.

[0003] Various software testing methods have been proposed. For example, one method has been proposed that improves legacy code by examining software from the perspective of junctions, identifying clues for eliminating dependencies already present in the code. A junction is defined as a location where program behavior can be changed without directly editing that location (Non-Patent Document 2). Furthermore, for example, the device mapper function in Linux (registered trademark) can be used to simulate disk failures (Non-Patent Document 3).

[0004] "The ABCs of Testing You Need to Know," Nikkei XTECH, November 28, 2005, [online], [Retrieved July 11, 2024], Internet <URL: https: / / xtech.nikkei.com / it / article / COLUMN / 20051102 / 223936 / > Michael C. Feathers, "Legacy Code Improvement Guide," Chapter 4: Junction Model, UL Systems, Inc., Shoeisha, July 13, 2009. S. Takeuchi, "Emulating Disk I / O Faults with Device Mapper," December 2, 2023, [online], [Retrieved July 26, 2024], Internet <URL: https: / / speakerdeck.com / sat / ozhang-hai-noemiyuresiyon>

[0005] Depending on the function being tested, it may be difficult to prepare test cases for execution, resulting in inefficient testing. Specifically, first, it may be difficult to create input data that satisfies the conditions. For example, consider a test to verify error handling within the func function 901 shown in FIG. 10. In this case, it may be difficult to know the input data that will cause an error because the query is converted in a complex way depending on the mode value, or the behavior of the complex_func function 902 is complex. Furthermore, it may be difficult to create input data in cases where the internal implementation of the complex_func function 902 is unknown.

[0006] Second, it is difficult to prepare test cases when external factors are involved in the conditions. For example, it requires ingenuity to reproduce a condition such as "when a disk failure occurs" as a test case.

[0007] Third, it is difficult to prepare test cases when the function being tested involves a function (class) that communicates with the outside world. For example, the update_external_db function 903 shown in Figure 11 is itself a user-defined function, but it is possible that it may communicate internally with an external database (DB). It is also possible that an error will occur if the DB does not exist. As a result, it becomes necessary to prepare an external DB just to avoid the error.

[0008] In contrast, conventional techniques can derive input data conditions, for example, through symbolic execution. Symbolic execution is a technique for analyzing a program by treating variables in the program as symbols without values ​​and simulating all execution paths. For example, to determine the input data required for testing the program shown in FIG. 12 , the condition that the desired input satisfies can be determined by starting from the line containing the desired condition (z = 10) and tracing (inferring) the process backward. However, in practice, symbolic execution is limited to cases such as integer calculations shown in FIG. 12 . In other words, symbolic execution makes inference too complex when involving floating-point numbers or character strings such as those shown in FIG. 10 , and is also vulnerable to loops.

[0009] Other conventional techniques allow for the preparation of a fake implementation, known as a mock (dummy), which can be used to replace the original implementation. In this case, if the function related to the function being tested is an externally defined function, fake source and header files defining the function with the same name are prepared. Then, the files referenced by the function related to the function being tested during linking are replaced from the real ones with the fake ones by some means. This technique can change the behavior of the target function. Furthermore, if you want to execute all lines of a function except for specific lines, a method similar to mocking is known, in which a method with an identical signature is added or other modifications are made, effectively disabling the behavior of calling the method written on a specific line (see Non-Patent Document 2). However, as described in Non-Patent Document 2, mocking techniques cannot be used with source code that does not have a junction, so the conditions under which they can be applied are limited. Furthermore, mocking techniques require some kind of modification (such as code changes or environment variable rewrites) during compilation and execution, and these modifications must be managed separately from the test cases.

[0010] More advanced conventional techniques include simulating disturbances using special functions or libraries, or monkey patching. For example, special functions such as device mappers (Non-Patent Document 3) are not very common, so preparing test cases heavily depends on the skills of the tester, and similar problems arise when maintaining the tests. Monkey patches are patches that dynamically and ad hoc change the behavior of a program. Monkey patches are primarily used in scripting languages, but are difficult to implement in compiled languages ​​(especially simple languages ​​like C), and their implementation heavily depends on the skills of the tester.

[0011] The present invention has been made in view of the above circumstances, and an object of the present invention is to perform tests efficiently.

[0012] The software testing device of the present invention comprises a processing device, an auxiliary storage device for storing the original of the software to be tested, and a main storage device for storing an executable file copied from the original, and is characterized in that when testing the copied executable file, the processing device dynamically changes the return value of a function or the value passed via an argument to a specified value for some or all of the functions implemented as the function to be tested or some or all of the functions called by the function in the copied executable file, regardless of the content of the processing, without modifying the original executable file.

[0013] According to the present invention, tests can be performed efficiently.

[0014] FIG. 1 is a schematic configuration diagram of a software testing device according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of a user-defined function of a hook. FIG. 3 is a diagram showing a test execution example according to a first embodiment. FIG. 4 is a schematic diagram showing that a part of an implementation binary has been rewritten. FIG. 5 is a schematic diagram showing that a return value has been rewritten in a data area. FIG. 6 is a diagram showing an example of implementation code. FIG. 7 is a diagram showing an example of test code. FIG. 8 is a diagram showing a test execution example according to a second embodiment. FIG. 9 is a diagram showing a test execution example according to a third embodiment. FIG. 10 is a diagram showing a test execution example according to a fourth embodiment. FIG. 11 is a diagram showing an example of a function implemented as a function to be tested. FIG. 12 is a diagram showing another example of a function implemented as a function to be tested. FIG. 13 is a diagram showing an example of conventional test case inference.

[0015] A software testing device according to this embodiment will be described in detail below with reference to the drawings. The software testing device 100 shown in Fig. 1 is used during software development or maintenance management. The software testing device 100 is implemented by a computer and includes, for example, a CPU (Central Processing Unit) 101, a main memory device 102, an auxiliary memory device 103, an input / output I / F (Interface) 104, a communication I / F 105, and a media I / F 106.

[0016] The CPU (processing device) 101 operates based on a program stored in the main memory device 102 or the auxiliary memory device 103. This program can be distributed via a communication line or recorded on a recording medium 116 such as a CD-ROM. The main memory device 102 includes a read-only memory (ROM) and a random access memory (RAM), and stores a boot program executed by the CPU 101 when the computer starts up, as well as programs related to the computer's hardware. The auxiliary memory device 103 is, for example, a solid-state drive (SSD) or a hard disk drive (HDD). The auxiliary memory device 103 may be implemented inside the software testing device 100 or may be an external device.

[0017] The CPU 101 controls an input / output device 114 consisting of input units such as a mouse and a keyboard, and output units such as a display and a printer, via an input / output I / F 104. The communication I / F 105 receives data from other devices via a communication network 115 and outputs it to the CPU 101, and also transmits data generated by the CPU 101 to other devices via the communication network 115. The media I / F 106 reads programs or data stored in a recording medium 116 and outputs it to the CPU 101 via the main memory device 102.

[0018] The auxiliary storage device 103 stores an implementation binary (executable file) 210 of the software under test. The main storage device 102 stores an implementation binary 220. The implementation binary 210 is the original, and the implementation binary 220 is a copy of the implementation binary 210. When testing the implementation binary 220, the CPU 101 dynamically changes the function return value or the value passed via an argument to a specified value in the copy of the implementation binary 220, regardless of the content of the function's processing, without modifying the original implementation binary 210. Here, a function refers to some or all of the functions implemented as the function under test, or some or all of the functions called by the function. In the main storage device 102, the data area 150 stores the function return value, arguments, etc. The software testing device 100 not only changes the operation of the target software so that a specified function passes a specified value, but also can cancel this change.

[0019] In this embodiment, the CPU 101 changes the return value of a function or a value passed via an argument by using, for example, a hook function of an extended Berkeley Packet Filter (eBPF). In the following, as an example, the software testing device 100 will be described as changing the return value of a function of the implementation binary 220 by using the hook function of eBPF.

[0020] The software test device 100 uses eBPF to hook a function and rewrite the return value of the function. For example, assuming a test to verify error handling inside the func function 901 shown in FIG. 10 , the eBPF program hooks the complex_func function 902, rewrites the return value of the complex_func function 902 to true regardless of the actual result, and returns processing to the complex_func function 902. In the software test device 100, the implementation binary 220 dynamically attaches the eBPF program only when the test is being performed so as to hook the complex_func function 902.

[0021] 2 is a diagram showing an example of implementation code 200 that is the basis of implementation binary 220. A uret probe 301 shown in Fig. 2 indicates a hook for the return of a user function. The eBPF program hooks uret probe 301 and rewrites the return value.

[0022] In this embodiment, the change condition for changing the return value of a function to a specified value is written in the same format as the condition for a test case. A test case is a set of execution procedures and input data used in a test. The main memory device 102 stores the change condition together with the test case. Typically, software tests are written in separate files for specific sections, such as for each function to be tested, as in AFLplusplus (see Reference 1 below). Reference 1: "AFLplusplus", [online], [searched July 19, 2024], Internet <URL: https: / / github.com / AFLplusplus / AFLplusplus>

[0023] In the software testing device 100 of this embodiment, specifications related to function hooks are written together in files written in specific sections. This allows hooks to be treated as part of test cases, making it easier to write tests. In addition, because change conditions for changing the return value of a function to a specified value are stored together with the test case, this information is easier to manage.

[0024] Furthermore, the main memory device 102 collectively manages the following test-related contents T1 to T3: (T1) Functions whose return values ​​(or values ​​passed via arguments) are overwritten when a test is executed, and those values. (T2) Information indicating the state of some or all of the data referenced by a function being tested when that function is called. Here, the data referenced by the function refers to arguments, external database values, etc. (T3) Judgments made on some or all of the data affected by the calling of the function being tested, or the contents of the indices calculated from them.

[0025] [Software Testing Method] Next, a software testing method will be described. The software testing method includes a step of specifying a value to be obtained as a return value of a function, regardless of the content of the function's processing, before testing an implementation binary 220 (see FIG. 1) that is a copy of the original software to be tested (implementation binary 210: see FIG. 1). The software testing method includes a step of dynamically changing the operation of the copy implementation binary 220 so that the specified function passes a specified value, without modifying the original implementation binary 210, when testing the copy implementation binary 220.

[0026] In a software testing method, the process performed before a test is performed is performed manually, for example, by an operator. An overview of several embodiments of the software testing method will be described below. In a first embodiment, before a test is performed, an operator specifies a value to be obtained as a return value of a function in implementation code before compiling the software under test. In a second embodiment, before a test is performed, an operator specifies a value to be obtained as a return value of a function in test code that tests the software under test. In a third embodiment, before a test is performed, an operator interactively specifies a value to be obtained as a return value of a function using a dedicated tool for testing the software under test. In this third embodiment, the operator continues to interactively use the dedicated tool during the test to obtain test execution results. In a fourth embodiment, like the second embodiment, an operator specifies a value to be obtained as a return value of a function in test code before a test is performed without using eBPF.

[0027] First Example Next, the flow of test execution according to a first example using the software testing device 100 will be described with reference to Figure 3 (and also with reference to Figures 1 and 2 as appropriate). Note that in Figures 3 and 7 to 9, thick vertical lines indicate the execution state of the corresponding program. Hatching on vertical lines indicates a state in which rewriting of the return value is valid.

[0028] In the first embodiment, it is assumed that the eBPF program 250 is prepared separately and that the eBPF program 250 has already been loaded into the software test device 100. Furthermore, operations and confirmations are basically performed manually.

[0029] First, the implementation code 200 is compiled by a compiler (not shown) (step S101) to generate an implementation binary 220. The implementation code 200 is source code to be implemented on an actual device such as a PC that will perform the target test. For example, in the implementation code shown in FIG. 2, the operator specifies a value to be obtained as a return value of a function.

[0030] In this example, the implementation code 200 is compiled using a compiler on a PC different from the software testing device 100, and the compiled code is stored as an original implementation binary 210 in the external auxiliary storage device 103. The external auxiliary storage device 103 is then connected to the software testing device 100, and the implementation binary 210 is read from the auxiliary storage device 103 and copied onto the main storage device 102 to become the implementation binary 220.

[0031] The implementation code 200 may be converted into the implementation binary 220 as follows: For example, the implementation code 200 is compiled by a compiler on the software testing device 100 to generate the original implementation binary 210, which is then stored in the internal auxiliary storage device 103. Meanwhile, the CPU 101 reads the implementation binary 210 from the auxiliary storage device 103 and copies it onto the main storage device 102 to generate the implementation binary 220.

[0032] Next, the operator 230 of the software testing device 100 performs an attach operation of the eBPF program 250 (step S201). As a result, the eBPF program 250 is attached to the implementation binary 220 (step S203). Then, the implementation binary 220 enters a state in which rewriting of the return value is valid. Note that at this point, the implementation binary 220 is not yet in an execution state.

[0033] Next, the operator 230 executes the implementation binary 220 (step S205). As a result, the implementation binary 220 enters an execution state. The CPU 101 executes, for example, a binary string (machine language) corresponding to the implementation code 200 shown in FIG. 2. Then, when the CPU 101 executes the binary string corresponding to "return -1;" shown in FIG. 2 in the implementation binary 220, the eBPF program 250 detects this execution, and processing transfers to the eBPF program 250 (step S207: Return). That is, the function (uret probe 301) of the implementation code 200 is hooked.

[0034] At this time, in order to implement the hook, a portion of the implemented binary 220 on the main memory 102 to which the eBPF program 250 is attached is rewritten. In Fig. 4, a rewritten area 221 of a portion of the implemented binary 220 on the main memory 102 is indicated by dots. At this time, as shown in Fig. 4, the original implemented binary 210 stored in the auxiliary memory 103 is not rewritten. Furthermore, the data area 150 in the main memory 102 that stores return values ​​and arguments is not rewritten.

[0035] The eBPF program 250 then rewrites the return value and returns the processing to the implementation binary 220 (step S209). The rewrites here are the return value and arguments themselves in the data area 150 of the main memory device 102. In FIG. 5, a rewrite area 151, which is a part of the data area 150 of the main memory device 102, is shown as a grid. As shown in FIG. 5, the rewrite area 221, which is a part of the implementation binary 220 in the main memory device 102, has already been rewritten due to the hook. Furthermore, the original implementation binary 210 stored in the auxiliary memory device 103 has not been rewritten.

[0036] The CPU 101 then ends the execution of the implementation binary 220 (step S211). At this point, the implementation binary 220 is still in a state where the return value can be rewritten. The operator 230 then performs an operation to detach the eBPF program 250 (step S213). As a result, the eBPF program 250 is detached from the implementation binary 220 (step S215). That is, the implementation binary 220 enters a state where rewriting is disabled. The operator 230 then performs an operation to check the result of the execution of the implementation binary 220 (step S217). As a result, the operator 230 obtains the execution result by displaying the result of the execution of the implementation binary 220 on an output device such as a display. This completes the test (step S219).

[0037] Second Example Next, a test execution flow according to a second example using the software testing device 100 will be described. In the second example, CUnit (see Reference 2 below) is extended using eBPF. CUnit is built as a static library and linked to the user's test code. It uses a simple framework for building test structures and provides a rich set of assertions for testing common data types. Reference 2: "Cunit A Unit Testing Framework for C," [online], [searched July 11, 2024], Internet <URL: https: / / cunit.sourceforge.net / index.html>

[0038] In the second embodiment, it is assumed that the implementation code 200 shown in Fig. 6A and the test code 260 shown in Fig. 6B are prepared. In addition to the original functions of CUnit, the test code 260 shown in Fig. 6B defines a unique return value rewrite macro 302 (CU_HOOK). In this test code 260, in addition to the contents specified during normal testing (call arguments and result determination), argument rewrites are also described.

[0039] The second embodiment will be described below with reference to FIGS. 6A, 6B, and 7 (and also with reference to FIGS. 1, 4, and 5 as appropriate). Note that a description of processes similar to those in the first embodiment will be omitted where appropriate. First, a compiler (not shown) compiles implementation code 200 (see FIG. 6A) to generate implementation binary 220 (step S301). Meanwhile, a compiler (not shown) compiles test code 260 (see FIG. 6B) to generate test binary 270 (step S303). The operator specifies a value obtained as a function return value in test code 260 shown in FIG. 6B. The test binary 270 is formed by statically linking the implementation binary 220 to a file generated from test code 260 (step S305).

[0040] Next, the operator 230 of the software testing device 100 executes the test binary 270 (step S401). This puts the test binary 270 into an execution state. The CPU 101 executes the test binary 270 to first generate the eBPF code 240 (step S403). Here, the eBPF code 240 is source code originally written in a language such as C. This eBPF code 240 is compiled into eBPF bytecode using LLVM or the like. The bytecode is binary data but not machine language. The eBPF bytecode is loaded into the kernel and converted into the eBPF program 250, which is executable machine language (step S405).

[0041] Next, the test binary 270 is executed to attach the eBPF program 250 to itself (the test binary 270) (step S407). As a result, the eBPF program 250 is attached to the test binary 270 (step S409). The test binary 270 then enters a state in which rewriting of the return value is valid. At this point, the test binary 270 is already in an execution state.

[0042] The CPU 101 executes the binary string corresponding to "CU_HOOK (complex_func, true);" in the test code 260 shown in Fig. 6B. Then, at the timing when the binary string corresponding to "return -1;" shown in Fig. 6A is executed in the implementation binary 220 statically linked to the test binary 270, the eBPF program 250 detects this execution, and processing transfers to the eBPF program 250 (step S411: Return). That is, the complex_func function in the implementation code 200 is hooked.

[0043] At this time, the implementation binary 220, which is statically linked to the test binary 270 to which the eBPF program 250 is attached, has a portion of the implementation binary 220 on the main memory 102 rewritten to implement the hook (see FIG. 4). Then, the eBPF program 250 rewrites the return value and returns the processing to the test binary 270 (step S413). Here, the return value and the argument themselves on the data area 150 of the main memory 102 are rewritten (see FIG. 5).

[0044] In this way, the test binary 270 is in a state where the operator 300 can determine the result of execution (step S415). Thereafter, execution of the test binary 270 detaches the eBPF program 250 from itself (the test binary 270) (step S417). As a result, the eBPF program 250 is detached from the test binary 270 (step S419). Then, the test binary 270 enters a state where rewriting of the return value is disabled. Then, execution of the test binary 270 ends (step S421). This completes the test (step S423).

[0045] Third Embodiment Next, a test execution flow according to a third embodiment using the software testing device 100 will be described with reference to FIG. 8 (and also with reference to FIGS. 1, 4, and 5 as appropriate). The description of processes similar to those in the first embodiment will be omitted as appropriate. In the third embodiment, an operator uses a dedicated tool 280 that can be operated interactively, like a debugger, to specify the return value of a function while the tool 280 is running.

[0046] First, the implementation code 200 is compiled by a compiler (not shown) (step S501) to generate the implementation binary 220. Meanwhile, the tool 280 is in an execution state. Next, the operator 230 specifies a test target via the tool 280 (step S601). The tool 280 executes the target implementation binary 220 (step S603). As a result, the implementation binary 220 enters an execution state.

[0047] Next, the operator 230 specifies the function and the return value to the tool 280 (step S605). Then, eBPF code is generated via the tool 280 (step S607). This eBPF code is compiled into eBPF bytecode, which is loaded into the kernel and converted into the eBPF program 250 (step S609).

[0048] Next, the implementation binary 220 is executed to attach the eBPF program 250 to itself (the implementation binary 220). As a result, the eBPF program 250 is attached to the implementation binary 220 (step S611). Then, the implementation binary 220 enters a state in which rewriting of the return value is valid.

[0049] Next, the operator 230 performs an operation to execute the function to be tested in the implementation binary 220 via the tool 280 (step S613). As a result, the CPU 101 executes a binary string (machine language) corresponding to the implementation code 200 shown in FIG. 2. Then, when the CPU 101 executes the binary string corresponding to "return -1;" shown in FIG. 2 in the implementation binary 220, the eBPF program 250 detects this execution, and processing transfers to the eBPF program 250 (step S615: Return). That is, the function of the implementation code 200 is hooked.

[0050] At this time, in order to implement the hook, the implemented binary 220 to which the eBPF program 250 is attached has a portion of the implemented binary 220 on the main memory 102 rewritten (see FIG. 4). Then, the eBPF program 250 rewrites the return value and returns the processing to the implemented binary 220 (step S617). Here, the return value and the argument themselves on the data area 150 of the main memory 102 are rewritten (see FIG. 5).

[0051] The operator 230 obtains the execution result of the implementation binary 220 by performing an operation to check the execution result of the implementation binary 220 via the tool 280 (step S619). Then, the operator 230 performs an operation to cancel the execution of the eBPF program 250 and the implementation binary 220 via the tool 280 (step S621). As a result, the eBPF program 250 is detached from the test binary 270 (step S623). Then, the implementation binary 220 enters a state in which rewriting of the return value is disabled. Then, the implementation binary 220 ends execution (step S625). This completes the test.

[0052] Fourth Embodiment The means by which the software testing device 100 dynamically changes the return value of a function to a specified value in the implementation binary 220 is not limited to the hook function of eBPF. In the fourth embodiment, the software testing device 100 uses Ptrace. In the fourth embodiment, the following processes are roughly executed in sequence: (1) starting a test process, (2) another process captures the test process using Ptrace, and (3) monitoring the operation and rewriting the return value when a specific function is called and returns.

[0053] The flow of test execution according to the fourth embodiment using the software testing device 100 will be described below with reference to FIG. 9 (and also with reference to FIGS. 1, 4, and 7 as appropriate). Note that explanations of processes similar to those in the second embodiment will be omitted where appropriate. In FIG. 9, the implementation code 200, implementation binary 220, test code 260, and test binary 270 are the same as those in the second embodiment. Pre-processing steps S301, S303, and S305 shown in FIG. 9 are the same as steps S301, S303, and S305 shown in FIG. 7, respectively.

[0054] First, when the operator executes the test binary 270 (a test process), the test binary 270 enters an execution state (step S801). Meanwhile, the Ptrace process 290 (another process) is also in an execution state. Then, the test binary 270 sends test-related information to the Ptrace process 290 (step S803). Here, the test-related information indicates the contents T1 (functions whose return values ​​are overwritten and their values) managed in the main memory 102, etc. As a result, the Ptrace process 290 captures the test process based on the test-related information (step S805).

[0055] Then, the Ptrace process 290 monitors the specific function call (step S807). The specific function call is expressed by a binary string equivalent to, for example, "CU_HOOK (complex_func, true);" in the test code 260 shown in FIG. 6B. When monitoring of the specific function call begins, the test binary 270 enters a state in which rewriting of the return value is enabled.

[0056] Thereafter, the CPU 101 executes the test binary 270, and when it executes the binary string corresponding to "CU_HOOK (complex_func, true);" in the test code 260 shown in FIG. 6B, it calls a specific function (complex_func function) in the implementation binary 220 (step S809). Then, in the implementation binary 220 statically linked to the test binary 270, at the timing when the binary string corresponding to "return -1;" shown in FIG. 6A is executed, the Ptrace process 290 detects this execution, and processing is transferred to the Ptrace process 290 (step S811: Return). That is, the complex_func function in the implementation code 200 is hooked.

[0057] At this time, the implementation binary 220, which is statically linked to the test binary 270 captured by the Ptrace process 290, has a portion of the implementation binary 220 on the main memory 102 rewritten to implement the hook (see FIG. 4). Then, the Ptrace process 290 rewrites the return value and returns the processing to the test binary 270 (step S813). Here, the return value and the argument themselves on the data area 150 of the main memory 102 are rewritten (see FIG. 5).

[0058] After that, when the Ptrace process 290 releases the capture of the test process, the test binary 270 enters a state in which rewriting of the return value is disabled. Thus, the test binary 270 enters a state in which the operator can determine the execution result. Then, execution of the test binary 270 ends. This completes the test (step S815).

[0059] [Effects] As described above, the software testing device 100 includes the CPU 101, the auxiliary storage device 103 that stores the original 210 of the software to be tested, and the main storage device 102 that stores the implementation binary 220 that is a copy of the original 210. When testing the copy implementation binary 220, the CPU 101 is characterized in that, without modifying the original implementation binary 210, for some or all of the functions implemented as functions to be tested in the copy implementation binary 220 or some or all of the functions called by the functions, dynamically changes the return values ​​of the functions or values ​​passed via arguments to specified values, regardless of the content of the processing.

[0060] By doing this, the software testing device 100 dynamically changes the return value of a function or a value passed via an argument to a specified value during testing. This allows the software testing device 100 to execute testing even when it is difficult to prepare test cases. Therefore, the software testing device 100 can perform testing efficiently. Furthermore, the software testing device 100 dynamically changes the return value of a specified function to a specified value only during testing. Therefore, unlike mocking, which is a conventional technology, the software testing device 100 does not require manual changes, such as rewriting environment variables, during testing. Another advantage of the software testing device 100 is that, unlike mocking, there is no need to manage changes during testing using a script separate from the test cases.

[0061] In the software testing device 100, the change conditions for changing the return value of a function or the value passed via an argument to a specified value are written in the same format as the conditions for a test case, which is a set of execution procedures and input data used in the test, and the main memory device 102 stores the change conditions together with the test case.

[0062] In this way, the software testing device 100 can simplify the management of test cases because the change conditions for function return values ​​(which function's return value is changed to what) are written in the same format as other conditions related to the test cases. Therefore, the software testing device 100 can also eliminate the need to manage configurations by other means when conducting tests.

[0063] The software test device 100 is characterized in that the CPU 101 uses a hook function of the extended Berkeley Packet Filter (eBPF) to change the return value of a function or a value passed via an argument.

[0064] In this way, the software testing device 100 uses the hook function of eBPF to change the return value of any function (whether in kernel or user space) to a specified value when testing software used for development or maintenance management. Therefore, the software testing device 100 makes it possible to execute tests even under conditions that are difficult to execute using existing testing methods.

[0065] The software testing method is characterized by comprising the steps of: before conducting a test on an implementation binary 220 that is a copy of an original 210 of the software to be tested, specifying a value to be obtained as a return value of a function or a value to be passed via an argument for some or all of the functions implemented as the function to be tested or some or all of the functions called by the functions, regardless of the content of the processing; and when conducting a test on the copy implementation binary 220, dynamically changing the operation of the copy implementation binary 220 so that the specified function passes the specified value without modifying the original implementation binary 210.

[0066] By doing so, the software testing method allows the operator to specify the value to be obtained as the desired function return value, etc., before conducting the test. Existing testing methods have problems such as being impractical, being limited in the conditions they can be applied to, and being dependent on the skill of the tester. In contrast, the software testing method allows for various modes of specifying functions and return values, etc., and makes it possible to conduct tests even under conditions that are difficult to conduct using existing testing methods. Therefore, the software testing method allows for efficient testing.

[0067] The present invention is not limited to the above-described embodiments, and many modifications can be made by a person skilled in the art within the technical spirit of the present invention. For example, the specified functions are not limited to some of the functions implemented as the function to be tested, but can be all of the functions. The specified functions can also be some or all of the functions called by the function to be tested. The specified values ​​are not limited to values ​​passed via the return value of the function, but can also be values ​​passed via arguments.

[0068] 100 Software test device 101 CPU 102 Main memory device 103 Auxiliary memory device 104 Input / output I / F 105 Communication I / F 106 Media I / F 150 Data area 151 Rewrite area 200 Implementation code 210 Implementation binary (original) 220 Implementation binary (copy) 221 Rewrite area 230 Operator 240 eBPF code 250 eBPF program 260 Test code 270 Test binary 280 Tool 290 Process for ptrace

Claims

1. A software testing device comprising: a processing device; an auxiliary storage device for storing the original of the software to be tested; and a main storage device for storing an executable file to which the original has been copied, wherein the processing device, when testing the copied executable file, dynamically changes the return value of a function or the value passed via an argument to a specified value for some or all of the functions implemented as the function to be tested or some or all of the functions called by the function in the copied executable file, regardless of the content of the processing, without modifying the original executable file.

2. The software testing device described in claim 1, characterized in that the change conditions for changing the return value of a function or the value passed via an argument to a specified value are written in the same format as the conditions for a test case, which is a set of execution procedures and input data used in testing, and the main memory stores the change conditions together with the test case.

3. A software testing device according to claim 1 or 2, characterized in that the processing device uses a hook function of eBPF (extended Berkeley Packet Filter) to change the return value of a function or a value passed via an argument.

4. A software testing method characterized by comprising the steps of: before conducting a test on an executable file copied from the original software to be tested, specifying a value to be obtained as a return value of a function or a value to be passed via an argument for some or all of the functions implemented as the function to be tested or some or all of the functions called by the function, regardless of the content of the processing; and when conducting a test on the copied executable file, dynamically changing the operation of the specified function in the copied executable file so that it passes the specified value, without modifying the original executable file.

Images