Analysis function providing device, analysis function providing method, and analysis function providing program

Abstract

An analysis function addition device (10) comprises a virtual machine analysis unit (121) that analyzes the VM of a script engine, an instruction set architecture analysis unit (122) that analyzes an instruction set architecture, which is a system of instructions of the VM, to collect VM instructions and determine the content of the collected VM instructions, and a calculation unit (123) that uses the architecture information acquired by the virtual machine analysis unit (121) and the instruction set architecture analysis unit (122) to construct a first control flow graph representing an entire path executed exhaustively through multipath execution, and a second control flow graph representing paths executed during execution of a test subject, and uses the first control flow graph and the second control flow graph to calculate a code coverage of a script executed on the script engine.

Description

[Technical Field]

[0001] The present invention relates to an analytical function providing device, an analytical function providing method, and an analytical function providing program. [Background technology]

[0002] Software testing is a technique for discovering latent defects in software. One type of software testing is called dynamic testing, which performs testing by actually providing input values ​​to the target program and observing its behavior.

[0003] One measure of whether dynamic testing has been performed comprehensively is code coverage (also called code coverage rate). Code coverage evaluates the percentage of the code in the program being tested that has been tested.

[0004] Some dynamic tests use code coverage as an indicator to evaluate the progress of testing and to plan tests.

[0005] One example is fuzzing, a dynamic testing technique for discovering potential vulnerabilities in software. Fuzzing involves repeatedly generating or mutating input values, running the target program, observing the program's state, and searching for inputs that cause problems such as crashes, thereby discovering vulnerabilities.

[0006] In this case, how the input values ​​are generated and mutated is an important factor in determining the efficiency of vulnerability discovery. For example, if there is a path that can only be executed within a specific range of input values, and a vulnerability exists there, it will take a long time to discover the vulnerability unless the input values ​​required to follow that path can be found efficiently.

[0007] The most primitive method of fuzzing is to randomly generate input values, and black-box testing, in which the only observation of a program is whether or not a crash occurs, has been adopted.

[0008] However, such a method cannot efficiently detect vulnerabilities in the above-mentioned cases.

[0009] For this reason, a technique called gray box fuzzing is used. Unlike black box fuzzing, gray box fuzzing also observes the execution paths, which are the internal state of the program, when it is executed. Gray box fuzzing then calculates code coverage for execution paths that have already been tested, and prioritizes input values ​​used as seeds for generation and mutation so that this value increases.

[0010] This gray-box fuzzing allows for efficient vulnerability detection across a wider range of programs.

[0011] To implement this gray-box fuzzing, it is necessary to observe the execution path and measure code coverage.

[0012] As mentioned above, measuring code coverage plays an important role in software testing, not only for executable binary programs but also for scripts.

[0013] Generally, code coverage is measured by instrumenting source code, binaries, bytecode, etc., adding code that records which parts of the code have been executed.

[0014] Currently, scripts are inputted, converted into bytecode via an intermediate representation, and then executed by a virtual machine (VM). Therefore, there are two methods for measuring code coverage: instrumentation for scripts and instrumentation for bytecodes.

[0015] In the technology described in Non-Patent Document 1, in order to measure the code coverage of JavaScript (registered trademark), static instrumentation is performed on the script so that which parts have been executed is reported. This makes it possible to calculate the code coverage by analyzing the reported execution trace.

[0016] The technology described in Non-Patent Document 2 investigates existing code coverage measurement tools and shows that instrumentation of bytecode, along with the above-mentioned instrumentation of scripts, is a method adopted by many tools. The methods described in Non-Patent Documents 3 to 5 make it possible to calculate code coverage for scripts in the scripting languages ​​Lua, Python, and Ruby, respectively. [Prior art documents] [Non-patent literature]

[0017] [Non-Patent Document 1] M. Fard and A. Mesbah, “JSNOSE: Detecting JavaScript Code Smells”, In Proceedings of the 2013 IEEE 13th International Working Conference on Source Code Analysis and Manipulation, pp. 116-125, 2013. [Non-patent document 2] F. Horvath, T. Gergely, A. Beszedes, D. Tengeri, G. Balogh, and T. Gyimothy, “Code coverage differences of Java bytecode and source code instrumentation tools”, Software Quality Journal, Vol. 27, pp. 79-123, 2019. [Non-patent document 3] “LuaCov”, [online], [Retrieved March 9, 2022], Internet<URL:https: / / keplerproject.github.io / luacov / > [Non-patent document 4] “coverage.py”, [online], [Retrieved March 9, 2022], Internet<URL:https: / / coverage.readthedocs.io / en / 6.3.2 / > [Non-patent document 5] “Ruby Coverage module”, [online], [Retrieved March 9, 2022], Internet<URL:https: / / docs.ruby-lang.org / en / 3.1 / Coverage.html> Summary of the Invention [Problem to be solved by the invention]

[0018] However, the technology described in Non-Patent Document 1 has the problem that it is not easy to implement complete instrumentation and it may be difficult to deal with code that is dynamically generated at runtime, so it is not necessarily possible to comprehensively realize all instrumentation.

[0019] Furthermore, the technology described in Non-Patent Document 2 requires information about the VM instructions that make up the bytecode and the VM that executes it in order to instrument the bytecode, which poses the problem that it cannot be applied to script engines whose internal VM specifications are unknown.

[0020] Furthermore, the methods described in Non-Patent Documents 3 to 5 use special functions provided by the script engine for debugging purposes or functions for calculating code coverage that are built into the script engine to calculate code coverage, and therefore have the problem that they cannot be applied to script engines other than those that can use such functions.

[0021] Measuring the code coverage of a script generally requires the use of support functions such as a debugger provided by the script engine. This is because the internal specifications of the VM in the script engine that controls the execution of the script are often not publicly available, making it difficult to observe the execution path and analyze the bytecode required for measuring code coverage without support functions.

[0022] However, if such support functions are not provided, it is necessary to reverse engineer the VM to reveal its internal specifications, independently observe execution paths and analyze bytecodes, and obtain information to calculate code coverage.

[0023] It is not realistic to manually and individually analyze, design, and implement this for each script engine, in view of the amount of work involved.

[0024] The present invention has been made in consideration of the above, and aims to provide an analysis function-imparting device, an analysis function-imparting method, and an analysis function-imparting program that impart to a script engine the function of calculating the code coverage of scripts executed on the script engine. [Means for solving the problem]

[0025] In order to solve the above-mentioned problems and achieve the object, the analysis function-imparting device of the present invention is characterized by having a first analysis unit that analyzes the virtual machine of a script engine, a second analysis unit that analyzes the instruction set architecture, which is the instruction system of the virtual machine, to collect virtual machine instructions and determine the instruction contents of the collected virtual machine instructions, and a calculation unit that constructs, based on the architecture information obtained by the first analysis unit and the second analysis unit, a first control flow graph that shows the entire path comprehensively executed by multi-pass execution and a second control flow graph that shows the path executed in the execution of the test target, and calculates the code coverage of the script executed on the script engine based on the first control flow graph and the second control flow graph. [Effects of the Invention]

[0026] According to the present invention, it is possible to realize an analysis function providing method and an analysis function providing program that provide a script engine with the function of calculating the code coverage of a script executed on the script engine. [Brief explanation of the drawings]

[0027] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a script engine. [Figure 2] FIG. 2 is a diagram showing pseudocode of a VM included in the script engine. [Figure 3] FIG. 3 is a diagram illustrating an example of the configuration of the analysis function imparting device according to the embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a test script used to detect a virtual program counter (VPC). [Figure 5] FIG. 5 is a diagram showing an example of a test script used for detecting a branch VM instruction. [Figure 6] FIG. 6 is a diagram illustrating an example of an execution trace. [Figure 7] FIG. 7 is a diagram illustrating an example of a VM execution trace. [Figure 8] FIG. 8 is a diagram illustrating the processing of the VM instruction boundary detection unit. [Figure 9] FIG. 9 is a diagram illustrating the processing of the virtual program counter detection unit. [Figure 10] FIG. 10 is a diagram illustrating the processing of the dispatcher detection unit. [Figure 11] FIG. 11 is a diagram illustrating the processing of the code cache detection unit. [Figure 12] FIG. 12 is a diagram illustrating the processing of the VM instruction determination unit. [Figure 13] FIG. 13 is a diagram illustrating the processing of the VM branch trace construction unit. [Figure 14]FIG. 14 is a diagram illustrating the control flow graph construction unit. [Figure 15] FIG. 15 is a flowchart illustrating a processing procedure for the analysis function providing process according to the embodiment. [Figure 16] FIG. 16 is a flowchart illustrating the processing procedure of the execution trace acquisition processing shown in FIG. [Figure 17] FIG. 17 is a flowchart illustrating the procedure of the VM instruction boundary detection process shown in FIG. [Figure 18] FIG. 18 is a flowchart illustrating the processing procedure of the virtual program counter detection processing shown in FIG. [Figure 19] FIG. 19 is a diagram illustrating the dispatcher detection process shown in FIG. [Figure 20] FIG. 20 is a flowchart illustrating the processing procedure of the conditional branch flag detection processing shown in FIG. [Figure 21] FIG. 21 is a flowchart illustrating the processing procedure of the code cache detection processing shown in FIG. [Figure 22] FIG. 22 is a flowchart illustrating the procedure of the VM execution trace acquisition process shown in FIG. [Figure 23] FIG. 23 is a flowchart illustrating the processing procedure of the VM instruction collection processing shown in FIG. [Figure 24] FIG. 24 is a flowchart illustrating the processing procedure of the VM instruction determination processing shown in FIG. [Figure 25] FIG. 25 is a flowchart showing the processing procedure of the multi-path execution processing shown in FIG. [Figure 26] FIG. 26 is a flowchart illustrating the processing procedure of the VM branch trace construction processing shown in FIG. [Figure 27] FIG. 27 is a flowchart showing the processing procedure of the control flow graph construction processing shown in FIG. [Figure 28] FIG. 28 is a flowchart showing the processing procedure of the execution processing shown in FIG. [Figure 29]FIG. 29 is a flowchart illustrating the processing procedure of the code coverage calculation processing shown in FIG. [Figure 30] FIG. 30 is a diagram illustrating an example of a computer that implements an analysis function-imparting device by executing a program. DETAILED DESCRIPTION OF THE INVENTION

[0028] Hereinafter, embodiments of an analysis function providing device, an analysis function providing method, and an analysis function providing program according to the present application will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described below.

[0029] [Embodiment Mode] The analysis function providing device according to the embodiment provides a script engine with a function for calculating the code coverage of a script executed on the script engine.

[0030] An analysis function-imparting device according to an embodiment executes a test script while monitoring the binary of a script engine, and acquires a branch trace and a memory access trace as an execution trace. The analysis function-imparting device analyzes a VM based on the execution trace, and acquires, as architecture information, a VM instruction boundary, a virtual program counter (VPC), a dispatcher, a conditional branch flag, and a code cache in which executed VM instructions are stored.

[0031] The analysis function-adding device executes the test script while monitoring the VPC and the dispatcher, and acquires a VM execution trace. The analysis function-adding device analyzes the VM execution trace to collect VM instructions, determine the contents of the VM instructions, and acquire information about the instruction set architecture.

[0032] Next, based on the acquired architecture information, the analysis function-imparting device constructs a first control flow graph showing the entire path comprehensively executed in multi-pass execution and a second control flow graph showing the path executed in the execution of the test target, and calculates the code coverage of the script executed on the script engine based on these. Code coverage is the ratio of the path executed in the execution of the test target to the entire path. In this way, the analysis function-imparting device imparts to the script engine the function of calculating the code coverage of the script executed on the script engine.

[0033] The configuration and operation of a typical script engine will be described with reference to Figures 1 and 2. Figure 1 is a diagram illustrating an example of the configuration of a script engine. As shown in Figure 1, the script engine 1 has a bytecode compiler 2 and a virtual machine (VM) 3. The bytecode compiler 2 also has a syntax analysis unit 4 and a bytecode generation unit 5. The VM 3 also has a code cache unit 6, a fetch unit 7, a decode unit 8, and an execution unit 9. The fetch unit 7, the decode unit 8, and the execution unit 9 are executed repeatedly and are called an interpreter loop. The script engine 1 then accepts a script as input.

[0034] The syntax analysis unit 4 receives a script as input, performs lexical analysis and syntax analysis, generates an Abstract Syntax Tree (AST), and outputs it to the bytecode generation unit 5. The bytecode generation unit 5 receives the AST as input, converts it into bytecode, and stores it in the code cache unit 6.

[0035] The fetch unit 7 fetches the VM opcode from the code cache unit 6 and outputs it to the decode unit 8. Here, the VM opcode refers to the opcode portion of the VM instruction. The decode unit 8 receives the VM opcode as input, interprets it using a decoder dispatcher, and dispatches it to the corresponding program. The execution unit 9 executes the program corresponding to the VM instruction. The contents written in the script are executed by executing the VM instructions one after another through repeated interpretation loops.

[0036] The functions of the components of the script engine will be described with reference to Figure 2. Figure 2 is a diagram showing pseudocode of a VM included in the script engine. As shown in Figure 2, the pseudocode first initializes a VPC (line 1). In the pseudocode, a while loop is an interpreter loop (line 2). In the pseudocode, the VM opcode pointed to by the VPC is obtained from the code cache (line 3), and is decoded and dispatched using a Switch statement (lines 4, 5, and 7). Then, in the pseudocode, the program corresponding to the VM opcode that was dispatched is executed (lines 6 and 8).

[0037] Furthermore, a branch VM instruction is a VM instruction that causes a branch to occur within a script, and a conditional branch flag is an area that holds a flag indicating whether or not a branch is taken at the time of a conditional branch.

[0038] [Configuration of the analysis function-providing device] Next, the configuration of the analysis function-imparting device 10 according to the embodiment will be specifically described with reference to Fig. 3. Fig. 3 is a diagram illustrating an example of the configuration of the analysis function-imparting device according to the embodiment.

[0039] 3, the analysis function-imparting device 10 includes an input unit 11, a control unit 12, a storage unit 13, and an output unit 14. The analysis function-imparting device 10 receives inputs of a test script, a script engine binary, and a script to be analyzed.

[0040] The input unit 11 is composed of input devices such as a keyboard and a mouse, and receives input of information from outside and inputs it to the control unit 12. The input unit 11 also has a communication interface for sending and receiving various information to and from other devices connected via a wired connection or a network, etc., and receives input of information sent from other devices. The input unit 11 receives input of test scripts, script engine binaries, and scripts to be analyzed, and outputs them to the control unit 12.

[0041] A test script is a script that is input when dynamically analyzing a script engine to obtain an execution trace and a VM execution trace. Details of test scripts will be described later. A script engine binary is an executable file that constitutes a script engine. A script engine binary may be composed of multiple executable files. A script to be analyzed is a script to be analyzed.

[0042] The control unit 12 has an internal memory for storing programs that define various processing procedures and required data, and executes various processes using these. For example, the control unit 12 is an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit 12 has a virtual machine analysis unit 121 (first analysis unit), an instruction set architecture analysis unit 122 (second analysis unit), and a calculation unit 123.

[0043] The virtual machine analysis unit 121 analyzes the VM of the script engine. The virtual machine analysis unit 121 acquires multiple execution traces by changing runtime conditions, analyzes the multiple execution traces using differential execution analysis, and acquires VPCs and conditional branch flags. The virtual machine analysis unit 121 also analyzes the script engine binary to acquire VM instruction boundaries and dispatchers. The virtual machine analysis unit 121 detects a code cache from the VM execution trace. The code cache stores VM instructions to be executed.

[0044] The virtual machine analysis unit 121 has an execution trace acquisition unit 1211 (first acquisition unit), a VM instruction boundary detection unit 1212 (first detection unit), a virtual program counter detection unit 1213 (second detection unit), a dispatcher detection unit 1214 (third detection unit), a conditional branch flag detection unit 1215 (fourth detection unit), and a code cache detection unit 1216.

[0045] The execution trace acquisition unit 1211 receives a test script and a script engine binary as input. The execution trace acquisition unit 1211 acquires an execution trace by executing the test script while monitoring the execution of the script engine binary.

[0046] An execution trace is composed of a branch trace and a memory access trace. A branch trace records the type of branch instruction at the time of execution, the branch source address, and the branch destination address. A memory access trace records the type of memory operation and the memory address of the operation target. It is known that branch traces and memory access traces can be acquired by instruction hooks. The execution trace acquired by the execution trace acquisition unit 1211 is stored in the execution trace DB 131.

[0047] The VM instruction boundary detection unit 1212 clusters the execution trace to detect the boundaries of each VM instruction. The VM instruction boundary detection unit 1212 clusters the execution trace and detects clusters with a threshold or more of execution count as VM instructions. Clustering detects consecutive code regions that are executed multiple times. For example, this may be done by grouping executed instructions that are close in code distance, by searching for common subsequences of executed code blocks, or by other methods. The analysis function providing device 10 detects the start and end points of consecutive instruction sequences that make up the detected VM instruction as boundaries. The VM instruction boundaries detected here are used in VPC detection and dispatcher detection.

[0048] The virtual program counter detection unit 1213 extracts and analyzes the execution trace for the first test script stored in the execution trace DB 131 to detect a VPC. The virtual program counter detection unit 1213 detects a VPC by analyzing multiple execution traces using differential execution analysis focusing on the number of times memory is read and the boundaries of each VM instruction detected by the VM instruction boundary detection unit 1212. The virtual program counter detection unit 1213 utilizes the fact that a read into memory that holds a VPC always occurs after the execution of each VM instruction, and detects the VPC by finding the destination of this read.

[0049] For this reason, the virtual program counter detection unit 1213 uses differential execution analysis, focusing on the number of memory reads, to detect VPCs. The virtual program counter detection unit 1213 compares execution traces of multiple test scripts acquired using the test scripts, and finds memories whose memory read counts change in proportion to both the number of repetitions and the number of repeated statements. The virtual program counter detection unit 1213 then refers to the boundaries of each VM instruction detected by the VM instruction boundary detection unit 1212, and narrows down the memory values ​​read to those whose values ​​always point to the start points of VM instructions. The virtual program counter detection unit 1213 detects these memories as VPCs.

[0050] The dispatcher detection unit 1214 extracts each VM instruction portion from the script engine binary based on the VM instruction boundaries detected by the VM instruction boundary detection unit 1212, and detects the portion with high similarity between each VM instruction as a dispatcher. As a premise, a dispatcher is realized by referencing a pointer cache and jumping to the pointer of the next VM instruction handler. Dispatchers are distributed and located behind each VM instruction handler, and their code is generally highly identical. The analysis function providing device detects dispatchers using a predetermined method by searching for code that exists behind such VM instruction handlers and has high similarity. The detection of the portion with high similarity may use, for example, a sequence alignment algorithm or other methods.

[0051] The conditional branch flag detection unit 1215 extracts and analyzes the execution trace for the second test script stored in the execution trace DB 131 to find the conditional branch flag. The conditional branch flag detection unit 1215 analyzes multiple execution traces using differential execution analysis that focuses on the number of times memory is read, and detects the conditional branch flag. The conditional branch flag detection unit 1215 executes conditional branches in various patterns, and detects the memory that stores the conditional branch flag by comparing the memory change patterns at that time with the conditional branch patterns in the test script.

[0052] The code cache detection unit 1216 detects a code cache, which is a cache that stores virtual machine instructions to be executed, from the VM execution trace based on the execution trace, VPC, and VM execution trace.

[0053] The code cache detection unit 1216 detects the memory area pointed to by the VPC as a code cache from the VM execution trace.The code cache detection unit 1216 detects the code location of the caller of the memory allocation function that allocated this code cache from the execution trace.The code cache detection unit 1216 detects all memory areas allocated at this code location from the VM execution trace as code caches.

[0054] The code cache detection unit 1216 detects a code location that writes to the code cache from the execution trace, and detects the write by this code location from the VM execution trace as an update of the code cache.

[0055] The instruction set architecture analysis unit 122 analyzes the instruction set architecture, which is the system of instructions for the VM. The instruction set architecture analysis unit 122 collects VM instructions and determines the instruction contents of the collected virtual machine instructions.

[0056] The instruction set architecture analysis unit 122 includes a VM execution trace acquisition unit 1221 (second acquisition unit), a VM instruction collection unit 1222 (first collection unit), and a VM instruction determination unit 1223 (first determination unit).

[0057] Like the execution trace acquisition unit 1211, the VM execution trace acquisition unit 1221 receives a test script and a script engine binary as input. The VM execution trace acquisition unit 1221 acquires a VM execution trace by monitoring a VPC and a pointer to a VM instruction handler dispatched by a dispatcher. The VM execution trace acquisition unit 1221 acquires a VM execution trace, which is an execution trace executed on a VM, by executing a test script while monitoring the execution of the script engine binary. The VM execution trace acquisition unit 1221 executes a large number of test scripts to detect a branch VM instruction and acquires a VM execution trace. The VM execution trace acquisition unit 1221 associates a pointer to a VM instruction with the VM instruction and virtually assigns a VM opcode as an identifier to each instruction.

[0058] A VM execution trace is an execution trace executed in a VM, to which a VM opcode is virtually assigned as an identifier, and which records a pointer to the executed VM handler and a VPC. A VM execution trace records a pointer to the executed VM instruction handler and a VPC. Specifically, a VM execution trace is composed of a VPC and a VM opcode for each executed VM instruction. VPCs can be recorded by monitoring the memory of the VPC detected by the virtual program counter detection unit 1213. A VM opcode is an identifier virtually assigned to each VM instruction, which is linked to a pointer to the VM instruction. The VM execution trace acquired by the VM execution trace acquisition unit 1221 is stored in the VM execution trace DB 133.

[0059] The VM instruction collector 1222 receives the VPC and the dispatcher as input, executes the script while monitoring the VPC and the dispatcher, and acquires the VM execution trace. The VM instruction collector 1222 collects VM instructions from the VM execution trace.

[0060] The VM instruction determination unit 1223 determines the instruction content of the VM instruction collected by the VM instruction collection unit 1222. The VM instruction determination unit 1223 detects a branch VM instruction based on the variation in the amount of change in VPC for each VM opcode in the VM execution trace.

[0061] The VM instruction determination unit 1223 retrieves and analyzes the VM execution trace stored in the VM execution trace DB 133 to determine whether it is a branch VM instruction. For each VM opcode assigned as an identifier, the VM instruction determination unit 1223 collects the amount of change in VPC before and after its execution. If the VM opcode is not a branch VM instruction, the amount of change in VPC is almost constant. On the other hand, if the VM opcode is a branch VM instruction, the VPC varies depending on the branch destination.

[0062] Therefore, the VM instruction determination unit 1223 determines a branch VM instruction based on the variation in the amount of change in the virtual program counter for each VM opcode in the VM execution trace. The VM instruction determination unit 1223 focuses on the fact that the amount of variation in the VPC value differs between a branch VM instruction and other VM instructions, determines a threshold, and determines instructions with larger variations in the VPC value as branch VM instructions. Specifically, the VM instruction determination unit 1223 evaluates the variation in the amount of change in the VPC for each VM opcode using variance, and determines instructions with variance equal to or greater than a certain threshold as branch VM instructions.

[0063] Furthermore, the VM instruction determination unit 1223 determines which branch VM instructions are conditional branch VM instructions in order to construct a precise control flow graph. When a conditional branch occurs, a conditional branch flag is always accessed to determine the branch destination. Therefore, a conditional branch VM instruction can be determined by verifying whether the conditional branch flag is accessed when each branch VM instruction is executed. In other words, if the conditional branch flag is accessed when a branch VM instruction is executed, it can be determined that the instruction is a conditional branch VM instruction, and if not, it can be determined that the instruction is not a conditional branch VM instruction. Therefore, the VM instruction determination unit 1223 determines that, among branch VM instructions, an instruction that involves access to a conditional branch flag is a conditional branch VM instruction based on the VM execution trace and the memory access trace.

[0064] Furthermore, the VM instruction determination unit 1223 also determines call and return VM instructions. A branch caused by a call VM instruction is characterized in that the address immediately following the call in the bytecode is saved, and after execution of the called subroutine, a return VM instruction is used to return to the saved address. Therefore, when a certain branch VM instruction is designated as instruction 1 and another subsequent branch VM instruction is designated as instruction 2, and instruction 2 returns to the address immediately following instruction 1 in the bytecode, the VM instruction determination unit 1223 determines that the pair of instruction 1 and instruction 2 are call and return VM instructions.

[0065] The calculation unit 123 constructs a first control flow graph indicating all paths exhaustively executed by multi-pass execution and a second control flow graph indicating paths executed in the execution of the test target, based on the architecture information acquired by the virtual machine analysis unit 121 and the instruction set architecture analysis unit 122. The calculation unit 123 calculates the code coverage of the script executed on the script engine based on the first control flow graph and the second control flow graph. The code coverage is the ratio of paths executed in the execution of the test target to all paths.

[0066] The calculation unit 123 has a multi-path execution unit 1231 (first execution unit), VM branch trace construction units 1232 and 1235 (first construction units), control flow graph construction units 1233 and 1236 (second construction units), an execution unit 1234 (second execution unit), and a code coverage calculation unit 1237 (first calculation unit).

[0067] The multi-path execution unit 1231 performs multi-path execution of the script to be analyzed while acquiring a VM execution trace based on the architecture information acquired by the virtual machine analysis unit 121 and the instruction set architecture analysis unit 122. The multi-path execution unit 1231 monitors VPC and VM instructions, and executes the script to be analyzed while acquiring a VM execution trace. The multi-path execution unit 1231 performs multi-path execution by forking the execution state for each conditional branch instruction, leaving one as is and rewriting the conditional branch flag for the other, thereby comprehensively executing multiple execution paths.

[0068] The VM branch trace construction units 1232 and 1235 detect branch VM instructions from the VM execution trace 41 that records the opcodes and VPCs of executed VM instructions, and construct a VM branch trace that associates the VPCs before and after the execution of the detected branch VM instructions.

[0069] The VM branch trace construction unit 1232 detects an actually executed branch VM instruction from the VM execution trace acquired by the multipath execution unit 1231, and constructs a first VM branch trace in which VPCs before and after the execution of the detected branch VM instruction are associated. The VM branch trace construction unit 1235 detects a branch VM instruction that was actually executed in the execution of the test target from the VM execution trace acquired by the execution unit 1234 (described later), and constructs a second VM branch trace in which VPCs before and after the execution of the detected branch VM instruction are associated.

[0070] The control flow graph constructing units 1233 and 1236 use the VM branch trace to construct a control flow graph in which basic blocks are nodes and branches resulting from the execution of branch VM instructions are edges.

[0071] The control flow graph construction unit 1233 constructs a first control flow graph based on the first VM branch trace. The first control flow graph is constructed based on information actually executed in multi-path execution, and is therefore a complete control flow graph showing all paths. Furthermore, the control flow graph construction unit 1236 constructs a second control flow graph based on the second VM branch trace. Note that the control flow graph construction unit 1233 may construct the graph by scanning the detected code cache, taking into account branch instructions that were not actually executed.

[0072] The execution unit 1234 executes the test target for the analysis target script while acquiring a VM execution trace, which is an execution trace executed in the VM. The execution unit 1234 records the VM instructions executed during this execution and acquires the VM execution trace.

[0073] The code coverage calculation unit 1237 calculates the code coverage of the script executed on the script engine based on the first control flow graph and the second control flow graph. The code coverage calculation unit 1237 calculates, as the code coverage, the ratio of the number of nodes and / or edges of the second control flow graph to the number of nodes and edges of the first control flow graph.

[0074] The storage unit 13 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk, and stores processing programs that operate the analysis function providing device 10, data used during execution of the processing programs, etc. The storage unit 13 has an execution trace database (DB) 131, a VM execution trace DB 133, and an architecture information DB 132 that stores architecture information acquired by the virtual machine analysis unit 121 and the instruction set architecture analysis unit 122.

[0075] The execution trace DB 131 and the VM execution trace DB 133 store the execution traces and VM execution traces acquired by the execution trace acquisition unit 1211 and the VM execution trace acquisition unit 1221, respectively. The execution trace DB 131 and the VM execution trace DB 133 are managed by the analysis function-imparting device 10. Of course, the execution trace DB 131 and the VM execution trace DB 133 may be managed by another device (such as a server). In this case, the execution trace acquisition unit 1211 and the VM execution trace acquisition unit 1221 output the acquired execution traces and VM execution traces to a management server or the like for the execution trace DB 131 and the VM execution trace DB 133 via the communication interface of the output unit 14, and store them in the execution trace DB 131 and the VM execution trace DB 133.

[0076] The output unit 14 is, for example, a liquid crystal display or a printer, and outputs various information including information related to the analysis function imparting device 10. The output unit 14 may also be an interface that controls input and output of various data between the output unit 14 and an external device, and may output various information to the external device.

[0077] [Test script configuration] This section explains test scripts. Test scripts are scripts that are input when dynamically analyzing a script engine. These test scripts focus on the number of branch instruction executions and memory read / writes, and are used to capture differences in the behavior of the script engine that occur when the test script is executed a different number of times. These test scripts are prepared in advance of the analysis and are created manually. Creating them requires knowledge of the specifications of the target script language.

[0078] Figure 4 shows an example of a test script (first test script) used to detect VPCs. The first test script uses a repetitive process (line 2). The first test script changes the execution conditions and generates differences by increasing or decreasing the number of repetitions (line 2) and the number of repeated statements (lines 3 to 5) in the test script.

[0079] 5 is a diagram showing an example of a test script (second test script) used to detect a branch VM instruction. The second test script uses multiple conditional branches (lines 4 to 8). In the second test script, the branch conditions are controlled so that the multiple conditional branches are either taken or not taken in a specific order pattern (lines 1 and 5). In the second test script, the number of conditional branches and the order pattern of branch success / failure are changed to generate differences.

[0080] Configure Execution Tracing Next, the execution trace will be explained. Fig. 6 is a diagram showing an example of an execution trace. As mentioned above, an execution trace is made up of a branch trace and a memory access trace. Fig. 6 shows an excerpt of an execution trace. The structure of an execution trace will be explained below using Fig. 6.

[0081] An execution trace has an element called trace, which indicates whether the log line is a branch trace or a memory access trace.

[0082] A branch trace log line is formatted as shown in lines 1 to 10 of Figure 6, and consists of three elements: type, src, and dst. type indicates whether the executed branch instruction was a call instruction, a jmp instruction, or a ret instruction. src indicates the address of the branch source, and dst indicates the address of the branch destination.

[0083] A log line of a memory access trace is formatted as shown in lines 11 to 13 of Figure 6, and consists of three elements: type, target, and value. type indicates whether the memory access is a read or write. target indicates the memory address that is the target of the memory access. value stores the result of the memory access.

[0084] [VM Execution Tracing Configuration] Next, we will explain the VM execution trace. Fig. 7 is a diagram showing an example of a VM execution trace. As mentioned above, the VM execution trace is a record of the VM opcode and VPC. Fig. 7 shows a portion of the VM execution trace. The structure of the VM execution trace will be explained below using Fig. 7.

[0085] A log line of a VM execution trace is, for example, in the format shown in Figure 7 and consists of two elements: vpc and vmop (vm opcode). vpc indicates the value of VPC. Also, vmop indicates the value of the VM opcode virtually assigned to each pointer that points to the beginning of the VM instruction handler to be executed, obtained from the pointer cache.

[0086] [VM instruction boundary detection processing] Next, a description will be given of the processing of the VM instruction boundary detection unit 1212. FIG.

[0087] The VM instruction boundary detection unit 1212 detects the boundaries of each VM instruction. At this time, the VM instruction boundary detection unit 1212 detects VM instructions and their boundaries for a threaded code VM, which does not have an interpreter loop and therefore makes it difficult to grasp the boundaries of VM instructions. Specifically, the VM instruction boundary detection unit 1212 extracts execution traces from the execution trace DB 131. Then, as shown in FIG. 8, the VM instruction boundary detection unit 1212 clusters the execution traces using a predetermined method and detects clusters whose execution counts are equal to or greater than a threshold as VM instructions (e.g., VM instruction handlers 1 to 3). The VM instruction boundary detection unit 1212 detects the start and end points of a continuous instruction sequence that constitutes a VM instruction as boundaries.

[0088] [Virtual Program Counter Detection Processing] Next, the processing of the virtual program counter detection unit 1213 will be described. The virtual program counter detection unit 1213 detects VPCs and pointer caches. The detection of the virtual program counter is achieved by analyzing the memory access trace log of the acquired execution trace. The virtual program counter detection unit 1213 uses differential execution analysis that focuses on the number of times memory is read. FIG. 9 is a diagram explaining the processing of the virtual program counter detection unit 1213.

[0089] The virtual program counter detection unit 1213 extracts one execution trace by the first test script from the execution trace DB 131. The number of VPC reads is proportional to the number of repetitions in the test script and the number of statements in the repetitive processing. When the number of repetitions is N and the number of repeated statements is M, approximately MN VPC reads occur. Therefore, the virtual program counter detection unit 1213 extracts memory areas that have increased by 4MN and 9MN in the execution trace for the first test script in which N and M have been increased to 2N and 2M, respectively, and 3N and 3M. Specifically, as shown in FIG. 9, the virtual program counter detection unit 1213 extracts memory areas that have a monotonically increasing read / write for each VM instruction execution ((1) in FIG. 9).

[0090] The virtual program counter detection unit 1213 then detects, as a VPC, a memory value that always points to the start point of a VM instruction. Specifically, the virtual program counter detection unit 1213 compares the VPC's pointer with the address of the VM instruction handler, and narrows down the memory area to the matching memory area ((2) in FIG. 9).

[0091] [Dispatcher detection processing] Next, a description will be given of the processing of the dispatcher detection unit 1214. The dispatcher detection unit 1214 detects dispatchers by analyzing the binary of the script engine using a predetermined method. Fig. 10 is a diagram illustrating the processing of the dispatcher detection unit 1214.

[0092] The dispatcher detection unit 1214 detects dispatchers. Based on the boundaries of VM instructions detected by the VM instruction boundary detection unit 1212, the dispatcher detection unit 1214 extracts each VM instruction portion from the script engine binary. Then, based on the assumption that the similarity of dispatcher code is high ((1) in FIG. 10), the dispatcher detection unit 1214 calculates the similarity between the codes of each VM instruction, and detects the portion with high similarity among all VM instructions as a dispatcher. The dispatcher detection unit 1214 can detect code that is commonly executed in the latter half of VM instructions as a dispatcher ((1) in FIG. 10).

[0093] [Code cache detection] Next, a description will be given of the processing of the code cache detection unit 1216. FIG.

[0094] The code cache detection unit 1216 detects the memory area pointed to by the VPC as a code cache from the VM execution trace ((1) in FIG. 11).

[0095] The code cache detection unit 1216 detects the code location that called the memory allocation function that allocated this code cache from the execution trace ((2) in FIG. 11). The code cache detection unit 1216 detects all memory areas allocated at this code location from the VM execution trace as code caches ((3) in FIG. 11).

[0096] The code cache detection unit 1216 detects a code location that writes to the code cache from the execution trace ((4) in FIG. 11). The code cache detection unit 1216 detects the write by this code location from the VM execution trace as an update of the code cache ((5) in FIG. 11).

[0097] [VM command judgment unit] Next, the processing of the VM instruction determination unit 1223 will be described. The VM instruction determination unit 1223 determines a branch VM instruction by analyzing the acquired VM execution trace log. The test script here may be any script that includes a branch VM instruction and includes a branch control syntax. For example, the test script may be prepared by collecting scripts from the Internet or official documents.

[0098] First, the VM instruction determination unit 1223 associates a pointer to a VM instruction with a VM instruction for each VM execution trace in the VM execution trace DB 133, and virtually assigns a VM opcode as an identifier to each VM instruction. Fig. 12 is a diagram illustrating the processing of the VM instruction determination unit 1223.

[0099] Here, when a VM instruction is a branch instruction, the advance of VPC changes depending on the branch destination. On the other hand, when it is not a branch instruction, the advance of VPC changes depending on the size of the VM instruction. Therefore, when pairs of VM instruction opcodes and pointers to VM instructions are collected and the advance of VPC is examined for each opcode, if it is a branch instruction, the advance of VPC will vary depending on the branch destination.

[0100] Therefore, the VM instruction determination unit 1223 uses variance to evaluate the variance of the pointer to this VM instruction. The VM instruction determination unit 1223 calculates the variance of the VPC change amount for each VM opcode and narrows down to only VM opcodes whose calculated variance is greater than a threshold. As a result, the VM instruction determination unit 1223 associates the pointer with the VM instruction and determines that the VM instruction with variance in the VPC advance (VM instruction handler 3 in the example of FIG. 12) is a branch VM instruction ((1) in FIG. 12).

[0101] Let O be the set of VPC advances for a certain opcode: O={o0,o1,...,o N} (see equation (1) for the average of VPCo), and t for the threshold, whether it is a branch instruction or not is determined based on the variance s (see equation (2)) as shown in equation (3). In this way, the VM instruction determination unit 1223 determines whether it is a branch VM instruction.

[0102]

number

[0103]

number

[0104]

number

[0105] In addition, for VM instructions other than branches, there is almost no variation, and the boundary between branch VM instructions and other VM instructions is often clear. For this reason, for example, the threshold value is set to a value that can divide the two groups obtained by plotting the obtained variance value on a number line.

[0106] [VM branch trace construction processing] Next, a description will be given of the processing of the VM branch trace construction units 1232 and 1235. Fig. 13 is a diagram for explaining the processing of the VM branch trace construction units.

[0107] 13, the VM branch trace constructing units 1232 and 1235 detect a branch VM instruction from the VM execution trace 41 that records the opcode of the executed VM instruction and the VPC ((1) in FIG. 13). The branch VM instruction can be recognized by referring to the branch VM instruction list 42 detected by the VM instruction determining unit 1223.

[0108] Then, the VM branch trace construction units 1232 and 1235 construct a VM branch trace 43 that associates VPCs before and after the execution of the detected branch VM instruction ((2) in FIG. 13). For example, the VM branch trace construction units 1232 and 1235 detect the branch VM instruction "0x1f" from line R41 of the VM execution trace, and construct a VM branch trace shown in line R61 based on line R41 and line R42 that follows line R42. That is, the VM branch trace construction units 1232 and 1235 associate the branch source VPC "0x555c7e48" of line R41 with the branch source VPC "0x555c82a0" of line R42.

[0109] Similarly, when the VM branch trace construction units 1232 and 1235 detect the branch VM instruction "0x21" from line R51 of the M execution trace, they associate the VPC "0x555c832c" of line R51 with the VPC "0x555c7514" of line R52 based on this line R51 and the next line R52 (line R71).

[0110] [Control flow graph construction process] 14 is a diagram illustrating the processing of the control flow graph construction units 1233 and 1236. Using the VM branch trace 43 constructed in the VM branch trace construction process, the control flow graph construction units 1233 and 1236 construct a control flow graph in which basic blocks are nodes and each branch of the VM branch trace 43 is an edge ((1) in FIG. 14).

[0111] Specifically, the control flow graph construction units 1233 and 1236 designate the branch shown in row R61 of the VM branch trace 43 as edge E61, the basic block from which edge E61 branches as node N61, and the basic block to which edge E61 branches as node N62.

[0112] The branch shown in row R71 of the VM branch trace 43 is defined as edge E71, the subblock from which edge E71 branches is defined as node N71, and the basic block to which edge E71 branches is defined as node N72. In the example of the control flow graph in Figure 14, node N71 has a branch shown by edge E72 in addition to edge E71, and the basic block to which this branch is directed is defined as node N73. In this way, the control flow graph construction units 1233 and 1236 construct a control flow graph that represents the branches of basic blocks in a graph structure based on the VM branch trace.

[0113] [Processing procedure for the analysis function device] Next, a description will be given of the processing procedure of the analysis function imparting process by the analysis function imparting device 10. Fig. 15 is a flowchart showing the processing procedure of the analysis function imparting process according to the embodiment.

[0114] First, the input unit 11 receives a test script and a script engine binary as input (step S1).

[0115] Then, the execution trace acquisition unit 1211 executes the test script while monitoring the binary of the script engine, thereby performing an execution trace acquisition process to acquire a branch trace and a memory access trace (step S2).

[0116] The VM instruction boundary detection unit 1212 detects VM instructions and performs VM instruction boundary detection processing to detect VM instruction boundaries (step S3). The virtual program counter detection unit 1213 extracts and analyzes the execution trace for the first test script stored in the execution trace DB 131 and performs virtual program counter detection processing to find VPCs (step S4).

[0117] The dispatcher detection unit 1214 extracts each VM instruction portion from the script engine binary and performs dispatcher detection processing to detect a portion with high similarity between each VM instruction as a dispatcher (step S5).

[0118] The conditional branch flag detection unit 1215 extracts and analyzes the execution trace for the second test script stored in the execution trace DB 131, and performs a conditional branch detection process to discover a conditional branch flag (step S6).

[0119] Based on the execution trace and VPC, the code cache detection unit 1216 detects the area of ​​the code location that called the memory allocation function as a code cache, and performs a code cache detection process to detect areas that are writing to the code location area as updates to the code cache (step S7).

[0120] The VM execution trace acquisition unit 1221 receives the test script and the script engine binary as input, and executes the test script while monitoring the execution of the script engine binary, thereby performing a VM execution trace acquisition process to acquire the VM execution trace (step S8).

[0121] The VM instruction collection unit 1222 performs a VM instruction collection process to acquire VM instructions from the VM execution trace (step S9). The VM instruction determination unit 1223 performs a VM instruction determination process to determine the instruction contents of the collected VM instructions (step S10).

[0122] The input unit 11 receives an input of a script to be analyzed (step S11). The multi-pass execution unit 1231 executes the script to be analyzed in a multi-pass manner while acquiring a VM execution trace, based on the architecture information acquired by the virtual machine analysis unit 121 and the instruction set architecture analysis unit 122 (step S12).

[0123] The VM branch trace construction unit 1232 detects a branch VM instruction from the VM execution trace, and executes a VM branch trace construction process to construct a VM branch trace in which VPCs before and after the execution of the detected branch VM instruction are associated with each other (step S13). In step S13, the VM branch trace construction unit 1232 constructs a first VM branch trace based on the multi-path execution by the multi-path execution unit 1231.

[0124] The control flow graph construction unit 1233 performs a control flow graph construction process to construct a control flow graph using the VM branch trace (step S14). In step S14, the control flow graph construction unit 1233 constructs a first control flow graph based on the first VM branch trace.

[0125] The execution unit 1234 performs an execution process to execute a test target for the analysis target script while acquiring a VM execution trace (step S15). The VM branch trace construction unit 1235 performs a VM branch trace construction process to construct a second VM branch trace by performing the same process as in step S13 on the VM execution trace acquired by the execution unit 1234 (step S16). The control flow graph construction unit 1236 performs a control flow graph construction process to construct a second control flow graph by performing the same process as in step S14 on the second VM branch trace (step S17).

[0126] The code coverage calculation unit 1237 performs a code coverage calculation process to calculate the code coverage of the script executed on the script engine based on the first control flow graph and the second control flow graph (step S18). The code coverage calculation unit 1237 outputs the calculated code coverage from the output unit 14 (step S19).

[0127] [Execution trace acquisition process procedure] Next, a description will be given of the flow of the execution trace acquisition process shown in Fig. 15. Fig. 16 is a flowchart showing the processing procedure of the execution trace acquisition process shown in Fig. 15.

[0128] First, the execution trace acquisition unit 1211 receives a test script and a script engine binary as input (step S21). Then, the execution trace acquisition unit 1211 hooks the received script engine to acquire a branch trace (step S22). The execution trace acquisition unit 1211 also hooks the received script engine to acquire a memory access trace (step S23).

[0129] Then, the execution trace acquisition unit 1211 inputs the received test script in this state into the script engine to execute it (step S24), and stores the execution trace acquired thereby in the execution trace DB 131 (step S25).

[0130] The execution trace acquisition unit 1211 determines whether or not all of the input test scripts have been executed (step S26). If all of the input test scripts have been executed (step S26: Yes), the execution trace acquisition unit 1211 ends the process. On the other hand, if all of the input test scripts have not been executed (step S26: No), the execution trace acquisition unit 1211 returns to the execution of the test script in step S24 and continues the process.

[0131] [VM instruction boundary detection processing procedure] Next, a description will be given of the flow of the VM instruction boundary detection process shown in Fig. 15. Fig. 17 is a flowchart showing the processing procedure of the VM instruction boundary detection process shown in Fig. 15.

[0132] First, the VM instruction boundary detection unit 1212 extracts execution traces from the execution trace DB 131 (step S31). The VM instruction boundary detection unit 1212 clusters the execution traces using a predetermined method (step S32). Any method may be used for clustering.

[0133] The VM instruction boundary detection unit 1212 detects, as a VM instruction, a cluster whose execution count is equal to or greater than a threshold (step S33). Then, the VM instruction boundary detection unit 1212 determines the start and end points of a continuous instruction sequence that constitutes a VM instruction as boundaries (step S34). The VM instruction boundary detection unit 1212 outputs the VM instruction boundaries as a return value (step S35), and ends the VM instruction boundary detection process.

[0134] [Procedure for detecting virtual program counter] Next, a description will be given of the flow of the virtual program counter detection process shown in Fig. 15. Fig. 18 is a flowchart showing the processing procedure of the virtual program counter detection process shown in Fig. 15.

[0135] First, the virtual program counter detection unit 1213 extracts one execution trace by the first test script from the execution trace DB 131 (step S41). Next, the virtual program counter detection unit 1213 focuses on memory access traces among the execution traces, and counts up the number of reads for each memory read destination (step S42).

[0136] The virtual program counter detection unit 1213 receives as input the first test script used to acquire the execution trace (step S43), and analyzes the first test script to acquire the number of repetitions and the number of repeated statements (step S44).

[0137] Next, the virtual program counter detection unit 1213 extracts another execution trace by the first test script, which has a different number of repetitions and number of repeated statements, from the execution trace DB 131 (step S45).The virtual program counter detection unit 1213 then focuses on the memory access trace and counts the number of reads for each memory read destination (step S46).The virtual program counter detection unit 1213 also receives as input the first test script used to acquire the execution trace (step S47), analyzes the test script, and acquires the number of repetitions and the number of repeated statements (step S48).

[0138] Here, the virtual program counter detection unit 1213 narrows down the memory read destinations to only those whose read counts change in proportion to the increase or decrease in the number of repetitions or repeated statements (step S49). Furthermore, the virtual program counter detection unit 1213 narrows down the memory read destinations narrowed down in step S49 to those whose read memory values ​​always point to the start point of the VM instruction (step S50).

[0139] Then, the virtual program counter detection unit 1213 determines whether the memory read destinations have been narrowed down to only one (step S51). If the virtual program counter detection unit 1213 has not narrowed down the memory read destinations to only one (step S51: No), it returns to step S45, extracts the next execution trace, and continues processing. On the other hand, if the virtual program counter detection unit 1213 has narrowed down the memory read destinations to only one (step S51: Yes), it stores the narrowed down memory read destination as a virtual program counter in the architecture information DB 132 (step S52), and ends processing.

[0140] [Dispatcher detection process procedure] Next, a description will be given of the flow of the dispatcher detection process shown in Fig. 15. Fig. 19 is a flowchart showing the processing procedure of the dispatcher detection process shown in Fig. 15.

[0141] First, the dispatcher detection unit 1214 receives the script engine binary as an input (step S61). The dispatcher detection unit 1214 receives the boundaries of VM instructions from the VM instruction boundary detection unit 1212 (step S62).

[0142] The dispatcher detection unit 1214 extracts each VM instruction portion from the script engine binary based on the boundaries of the VM instructions received from the VM instruction boundary detection unit 1212 (step S63). The dispatcher detection unit 1214 calculates the similarity between the codes of each VM instruction using a predetermined method (step S64). Any method for calculating the similarity may be used as long as it is capable of calculating the similarity between the codes.

[0143] The dispatcher detection unit 1214 extracts a part with high similarity among all VM instructions based on the similarity calculated in step S64 (step S65), and then determines whether it is the end part of the VM instruction (step S66).

[0144] If it is not the end of the VM instruction (step S66: No), the dispatcher detection unit 1214 returns to step S65 and continues the process. If it is the end of the VM instruction (step S66: Yes), the dispatcher detection unit 1214 outputs the extracted part as a dispatcher (step S67) and ends the process.

[0145] [Conditional branch flag detection processing procedure] Next, a description will be given of the flow of the conditional branch flag detection process shown in Fig. 15. Fig. 20 is a flowchart showing the processing procedure of the conditional branch flag detection process shown in Fig. 15.

[0146] First, the conditional branch flag detection unit 1215 extracts one execution trace by the second test script from the execution trace DB 131 (step S71). Then, the conditional branch flag detection unit 1215 focuses on memory access traces and counts the number of reads for each memory read destination (step S72).

[0147] The conditional branch flag detection unit 1215 also receives as input the second test script used to acquire the execution trace (step S73), analyzes this second test script, and acquires the number of conditional branches and the True / False order pattern (step S74).The conditional branch flag detection unit 1215 then narrows down the memory read destinations to only those whose read count changes in proportion to the number of conditional branches (step S75).Furthermore, the conditional branch flag detection unit 1215 narrows down the memory read destinations to only those whose read memory values ​​alternate between two values ​​in accordance with the True / False order pattern (step S76).

[0148] The conditional branch flag detection unit 1215 determines whether the memory read destinations have been narrowed down to only one (step S77). If the conditional branch flag detection unit 1215 has not narrowed down the memory read destinations to only one (step S77: No), it returns to step S71, extracts the next execution trace, and continues processing. On the other hand, if the conditional branch flag detection unit 1215 has narrowed down the memory read destinations to only one (step S77: Yes), it stores the narrowed down read destination in the architecture information DB 132 as a virtual program counter (step S78), and ends processing.

[0149] [Code cache detection process procedure] Next, a description will be given of the flow of the code cache detection process shown in Fig. 15. Fig. 21 is a flowchart showing the processing procedure of the code cache detection process shown in Fig. 15.

[0150] The code cache detection unit 1216 receives an execution trace and a VM execution trace as input (step S81), and acquires a memory area pointed to by a VPC from the VM execution trace (step S82). The VM execution trace is acquired by the VM execution trace acquisition unit 1221.

[0151] The code cache detection unit 1216 acquires, from the execution trace, the code location of the caller of the memory allocation function that allocated the memory area acquired in step S82 (step S83).The code cache detection unit 1216 detects, from the VM execution trace, all areas allocated at the code location acquired in step S83 as code caches (step S84).

[0152] The code cache detection unit 1216 acquires the code location that is writing to the code cache from the execution trace (step S85). The code cache detection unit 1216 detects all areas written to at the code location acquired in step S85 in the VM execution trace as code cache updates (step S86). The code cache detection unit 1216 returns the detected code cache and its updated location (step S87), and ends the code cache detection process.

[0153] [VM execution trace acquisition process procedure] Next, a description will be given of the flow of the VM execution trace acquisition process shown in Fig. 15. Fig. 22 is a flowchart showing the processing procedure of the VM execution trace acquisition process shown in Fig. 15.

[0154] First, the VM execution trace acquisition unit 1221 receives a test script and a script engine binary as input (step S91). Then, the VM execution trace acquisition unit 1221 hooks the received script engine to record the VPC and VM opcode (step S92).

[0155] The VM execution trace acquisition unit 1221 inputs the received test script into the script engine and executes it (step S93), and stores the VM execution trace acquired thereby in the VM execution trace DB 133 (step S94).

[0156] The VM execution trace acquisition unit 1221 determines whether all the input test scripts have been executed (step S95). If all the input test scripts have been executed (step S95: Yes), the VM execution trace acquisition unit 1221 ends the process. If all the input test scripts have not been executed (step S95: No), the VM execution trace acquisition unit 1221 returns to the execution of the test script in step S83 and continues the process.

[0157] [VM instruction collection process procedure] Next, a description will be given of the flow of the VM instruction collection process shown in Fig. 15. Fig. 23 is a flowchart showing the processing procedure of the VM instruction collection process shown in Fig. 15.

[0158] The VM instruction collection unit 1222 receives a VPC and a dispatcher as input (step S101), and acquires various scripts from the Internet (step S102). The VM instruction collection unit 1222 executes the scripts while monitoring the VPC and the dispatcher, and acquires a VM execution trace (step S103).

[0159] The VM instruction collection unit 1222 acquires a VM instruction from the VM execution trace (step S104) and adds it to a list of VM instructions (step S105). If the VM instruction collection unit 1222 finds a VM instruction that is not in the list (step S106: No), the VM instruction collection unit 1222 returns to step S102. If the VM instruction collection unit 1222 finds no VM instructions that are not in the list (step S106: Yes), the VM instruction collection unit 1222 returns the list of VM instructions (step S107) and ends the VM instruction collection process.

[0160] [VM instruction determination processing procedure] Next, a description will be given of the flow of the VM instruction determination process shown in Fig. 15. Fig. 24 is a flowchart showing the processing procedure of the VM instruction determination process shown in Fig. 15.

[0161] The VM instruction determination unit 1223 extracts one VM execution trace from the VM execution trace DB 133 (step S111). The VM instruction determination unit 1223 associates a pointer to the VM instruction with the VM instruction and assigns a VM opcode to each as an identifier (step S112). Then, the VM instruction determination unit 1223 counts the amount of change in VPC before and after execution for each VM opcode (step S113).

[0162] The VM instruction determination unit 1223 determines whether or not all the VM execution traces in the VM execution trace DB 133 have been processed (step S114). If all the VM execution traces in the VM execution trace DB 133 have not been processed (step S114: No), the VM instruction determination unit 1223 returns to step S111, and extracts and processes the next VM execution trace.

[0163] If all VM execution traces in the VM execution trace DB 133 have been processed (step S114: Yes), the VM instruction determination unit 1223 calculates the variance of the VPC change amount for each VM opcode (step S115). Then, the VM instruction determination unit 1223 receives a threshold as an input (step S116). The VM instruction determination unit 1223 narrows down the VM opcodes to only those whose variance is greater than the threshold (step S117), stores them as branch VM instructions in the architecture information DB 132 (step S118), and ends the process.

[0164] [Multipath execution process procedure] Next, a description will be given of the flow of the multi-path execution process shown in Fig. 15. Fig. 25 is a flowchart showing the processing procedure of the multi-path execution process shown in Fig. 15.

[0165] The multi-path execution unit 1231 receives an analysis target script as an input (step S121), and receives a VPC, a dispatcher, and a conditional branch flag as an input (step S122).

[0166] The multi-path execution unit 1231 monitors VPC and VM instructions, and executes the script to be analyzed while acquiring a VM execution trace (step S123). The multi-path execution unit 1231 forks the execution state for each conditional branch instruction, leaving one as is and rewriting the conditional branch flag for the other, thereby comprehensively executing multiple execution paths (step S124).

[0167] If all execution paths have not been exhaustively executed (step S125: No), the multi-path execution unit 1231 returns to step S123. If all execution paths have been exhaustively executed (step S125: Yes), the multi-path execution unit 1231 returns the VM execution trace (step S126) and ends the multi-path execution process.

[0168] [VM branch trace construction process procedure] Next, a description will be given of the flow of the VM branch trace construction process shown in Fig. 15. Fig. 26 is a flowchart showing the processing procedure of the VM branch trace construction process shown in Fig. 15.

[0169] The VM branch trace constructing units 1232 and 1235 receive the VM execution trace and the VM branch instruction list as inputs (step S131).

[0170] The VM branch trace constructing units 1232 and 1235 extract the entry of the VM execution trace (step S132). The VM branch trace constructing units 1232 and 1235 determine whether the VM opcode exists in the VM branch instruction list (step S133).

[0171] If the VM opcode exists in the VM branch instruction list (step S133: Yes), the VM branch trace constructing units 1232 and 1235 store the VPC as the branch source and the VPC of the next entry as the branch destination in the VM branch trace (step S134).

[0172] If the VM opcode does not exist in the VM branch instruction list (step S133: No), or after step S134 is completed, the VM branch trace constructing units 1232 and 1235 determine whether all entries of the VM execution trace have been processed (step S135).

[0173] If all entries of the VM execution trace have not been processed (step S135: No), the VM branch trace construction units 1232 and 1235 extract the next entry of the VM execution trace (step S136). Then, the VM branch trace construction units 1232 and 1235 return to step S133 and determine whether the VM opcode of the next entry exists in the VM branch instruction list.

[0174] On the other hand, if all entries of the VM execution trace have been processed (step S135: Yes), the VM branch trace construction units 1232 and 1235 end the VM branch trace construction process.

[0175] [Control flow graph construction process] Next, a description will be given of the flow of the control flow graph construction process shown in Fig. 15. Fig. 27 is a flowchart showing the processing procedure of the control flow graph construction process shown in Fig. 15.

[0176] When the control flow graph constructing units 1233 and 1236 receive a VM branch trace as an input (step S141), they extract an entry of the VM branch trace (step S142).

[0177] The control flow graph constructing units 1233 and 1236 add a basic block having the branch destination address as a starting point to the control flow graph as a node (step S143). The control flow graph constructing units 1233 and 1236 add an edge from the branch source address to the branch destination address to the control flow graph (step S144). The control flow graph constructing units 1233 and 1236 determine whether all entries of the VM branch trace have been processed (step S145).

[0178] If all entries of the VM branch trace have not been processed (step S145: No), the control flow graph constructing units 1233 and 1236 extract the next entry of the VM execution trace (step S146). Then, the control flow graph constructing units 1233 and 1236 return to step S143 and add the basic block having the branch destination address as the starting point for the next entry to the control flow graph as a node.

[0179] When all entries of the VM branch trace have been processed (step S145: Yes), the control flow graph constructing units 1233 and 1236 output the constructed control flow graph (step S147).

[0180] [Execution process procedure] Next, a description will be given of the flow of the execution process shown in Fig. 15. Fig. 28 is a flowchart showing the processing procedure of the execution process shown in Fig. 15.

[0181] The execution unit 1234 receives an analysis target script as an input (step S151), and receives a VPC, a dispatcher, and a conditional branch flag as an input (step S152).

[0182] The execution unit 1234 monitors the VM instructions and VPCs being executed (step S153) and executes the script to be analyzed (step S154). The execution unit 1234 records the VM instructions executed during execution and acquires a VM execution trace (step S155). The execution unit 1234 returns the VM execution trace (step S156) and ends the execution process.

[0183] [Code coverage calculation procedure] Next, a description will be given of the flow of the code coverage calculation process shown in Fig. 15. Fig. 29 is a flowchart showing the processing procedure of the code coverage calculation process shown in Fig. 15.

[0184] The code coverage calculation unit 1237 counts the number of all nodes and edges in the first control flow graph (step S161). The code coverage calculation unit 1237 counts the number of all nodes and edges in the second control flow graph (step S162).

[0185] The code coverage calculation unit 1237 calculates the ratio of the number of nodes and the number of edges of the second control flow graph to the number of nodes and the number of edges of the first control flow graph (step S163). As the code coverage of the script executed on the script engine, the code coverage calculation unit 1237 may calculate both the ratio of the number of nodes of the second control flow graph to the number of nodes of the first control flow graph and the ratio of the number of edges of the second control flow graph to the number of edges of the first control flow graph, or may calculate either one of them.

[0186] The code coverage calculation unit 1237 returns the calculated ratio as the code coverage of the script executed on the script engine (step S164), and ends the code coverage calculation process.

[0187] [Effects of the embodiment] In this way, the analysis function-adding device 10 according to the embodiment analyzes the VM of the script engine, collects VM instructions, determines the contents of the collected VM instructions, and acquires information on the instruction set architecture, which is the system of instructions for the VM machine. Based on the acquired architecture information, the analysis function-adding device 10 constructs a first control flow graph showing the entire path comprehensively executed by multi-pass execution and a second control flow graph showing the path executed in the execution of the test target, and calculates the code coverage of the script executed on the script engine based on the first control flow graph and the second control flow graph.

[0188] Therefore, even for script engines whose internal specifications are unknown, the analysis function-imparting device 10 can analyze the VM of the script engine and obtain information on the instruction set architecture, which is the system of instructions for the VM machine, thereby imparting to the script engine the function of calculating the code coverage of scripts executed on the script engine, without requiring individual manual analysis, design, and implementation.

[0189] Specifically, the analysis function adding device 10 executes a test script while monitoring the binary of the script engine, and acquires branch traces and memory access traces as execution traces. The analysis function adding device 10 analyzes the virtual machine based on the execution trace, and acquires architecture information on VM instruction boundaries, VPCs, dispatchers, conditional branch flags, and code caches. Furthermore, the analysis function adding device 10 executes the test script while monitoring the VPCs and dispatchers, and acquires a VM execution trace. By analyzing the VM execution trace, the analysis function adding device collects VM instructions, determines the contents of the VM instructions, and acquires information on the instruction set architecture.

[0190] In this way, the analysis function-imparting device 10 can detect architecture information including information indicating where in the VM the bytecode generated by the script engine is stored, and information on the instruction set architecture of the bytecode that the VM can interpret, even for a script engine whose internal specifications are unknown.

[0191] Then, based on the acquired architecture information, the analysis function-imparting device 10 constructs a first control flow graph of the comprehensive overall path in multi-pass execution and a second control flow graph of the path executed in the execution of the test target, and based on these, newly imparts to the script engine the function of calculating the code coverage of the script executed on the script engine.

[0192] As a result, the analysis function-imparting device 10 can detect various architectural information by analyzing the execution trace and VM execution trace obtained, even for script engines whose VM internal specifications are unknown, and can provide code coverage measurement functions for scripts executed on the script engine without requiring manual reverse engineering.

[0193] Furthermore, the analysis function-imparting device 10 can automatically impart code coverage measurement functionality to various script engines for scripts executed on the script engine as long as a test script is prepared, thereby realizing the imparting of code coverage measurement functionality without the need for individual design or execution.

[0194] As a result, the analysis function-imparting device 10 can provide a function that enables measurement of the code coverage of scripts executed on a script engine, even for scripts created in various script languages, thereby enabling more comprehensive testing.

[0195] In this way, according to the analysis function-imparting device 10 of this embodiment, by retrofitting the function of analyzing the script engine and measuring code coverage, it is possible to automatically impart the function of measuring code coverage of scripts executed on the script engine to script engines of a wide variety of script languages.

[0196] In addition, the analysis function-providing device 10 is useful for measuring the code coverage of scripts written in a wide variety of scripting languages, and is suitable for measuring the code coverage of scripts for which it is difficult to measure due to the absence of support functions such as a debugger or the unknown internal specifications of the VM.

[0197] Therefore, by using the analysis function-imparting device 10 to impart to various script engines a code coverage measurement function for the scripts executed on the script engine, it is possible to measure the code coverage of the scripts implemented for testing and use this to realize highly comprehensive testing.

[0198] [System configuration of the embodiment] 3 are conceptual functional components of the analysis function-imparting device 10 and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of the functions of the analysis function-imparting device 10 is not limited to that shown, and all or part of them can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.

[0199] Furthermore, all or any part of the processes performed in the analysis function-imparting device 10 may be realized by a CPU and a program analyzed and executed by the CPU. Furthermore, each process performed in the analysis function-imparting device 10 may be realized as hardware using wired logic.

[0200] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters described above and illustrated can be changed as appropriate unless otherwise specified.

[0201] [program] 30 is a diagram showing an example of a computer in which a program is executed to realize the analytical function-imparting device 10. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0202] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0203] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the analysis function-imparting device 10 is implemented as a program module 1093 in which code executable by the computer 1000 is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing the same process as the functional configuration of the analysis function-imparting device 10 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0204] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, CPU 1020 reads program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary and executes them.

[0205] The program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0206] Although the present invention has been described above as an embodiment, the present invention is not limited to the description and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention. [Explanation of symbols]

[0207] 10. Analysis function device 11 Input section 12 Control Unit 13 Storage section 14 Output section 121 Virtual Machine Analysis Department 122 Instruction Set Architecture Analysis Unit 123 Calculation Unit 131 Execution Trace Database (DB) 132 Architecture Information DB 133 VM Execution Trace DB 1211 Execution trace acquisition unit 1212 VM instruction boundary detection unit 1213 Virtual Program Counter Detector 1214 Dispatcher Detector 1215 Conditional Branch Flag Detector 1216 Code Cache Detector 1221 VM Execution Trace Acquisition Unit 1222 VM Instruction Collection Unit 1223 VM instruction judgment unit 1231 Multipath Execution Unit 1232,1235 VM branch trace construction section 1233,1236 Control flow graph construction part 1234 Executive Department 1237 Code Coverage Calculation Unit

Claims

1. a first analysis unit that analyzes a virtual machine of the script engine; a second analysis unit that analyzes an instruction set architecture, which is a system of instructions for the virtual machine, collects virtual machine instructions, and determines the instruction contents of the collected virtual machine instructions; a calculation unit that constructs a first control flow graph showing all paths comprehensively executed by multi-pass execution and a second control flow graph showing paths executed in execution of a test target based on the architecture information acquired by the first analysis unit and the second analysis unit, and calculates code coverage of a script executed on the script engine based on the first control flow graph and the second control flow graph; An analytical function-imparting device comprising:

2. 2. The analysis function providing device according to claim 1, wherein the calculation unit calculates, as the code coverage, a ratio of the number of nodes and / or edges of the second control flow graph to the number of nodes and / or edges of the first control flow graph.

3. The analysis function-imparting device described in claim 1, characterized in that the first analysis unit analyzes multiple execution traces obtained by changing execution conditions using differential execution analysis, and detects a conditional branch flag, which is an area that holds a flag indicating whether a branch will be taken when a conditional branch occurs in the execution state.

4. The first analysis unit a first acquisition unit that acquires a plurality of execution traces by changing execution conditions; a first detector that clusters the execution trace to detect boundaries of each virtual machine instruction; a second detection unit that analyzes the execution traces using a differential execution analysis that focuses on the number of times memory is read and the boundaries of each virtual machine instruction detected by the first detection unit, and detects a virtual program counter that is a variable that indicates the next instruction of the virtual machine to be executed; a third detection unit that analyzes the binary of the script engine based on the boundaries of each virtual machine instruction detected by the first detection unit and detects a dispatcher; a fourth detection unit that analyzes the plurality of execution traces using differential execution analysis that focuses on the number of times memory is read, and detects a conditional branch flag that is an area that holds a flag indicating whether or not a branch is taken at the time of a conditional branch in the execution state; 4. The analysis function providing device according to claim 3, further comprising:

5. The second analysis unit a second acquisition unit that acquires a virtual machine execution trace, which is an execution trace executed in the virtual machine; a first collection unit that executes a test script while monitoring the virtual program counter and the dispatcher to acquire the virtual machine execution trace, and collects virtual machine instructions from the virtual machine execution trace; a first determination unit that determines whether a virtual machine instruction is a branch instruction based on variations in a change amount of a virtual program counter for each virtual machine opcode in the virtual machine execution trace; 5. The analysis function providing device according to claim 4, further comprising:

6. The calculation unit a first execution unit that executes a script to be analyzed in multiple passes while acquiring a virtual machine execution trace, which is an execution trace executed in the virtual machine, based on architecture information acquired by the first analysis unit and the second analysis unit; a second execution unit that executes a test target for the analysis target script while acquiring a virtual machine execution trace that is an execution trace executed in the virtual machine; a first construction unit that detects a branch virtual machine instruction from the virtual machine execution trace acquired by the first execution unit, constructs a first virtual machine branch trace that associates virtual program counters before and after execution of the detected branch virtual machine instruction, and detects a branch virtual machine instruction from the virtual machine execution trace acquired by the second execution unit, and constructs a second virtual machine branch trace that associates virtual program counters before and after execution of the detected branch virtual machine instruction; a second construction unit that constructs the first control flow graph based on the first virtual machine branch trace, with basic blocks as nodes and branches resulting from execution of branch virtual machine instructions as edges, and that constructs the second control flow graph based on the second virtual machine branch trace, with basic blocks as nodes and branches resulting from execution of branch virtual machine instructions as edges; a first calculation unit that calculates the code coverage of a script executed on the script engine based on the first control flow graph and the second control flow graph; 6. The analysis function providing device according to claim 5, further comprising:

7. An analysis function imparting method executed by an analysis function imparting device, a first analysis step of analyzing a virtual machine of a script engine; a second analysis step of analyzing an instruction set architecture, which is a system of instructions for the virtual machine, to collect virtual machine instructions and determine the instruction contents of the collected virtual machine instructions; a calculation step of constructing a first control flow graph showing the entire path comprehensively executed by multi-pass execution and a second control flow graph showing the path executed in the execution of the test target based on the architecture information acquired in the first analysis step and the second analysis step, and calculating code coverage of the script executed on the script engine for the entire path based on the first control flow graph and the second control flow graph; An analytical function imparting method comprising:

8. a first analysis step of analyzing a virtual machine of a script engine; a second analysis step of analyzing an instruction set architecture, which is a system of instructions for the virtual machine, to collect virtual machine instructions and determine the instruction contents of the collected virtual machine instructions; a calculation step of constructing a first control flow graph showing the entire path exhaustively executed by multi-pass execution and a second control flow graph showing the path executed in the execution of the test target based on the architecture information acquired in the first analysis step and the second analysis step, and calculating code coverage of the script executed on the script engine for the entire path based on the first control flow graph and the second control flow graph; A program that provides analytical functions to enable a computer to execute the above.

Images