Prediction system and prediction method

Abstract

To provide a prediction system and a prediction method that predict test execution time and coverage of a designed model.SOLUTION: A prediction system for predicting test execution time, which is configured with a computer having an arithmetic unit that executes a program and a storage device that is accessible by the arithmetic unit, where the storage device stores, according to the number of blocks constituting a model and complexity of the model, an operation result of a source code including test execution time and a coverage rate, and the arithmetic unit refers to the operation result and predicts at least one of a coverage rate at a target test execution time and test execution time at a target coverage rate from the number of blocks and complexity of a model to be analyzed.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to model-based development in which control logic is treated as a model in embedded software development, and in particular to a technique for verifying operation while simulating the model. [Background technology]

[0002] In recent years, the embedded software implemented in on-board electronic control units has become more diverse and complex, and the amount of work required to design and verify the embedded software is increasing. One software development method that can address this situation is model-based development, which treats control logic as a model.

[0003] This model-based development is generally carried out using modeling tools. However, to use modeling tools, it is necessary to use highly readable models that allow the operation specifications (processing) to be recognized from the model. Using highly readable models makes it possible to carry out verification through highly accurate simulations from the design stage, thereby reducing development man-hours.

[0004] Furthermore, in model-based development, source code can be automatically generated from a model. A typical method for verifying the automatically generated source code is to use the test cases used in the model simulation to perform back-to-back testing (hereafter referred to as B2B) to check whether the operation results of the model and the source code match.

[0005] The following prior art is included as background art in this technical field. Patent Document 1 (JP Patent Publication 2009-294846 A) describes a method for identifying a boundary between a subsystem including the block and a subsystem preceding the subsystem when a target value is set for output data (i.e., a test item) of one block in a model, and calculating a setting value for input data for the subsystem at the boundary so that the test item becomes the target value. The device further describes a test case generation device that sets the calculated setting value as a new target value for the boundary, sets the output data at the boundary as a new test item, and generates test cases for the model. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2009-294846 A Summary of the Invention [Problem to be solved by the invention]

[0007] In B2B testing, test cases are created from a model, so as the number of inputs and outputs and conditional branches increase and the model becomes more complex, there is a possibility that test coverage will decrease. Therefore, the challenge is whether B2B can be performed with high test coverage.

[0008] In the technology for generating test cases from a model described in Patent Document 1, the test coverage of a model to be verified decreases as the model becomes more complex, but by generating test cases from the output values ​​of the model, the decrease in test coverage is suppressed even if the model becomes more complex. However, in the above Patent Document 1, since test cases are generated by tracing back branches and elements from the output values ​​of the model, it is possible to generate test cases that satisfy 100% of instruction coverage (hereinafter referred to as C0) and branch coverage (hereinafter referred to as C1), which are one of the test coverages, but it is difficult to generate test cases that satisfy 100% of modified condition decision coverage (hereinafter referred to as MC / DC), which is the test coverage required by the functional safety standard, and there is a problem that tests of user conditions such as boundary value tests cannot be covered.

[0009] Therefore, the object of the present invention is to predict the completeness (test execution time, coverage) of a designed model without actually creating test cases by comparing a designed model with a previously created model at the model design stage before actually creating test cases, acquiring information on test execution time and coverage from a database of previously created models similar to the designed model, and predicting the test execution time and coverage of the designed model.Then, by reviewing the model design from the prediction results, a technology is provided that can create a highly complete model that satisfies high test coverage at the design stage. [Means for solving the problem]

[0010] A representative example of the invention disclosed in the present application is as follows: That is, a prediction system for predicting a test execution time is configured by a computing device that executes a program and a computer having a storage device accessible by the computing device, the storage device stores operation results of a source code including a test execution time and a coverage rate corresponding to the number of blocks constituting a model and the complexity of the model, and the computing device refers to the operation results and predicts at least one of the coverage rate at a target test execution time and the test execution time at the target coverage rate from the number of blocks and the complexity of the model to be analyzed. Effect of the Invention

[0011] According to one aspect of the present invention, a model design that satisfies high coverage can be achieved, and the number of steps required for reworking from the test process can be reduced. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 illustrates an example of the configuration of a computer in which a test case coverage prediction system based on performance data according to an embodiment is implemented. [Figure 1.1] FIG. 2 is a diagram showing an example of a block diagram of a model according to an embodiment; [Figure 1.2] FIG. 2 is a diagram illustrating an example of a database configuration according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a logical configuration of a coverage prediction system according to an embodiment. [Diagram 3] FIG. 2 illustrates an example of a subsystem configuration of a model information extraction unit according to the embodiment. [Figure 4] FIG. 13 is a diagram illustrating an example of a subsystem configuration of a model information verification data comparison unit in the embodiment. [Diagram 5] FIG. 13 illustrates an example of the configuration of a verification data report generating unit in the embodiment. [Figure 6] 11 is a flowchart of an example of processing executed by a model structure analysis unit of the embodiment. [Figure 7]11 is a flowchart of an example of processing executed by an exclusion target block extraction unit of the embodiment. [Figure 8] 13 is a flowchart of an example of processing executed by a model information verification unit of the embodiment. [Figure 9] 11 is a flowchart of an example of a process executed by a target setting data extraction unit of the embodiment. [Figure 10] 11 is a flowchart of an example of a process executed by a prediction data comparison unit in the embodiment. [Figure 11] 11 is a flowchart of an example of a process executed by a proposal data comparison unit in the embodiment. [Figure 12] 13 is a flowchart of an example of a process executed by a verification data editing unit of the embodiment. [Figure 13] 13 is a flowchart of an example of a process executed by a report file output unit of the embodiment. [Figure 14] FIG. 13 is a diagram illustrating an example of a configuration of a verification data prediction result report in the embodiment. [Figure 15] FIG. 13 is a diagram illustrating an example of data acquisition using an exclusion target block setting table according to the embodiment. [Figure 16] FIG. 13 is a diagram showing an example of data acquisition using a target setting data table according to an embodiment. [Figure 17] FIG. 13 is a diagram illustrating an example of a configuration of a prediction test execution time acquisition table according to the embodiment. [Figure 17.1] FIG. 13 is a diagram illustrating an example of acquiring a test execution time using a prediction test execution time acquisition table according to the embodiment. [Figure 18] FIG. 13 is a diagram illustrating an example of a configuration of a prediction coverage rate acquisition table in the embodiment. [Figure 19] FIG. 13 is a diagram illustrating an example of a configuration of a proposed test execution time acquisition table in the embodiment. [Figure 20] FIG. 13 is a diagram illustrating an example of the configuration of a proposal coverage rate acquisition table in the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same reference numerals denote the same parts.

[0014] Furthermore, numbers preceded by the letter S in the text (e.g. S601) indicate the corresponding process in the data flow.

[0015] <Example 1> An example of the configuration of a computer on which a test case coverage prediction system 203 based on performance data according to an embodiment is implemented will be described with reference to FIG.

[0016] The test case coverage prediction system 203 based on performance data is a computer system that operates on a computer 109 having a processor (CPU) 102 , a memory 101 , an auxiliary storage device 103 , a communication interface 106 , an input interface 104 , and an output interface 105 .

[0017] The processor (CPU) 102 has a calculation device that executes an execution program temporarily stored in the memory 101 and reads and writes data (e.g., model 107) used during program execution, and the processor 102 executes various programs to realize a test case coverage prediction system 203 based on performance data. Note that some of the processes performed by the processor 102 executing the programs may be executed by other calculation devices (e.g., hardware such as ASIC and FPGA).

[0018] The memory 101 has a high-speed and volatile storage element such as a dynamic random access memory (DRAM), and temporarily stores programs executed by the processor 102 and data (e.g., model 107) used when the programs are executed.

[0019] The auxiliary storage device 103 has a large capacity non-volatile storage device such as a magnetic storage device (HDD) or a flash memory (SSD), and stores data used by the processor 102 when executing a program (such as data obtained from the database 202 and expanded in the memory 101) and the program executed by the processor 102. That is, the program is read from the auxiliary storage device 103, loaded into the memory 101, and executed by the processor 102 to realize each function of the performance data-based test case coverage prediction system 203.

[0020] The communication interface 106 is a network interface device that controls communication with other devices according to a predetermined protocol. The communication interface 106 may be used to read the model 107 and the database 202 stored in the auxiliary storage device of another computer on the network.

[0021] The input interface 104 is an interface that receives input from a user, such as a keyboard or a mouse. For example, the input interface 104 receives input of a model 107. The input interface 104 may also provide a GUI and receive data registration to the database 202 from the user.

[0022] The output interface 105 is an interface that outputs the execution result of a program in a format that is visible to a user, such as a display device or a printer. For example, in the performance data-based test case coverage prediction system 203 of the first embodiment, the output interface 105 outputs or displays data such as a predicted test time and a coverage rate. The output interface 105 may also be a data output port that outputs the execution result of a program in a format that is visible to a user. For example, the output interface 105 may output a report in a predetermined data format (e.g., pdf, html) that displays data such as a predicted test time and a coverage rate output by the performance data-based test case coverage prediction system 203.

[0023] An example of the block diagram configuration of model 107 will be described with reference to Figure 1.1.

[0024] The model 107 is input to a performance data-based test case coverage prediction system 203. The performance data-based test case coverage prediction system 203 analyzes the model structure of the input model 107. The analysis results provide model metrics (number of blocks, complexity) information. The model 107 is also simulable, and operates according to the settings of the blocks in the model 107.

[0025] Model 107 shown in Figure 1.1 is composed of input port blocks 1101, 1102, 1104, constant blocks 1103 and 1106 that output fixed values, a multiplication block 1109 that multiplies input values, an addition block 1108 that adds input values, a branch block 1105 that switches processing based on the input value, a counter block 1107 that performs counting processing based on the input value, and an output port block 1110.

[0026] An example of the configuration of the database 202 will be described with reference to FIG.

[0027] The database 202 is composed of an exclusion target block storage unit 304 , a target setting data storage unit 404 , a prediction verification data storage unit 405 , and a proposal verification data storage unit 406 .

[0028] The database 202 is connected to a performance data-based test case coverage prediction system 203, and information required for the performance data-based test case coverage prediction system 203 is registered. In addition, it is preferable to register the registration data in advance before the program is executed.

[0029] The database 202 is a collection of data that can be registered, updated, and searched, and the test case coverage prediction system 203 based on performance data manages the data in a table.

[0030] The excluded block storage unit 304 includes an excluded block setting table T1501 in which information on block types to be excluded during coverage measurement is registered.

[0031] The target setting data storage unit 404 includes a target setting data table T1601 in which target B2B test execution time and coverage rate information are registered.

[0032] The prediction verification data storage unit 405 includes a prediction test execution time acquisition table T1701 in which model metrics information of past models for which B2B was implemented and the test execution time at that time are registered, and a prediction coverage rate acquisition table T1801 in which model metrics information of past models for which B2B was implemented and the coverage rate at that time are registered.

[0033] The proposed verification data storage unit 406 includes a proposed test execution time acquisition table T1901 in which model metrics information, coverage rate, and test execution time at that time of past models for which B2B was implemented are registered, and a proposed coverage rate acquisition table T2001 in which model metrics information, coverage rate, and coverage rate at that time of past models for which B2B was implemented are registered.

[0034] An example of a logical configuration of a system 203 for predicting test case coverage based on performance data will be described with reference to FIG.

[0035] The test case coverage prediction system 203 based on performance data is connected to a database 202 and is composed of a model information extraction unit 204 that analyzes the model 107 input by a user 201 and acquires the model structure, a model information verification data comparison unit 205 that acquires prediction data from the database 202, and a verification data report creation unit 206 that edits the acquired information and outputs a report 108.

[0036] The coverage prediction system 203 has a function of predicting the B2B test execution time and coverage rate of the model 107 and outputting the prediction result as a report. The coverage prediction system 203 can execute the coverage prediction process of the test case according to the model 107 input by the user 201.

[0037] An example of the subsystem configuration of the model information extraction unit 204 of the system 203 for predicting test case coverage based on performance data will be described with reference to FIG.

[0038] The model information extraction unit 204 is connected to the database 202 and is composed of a model structure analysis unit 301 that analyzes the input model 107 and acquires model metrics, an excluded block extraction unit 302 that acquires block type information to be excluded from coverage measurement registered in an excluded block storage unit 304, and a model information verification unit 303 that searches the model 107 for paths with a complexity of 50 or more (hereinafter, bottleneck paths) that are obstacles to achieving a coverage rate of 100%. The information acquired by the model information extraction unit 204 during processing is stored in the memory 101.

[0039] The model information extraction unit 204 executes a process of acquiring model information and excluded block types required for execution from the input model 107 and the database 202 connected to the coverage prediction system 203, and a process of storing the acquired model metrics data, excluded block data, and bottleneck path data in the memory 101. The model information extraction unit 204 also acquires information from an excluded block storage unit 304 in the database 202 connected to the input model 107, and executes a model information extraction process.

[0040] An example of processing executed by the model structure analysis unit 301 of the model information extraction unit 204 will be described with reference to FIG.

[0041] The model structure analysis unit 301 receives the input model 107, reads model information from the model 107 (S601), and extracts the hierarchical information of the model and the block types arranged for each hierarchy (S602). The model structure analysis unit 301 stores the extracted data in the model information data (S603).

[0042] In model information data loop processing S604, the model structure analysis unit 301 starts repetitive processing (S605 to S607) the number of times corresponding to the number of data items stored in the model information data.

[0043] The model structure analysis unit 301 calculates the number of blocks and the complexity for each layer from the layer information and block type stored in the model information data (S605, S606). The model structure analysis unit 301 stores the calculated data in the model metrics data (S607). After storing the data, the model structure analysis unit 301 determines whether or not the loop has been completed for the number of stored items of model information data (S608), and if the loop has been completed for the number of stored items of model information data (S608: Yes), the process ends.

[0044] With reference to FIG. 1.1, an example of obtaining the hierarchical information, number of blocks, and complexity of a model when the model 107 is input will be described.

[0045] The hierarchical information means the hierarchy (path) of the model. Since the model 107 has one hierarchy, the model name is acquired. For example, if the model 107 has multiple hierarchical levels, the hierarchical information can be acquired as model name / hierarchical name.

[0046] The model type means attribute information of a block. In the case of the input port block 1101 of the model 107, the input port is acquired as the block type.

[0047] The number of blocks is the number of blocks in the model. For example, the number of blocks 1101 to 1110 arranged in the model 107 is acquired as 10.

[0048] The complexity is calculated based on the number of processing elements in the block and the number of input / output branches, and the higher the complexity number, the larger and more complex the model. For example, in the case of model 107, there is a branch block 1105 with 2 branches, and there is processing inside counter block 1107, so the complexity is calculated as 5.

[0049] The acquired information is stored as follows: hierarchical information: model 107, block type: blocks 1101 to 1110 in the model information data; and hierarchical information: model 107, number of blocks: 10, complexity: 5 in the model metrics data.

[0050] The process executed by the exclusion target block extraction unit 302 of the model information extraction unit 204 will be described with reference to FIG.

[0051] The exclusion target block extraction unit 302 acquires (S701) the block type and block type complexity registered in the exclusion target block type T1502 of the exclusion target block setting table T1501 (see FIG. 15) stored in the exclusion target block storage unit 304. The exclusion target block extraction unit 302 stores the acquired block type and block type complexity in the exclusion target block data (S702), and ends the process.

[0052] An example of data acquisition using the exclusion block setting table T1501 will be described with reference to FIG.

[0053] There are no conditions for obtaining data in the excluded block setting table T1501, and all registered data is obtained. For example, since the block types, counter and timer, and the block type complexity T1503 in the excluded block setting table T1501 are registered as counters and timers, the information on the excluded block type T1502 (counter and timer) and the information on the block type complexity T1503 (complexity of counter and timer) are obtained from the excluded block setting table T1501 and stored in the excluded block data.

[0054] The process executed by the model information verification unit 303 of the model information extraction unit 204 will be described with reference to FIG.

[0055] The model information verification unit 303 searches for data that matches the block type stored in the block data to be excluded from among the data stored in the model information data (S801). If a matching block type is found in the model information data (S802: YES), the model information verification unit 303 stores the hierarchical information and block type of the found model information data in the block data to be excluded (S803). Next, the model information verification unit 303 compares the complexity of the model metrics data (S804), and if the complexity is equal to or greater than a predetermined threshold (e.g., 50) (S805: YES), it determines that the path is a bottleneck path that is difficult to achieve a coverage rate of 100% and is a factor of increasing the test time, stores the target hierarchical information, block type, and complexity in the bottleneck data (S806), and ends the process.

[0056] A process of searching for block data to be excluded from model information data will be described. For example, when model 107 is input, the model information data is registered with the block types of hierarchical information: model 107 and model type: blocks 1101 to 1110, and the block data to be excluded is stored with block type: counter, timer, and block type complexity: counter and timer complexity. Since the model information includes counter block 1107, it is determined to be a block to be excluded, and the excluded block data is stored with hierarchical information: model 107, block type: counter block 1107, and block type complexity of the block data to be excluded: counter complexity: 3. Next, a bottleneck block determination will be described. Regarding model 107, hierarchical information: model 107, number of blocks: 10, and complexity: 5 are stored in the input model metrics data. Since the complexity of model 107 is 50 or less, it is not a bottleneck target, and the data is not stored in the bottleneck data.

[0057] An example of the subsystem configuration of the model information verification data comparison unit 205 of the test case coverage prediction system 203 based on performance data will be described with reference to FIG.

[0058] The model information verification data comparison unit 205 is composed of a target setting data extraction unit 401 that is connected to the database 202 and acquires B2B target information registered in a target setting data storage unit 404, a prediction data comparison unit 402 that acquires prediction data of the test execution time and the coverage rate from past B2B performance data registered in a prediction verification data storage unit 405, and a proposal data comparison unit 403 that acquires proposal data of the test execution time and the coverage rate from past B2B performance data registered in a proposal verification data storage unit 406. The information acquired by the model information verification data comparison unit 205 during processing is stored in the memory 101.

[0059] The model information verification data comparison unit 205 has a process of acquiring model metrics data and target setting data required for execution from the database 202 connected to the memory 101, and a process of storing the acquired prediction data, proposal data, and bottleneck path excluded prediction data in the memory 101. The model information verification data comparison unit 205 acquires information from the target setting data storage unit 404, the prediction verification data storage unit 405, and the proposal verification data storage unit 406 of the database 202 connected to the memory 101, and executes the model information verification data comparison process.

[0060] The process executed by the target setting data extraction unit 401 of the model information verification data comparison unit 205 will be described with reference to FIG.

[0061] The target setting data extraction unit 401 acquires the target test execution time and the target coverage rate registered in the target test execution time T1602 and the target coverage rate T1603 from the target setting data table T1601 (see FIG. 16) stored in the target setting data storage unit 404 (S901). The target setting data extraction unit 401 stores the acquired target test execution time and the target coverage rate in the target setting data (S902), and ends the process.

[0062] An example of data acquisition using the target setting data table T1601 will be described with reference to FIG.

[0063] There are no data acquisition conditions in the target setting data table T1601, and all registered data is acquired. For example, in the target setting data table T1601, 10 is registered in the target test execution time T1602 and 100 is registered in the target coverage rate T1603, and the target test execution time and target coverage time (10, 100) are acquired from the target setting data table T1601 and stored in the target setting data.

[0064] The process executed by the prediction data comparison unit 402 of the model information verification data comparison unit 205 will be described with reference to FIG.

[0065] The prediction data comparison unit 402 determines whether data is stored in the excluded block data (S1001). If data is stored in the excluded block data (S1001: YES), the prediction data comparison unit 402 calculates the number of blocks from the block type stored in the excluded block data, and subtracts the calculated number of blocks from the number of blocks stored in the model metrics data. If the model metrics data includes a target block type (hierarchy) of the excluded block data, the prediction data comparison unit 402 calculates the complexity by subtracting the block type complexity of the excluded target block type stored in the excluded block data from the complexity of the hierarchy stored in the model metrics data (S1002).

[0066] For example, if the model metrics data for model 107 is registered as hierarchical information: model 107, number of blocks: 10, and complexity: 5, and the excluded block data is registered as hierarchical information: model 107, block type: counter block 1107, and block type complexity of the block data to be excluded: counter complexity: 3, then the number of blocks is 9, subtracting one block type of counter block 1107, and the complexity is calculated as 2, obtained by subtracting the complexity of the model metrics (5) from the complexity of the block data to be excluded (3).

[0067] The calculated number of blocks and complexity are stored in the prediction search data (S1003). On the other hand, if no data is stored in the excluded block data (S1001: NO), the prediction data comparison unit 402 stores the number of blocks and complexity of the model metrics data in the prediction search data (S1004).

[0068] Then, the prediction data comparison unit 402 compares the number of blocks and complexity of the stored prediction search data with the data of the number of blocks T1702 and complexity T1703 registered in the prediction test execution time acquisition table T1701 (see Figure 17) stored in the prediction verification data storage unit 405.

[0069] If the calculated number of blocks and complexity of the prediction search data match the registered number of blocks and complexity, the prediction data comparison unit 402 acquires the test execution time of the matching data (S1005). If the calculated number of blocks and complexity do not match the registered number of blocks and complexity, the prediction data comparison unit 402 acquires the test execution time by linear interpolation from the registered number of blocks and complexity (S1005).

[0070] Now, with reference to FIG. 17.1, an example of acquiring a test execution time using the predicted test execution time acquisition table T1701 (hereinafter referred to as table T1701) will be described below in (1) and (2).

[0071] (1) If the number of blocks and complexity obtained from model 107 match the number of blocks and complexity registered in table T1701 When the number of blocks and complexity obtained from model 107 are 10 and 2, matching data of the number of blocks and complexity of 10 and 2 is registered in table T1701, so the test execution time of 3 is obtained from the intersection of the data.

[0072] (2) The number of blocks obtained from model 107 does not match the number of blocks and complexity registered in table T1701. When the number of blocks obtained from model 107 is 20 and complexity: 30, since no matching data is registered in table T1701, it is determined which data in table T1701 the number of blocks: 20 and complexity: 30 fall between. First, since the number of blocks: 20 is data between the two points of the number of blocks: 10 and 50, the test execution time for complexity: 20 and 50 is calculated using this as the base point using the following linear interpolation formula.

[0073] <Linear Interpolation Formula> Y=Y1+((Y2-Y1)÷(X2-X1))×(X-X1)

[0074] From the above linear interpolation formula, the test execution time when the number of blocks is 20 and the complexity is 20 can be calculated as follows. In this case, Y is the desired test execution time, X is the number of blocks in model 107, X1 is the number of blocks in table T1701: 10, X2 is the number of blocks in table T1701: 50, Y1 is the test execution time when the number of blocks in table T1701 is 10 and the complexity is 20: 40, and Y2 is the test execution time when the number of blocks in table T1701 is 50 and the complexity is 20: 60.

[0075] Y=40+((60-40)÷(50-10))×(20-10) =40+(20÷40)×10 =45

[0076] Next, in a similar manner, calculate the test execution time when the number of blocks is 20 and the complexity is 50. In this case, Y is the desired test execution time, X is the number of blocks in model 107, X1 is the number of blocks in table T1701: 10, X2 is the number of blocks in table T1701: 50, Y1 is the test execution time when the number of blocks in table T1701 is 10 and the complexity is 50: 70, and Y2 is the test execution time when the number of blocks in table T1701 is 50 and the complexity is 50: 90.

[0077] Y=70+((90-70)÷(50-10))×(20-10) =70+(20÷40)×10 =75

[0078] Next, the test execution time for complexity level 30 is calculated from the test execution times for complexity levels 20 and 50 for number of blocks 20 calculated using the above formula using a similar linear interpolation formula.

[0079] In this case, Y is the desired test execution time, X is the number of blocks in model 107, X1 is the number of blocks in table T1701: 20, X2 is the number of blocks in table T1701: 50, Y1 is the test execution time when the number of blocks in table T1701: 20 and complexity: 20: 45, and Y2 is the test execution time when the number of blocks in table T1701: 20 and complexity: 50: 75.

[0080] Y=45+((75-45)÷(50-20))×(30-20) =45+(30÷30)×10 =55

[0081] From the above calculation results, we can predict the test execution time: 55 from the model metrics of model 107.

[0082] Next, the prediction data comparison unit 402 compares the number of blocks and the complexity of the prediction search data with the data of the number of blocks T1802 and the complexity T1803 registered in a prediction coverage rate acquisition table T1801 (see FIG. 18).

[0083] If the calculated number of blocks and complexity of the predictive search data match the registered number of blocks and complexity, the predictive data comparison unit 402 acquires the coverage rate of the matching data (S1005). If the number of blocks and complexity of the predictive search data do not match the registered number of blocks and complexity, the coverage rate is acquired by linear interpolation from the registered number of blocks and complexity (S1005).

[0084] Although the reference table is different, the coverage rate can be obtained from the predicted coverage rate acquisition table T1801 using the same logic as that for obtaining data from the predicted test execution time acquisition table T1701, and therefore a description thereof will be omitted.

[0085] Thereafter, the calculation data comparison unit 402 stores the acquired test execution time and coverage rate in the calculation data (S1006).

[0086] Next, if data is stored in the bottleneck data (S1007: YES), the prediction data comparison unit 402 calculates the number of blocks and complexity of the bottleneck data minus the number of blocks and complexity of the prediction search data, respectively (S1008).

[0087] After that, the prediction data comparator 402 compares the calculated data with the data of the number of blocks T1702 and the complexity T1703 registered in the prediction test execution time acquisition table T1701 (FIG. 17). If the calculated data matches the registered number of blocks and complexity, the test execution time of the matching data is acquired. If the calculated data does not match the registered number of blocks and complexity, the test execution time is acquired by linear interpolation from the registered number of blocks and complexity (S1009).

[0088] Similarly, the prediction data comparison unit 402 compares the calculated data with the data of the number of blocks T1802 and the complexity T1803 registered in the prediction coverage rate acquisition table T1801 (FIG. 18). If the calculated data matches the registered number of blocks and complexity, the coverage rate of the matching data is acquired. If the calculated data does not match the registered number of blocks and complexity, the coverage rate is acquired by linear interpolation from the registered number of blocks and complexity (S1009).

[0089] The method of obtaining the test time and coverage rate by linear interpolation here uses the same logic as described above, so the explanation will be omitted.

[0090] The obtained test execution time and coverage rate are stored as prediction data for bottleneck path removal (S1010), and the process ends.

[0091] An example of processing executed by the proposal data comparison unit 403 of the model information verification data comparison unit 205 will be described with reference to FIG.

[0092] The proposed data comparison unit 403 compares the number of blocks, complexity, and coverage rate of the target data of the prediction search data with the data of the number of blocks T1902, complexity T1903, and coverage rate T1904 registered in the proposed test execution time acquisition table T1901 (see Figure 19) stored in the proposed verification data storage unit 406.

[0093] If the number of blocks, complexity, and coverage rate of the prediction search data match the registered number of blocks, complexity, and coverage rate, the proposal data comparison unit 403 acquires the test execution time of the matching data (S1101). If the number of blocks, complexity, and coverage rate of the prediction search data and the target data do not match the registered number of blocks, complexity, and coverage rate, the proposal data comparison unit 403 acquires the test execution time by linear interpolation from the registered number of blocks, complexity, and coverage rate (S1101). The proposal data comparison unit 403 stores the acquired test execution time in the proposal data (S1102). The test execution time stored in the proposal data is output to the user.

[0094] In addition, the proposed data comparison unit 403 compares the number of blocks, complexity, and test execution time of the target data of the prediction search data with the verification data of the number of blocks T2002, complexity T2003, and test execution time T2004 of the verification data registered in the proposed coverage rate acquisition table T2001 (see Figure 20) stored in the proposed verification data storage unit 406.

[0095] If the predicted search data and the registered number of blocks, complexity, and test execution time match, the proposed data comparison unit 403 acquires the coverage rate of the matching data (S1103). If the predicted search data and the registered number of blocks, complexity, and test execution time do not match, the proposed data comparison unit 403 acquires the coverage rate by linear interpolation using the registered number of blocks, complexity, and test execution time (S1103). The proposed data comparison unit 403 stores the acquired coverage rate in the proposed data (S1104) and ends the process. The coverage rate stored in the proposed data is output to the user.

[0096] In this way, the proposed data comparison unit 403 can propose a test execution time to the user based on the target coverage rate even if the target coverage rate cannot be obtained with the target test execution time, and can propose a coverage rate to the user based on the target test execution time even if the target coverage rate cannot be obtained with the target test execution time.

[0097] In addition, when obtaining the test execution time and coverage rate from the proposed test execution time acquisition table T1901 and the proposed coverage rate acquisition table T2001, the table configuration used is different between a two-dimensional table (Figure 17.1) and a three-dimensional table (Figure 20), but since the acquisition logic is the same, the explanation will be omitted.

[0098] An example of the configuration of the verification data report generating unit 206 of the system 203 for predicting test case coverage based on performance data will be described with reference to FIG.

[0099] The verification data report generator 206 includes a verification data editor 501 that edits the data stored in the memory 101 into a report format, and a report file output unit 502 that outputs the edited data as the report 108.

[0100] The verification data report creation unit 206 executes a process of acquiring model metrics data, excluded block data, bottleneck data, target setting data, prediction data, bottleneck removal prediction data, and proposed data required for execution from the memory 101, and a process of outputting edited data to the report 108. The verification data report creation unit 206 can execute the verification data report creation process by acquiring information stored in the memory 101.

[0101] The process executed by the verification data editing unit 501 of the verification data report generator 206 will be described with reference to FIG.

[0102] In the stored data loop process S1201, the verification data editing unit 501 starts a loop process of S1202 to S1203 for the stored model metrics data, excluded block data, prediction data, bottleneck data, prediction data for bottleneck removal, proposal data, and target setting data, the same number of times as the stored items.

[0103] The verification data editing unit 501 edits the stored data (S1202) and stores it in the edited data (S1203) in order to create a verification data prediction result report (see FIG. 14) including descriptions of a model metrics information section R1401, an excluded block type section R1402, a predicted test information section R1403, a predicted test information section R1404 when a bottleneck path is excluded, and a proposed test information section R1405.

[0104] (1) Model Metrics Information Section R1401 The model metrics information section R1401 has the title "■ Model metrics information". The number of blocks stored in the model metrics data is recorded in the item name "Number of blocks", and the complexity stored in the model metrics data is recorded in the item name "Complexity".

[0105] (2) Exclusion block type part R1402 The excluded block type section R1402 has the title "■ excluded block type" written therein, and contains information combining the hierarchical information and block type stored in the excluded block data.

[0106] (3) Predictive Test Information Unit R1403 The predicted test information section R1403 has a title: "■ Predicted test information". The test execution time stored in the predicted data is written in the item name: "Test execution time [min]", and the coverage rate stored in the predicted data is written in the item name: "Coverage rate [%]".

[0107] (4) Prediction test information when excluding bottleneck paths R1404 The predicted test information when bottleneck path is excluded section R1404 has the title: "■ Predicted test information when bottleneck path is excluded." The item name: "Bottleneck path" has the hierarchical information stored in the bottleneck data, the item name: "Test execution time when excluded [min]" has the test execution time stored in the prediction data when bottleneck is removed, and the item name: "Coverage rate when excluded [%]" has the coverage rate stored in the prediction data when bottleneck is removed.

[0108] (5) Proposal Test Information Section R1405 The proposed test information section R1405 has a title: "■ Proposed test information". The target test execution time of the target setting data is recorded in the item name: "Target test execution time [min]", the predicted coverage rate stored in the proposed data is recorded in the item name: "Predicted coverage rate [%]", the target coverage rate stored in the target setting data is recorded in the item name: "Target coverage rate [%]", and the predicted test execution time stored in the proposed data is recorded in the item name: "Predicted test execution time [min]".

[0109] For example, when model 107 is input, the model metrics information section R1401 stores model metrics data with the number of blocks: 10 and complexity: 5, so that the model metrics information section R1401 is written as shown in the figure. ■Model Metrics Information Number of blocks: 10 Complexity: 5

[0110] For other items as well, the report 108 is compiled according to the following logic.

[0111] For example, when hierarchical information: model 107 and block type: counter block 1107 are stored in the excluded block data, excluded block type R 1402 is recorded as shown in the figure. ■Exclusion block type Model 107 / Canterblock 1107

[0112] Also, for example, when the predicted data stores a test execution time of 2.8 and a coverage rate of 100, predicted test information R1403 is written as shown in the figure. ■ Predictive test information Test execution time [min]: 2.8 Coverage rate [%]: 100

[0113] Also, for example, if the bottleneck data stores hierarchical information: model 107 / table, test execution time of prediction data when bottleneck is removed: 10.0, and coverage rate of prediction data when bottleneck is removed: 98, predicted test information R1404 when the bottleneck path is excluded is recorded as shown in the figure. ■Predictive test information when bottleneck paths are excluded Bottleneck Path Model 107 / Table Exclusion test execution time [min]: 10.0 Exclusion coverage rate [%]: 98

[0114] Also, for example, if the target setting data has a target test execution time of 5.0 and a target coverage rate of 100, and the proposed data has a predicted test execution time of 30.0 and a predicted coverage rate of 80, the proposed test information R1405 is written as shown in the figure. ■ Proposed test information Target test execution time [min]: 5.0 Predicted coverage rate [%]: 80 Target coverage rate [%]: 100 Target test execution time [min]: 30.0

[0115] If the loop has been completed for the number of stored data items (S1204: YES), the process ends.

[0116] The process executed by the report file output unit 502 of the verification data report generator 206 will be described with reference to FIG.

[0117] The report file output unit 502 starts the process of S1302 for the number of data items stored in the edited data in edited data loop process S1301, and outputs the edited data to the report 108 (S1302). When the loop for the number of edited data items is completed (S1303: YES), the process ends.

[0118] As described above, according to the embodiment of the present invention, it is possible to predict the execution time and test coverage of a test case at the design stage without actually creating the test case. This makes it possible to review the model design based on the prediction results and to design a model that achieves high test coverage, thereby reducing the number of reworks required from the test process.

[0119] The present invention is not limited to the above-described embodiments, and includes various modified examples and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the configurations described. Furthermore, a part of the configuration of one embodiment may be replaced with a configuration of another embodiment. Furthermore, a configuration of another embodiment may be added to a configuration of one embodiment. Furthermore, a part of the configuration of each embodiment may be added, deleted, or replaced with another configuration.

[0120] Furthermore, each of the aforementioned configurations, functions, processing units, processing means, etc. may be realized in hardware, for example by designing some or all of them as an integrated circuit, or may be realized in software by a processor interpreting and executing a program that realizes each function.

[0121] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC card, an SD card, or a DVD.

[0122] In addition, the control lines and information lines shown are those considered necessary for the explanation, and do not necessarily show all the control lines and information lines necessary for implementation. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0123] 101 Memory 102 Processor (CPU) 103 Auxiliary storage device 104 Input Interface 105 Output Interface 106 Communication Interface 107 Model 108 Report 109 Calculator 201 users 202 Database 203 Test Case Coverage Prediction System Based on Actual Data 204 Model Information Extraction Unit 205 Model Information Verification Data Comparison Unit 206 Verification Data Report Creation Department 301 Model Structural Analysis Department 302 Excluded block extraction unit 303 Model Information Verification Department 304 Exclusion block memory 401 Goal Setting Data Extraction Unit 402 Prediction Data Comparison Unit 403 Proposal Data Comparison Section 404 Target setting data storage unit 405 Prediction verification data storage unit 406 Proposal verification data storage unit 501 Verification Data Editorial Department 502 Report file output section 1101~1110 Control block

Claims

1. 1. A prediction system for predicting test execution time, comprising: A computer is configured with an arithmetic unit that executes a program and a storage device that can be accessed by the arithmetic unit, the storage device stores the operation results of the source code, including the test execution time and the coverage rate, in accordance with the number of blocks constituting the model and the complexity of the model; The calculation device refers to the operation results and predicts at least one of the coverage rate at the target test execution time and the test execution time at the target coverage rate based on the number of blocks and complexity of the model to be analyzed.

2. The prediction system according to claim 1 , The prediction system is characterized in that the calculation device outputs metrics information including the number of blocks used in the prediction and the complexity.

3. The prediction system according to claim 1 , The prediction system is characterized in that, when the complexity of the model is equal to or greater than a predetermined threshold, the calculation device extracts model metric information and the type of the block as a bottleneck path.

4. The prediction system according to claim 3, The prediction system is characterized in that the calculation device excludes blocks of the type extracted as the bottleneck path from the model to be analyzed, and predicts at least one of a coverage rate at a target test execution time and a test execution time at the target coverage rate.

5. The prediction system according to claim 1 , The computing device includes: If the coverage rate target for the target test execution time cannot be achieved, suggest a coverage rate that can be achieved for the target test execution time; A prediction system characterized by proposing a target test execution time that can achieve a target coverage rate when a target test execution time for the target coverage rate cannot be achieved.

6. The prediction system according to claim 1 , The computing device includes: Get the blocks to be excluded, A prediction system characterized by referring to the operation results and predicting at least one of the coverage rate at a target test execution time and the test execution time at the target coverage rate based on the number of blocks excluding the acquired exclusion target blocks and the complexity of the model to be analyzed.

7. 1. A method for predicting a test execution time by a prediction system, comprising: the prediction system is configured with a computer having an arithmetic unit that executes a program and a storage device that is accessible by the arithmetic unit, the storage device stores the operation results of the source code, including the test execution time and the coverage rate, in accordance with the number of blocks constituting the model and the complexity of the model; The prediction method includes: The prediction method includes a step in which the calculation device refers to the operation result and predicts at least one of a coverage rate at a target test execution time and a test execution time at a target coverage rate from the number of blocks and complexity of a model to be analyzed.

Images