Software integration test execution method and computer

Abstract

To preferentially and efficiently perform integration test for control according to importance.SOLUTION: A computer is configured to: store, in a control component storage unit, a partial control model with importance set thereto, a model source generated from the partial control model, unit test patterns used for equivalence verification conducted on the partial control model and the model source with input values covering all the processes in the model source, and a unit test execution time required for executing the unit test patterns; receive an overall control specification model combined with the partial control model, and software satisfying overall control specifications; collect, from the control component storage unit, unit test patterns of the partial control model in the overall control specification model, and the unit test execution time; acquire importance of the unit test patterns to schedule tests of software components in descending order of the importance; and execute tests of the software satisfying the overall control specifications according to the result of the scheduling.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a technique for performing integration testing of software configured by combining a plurality of control software components in model-based development for controlling the operation of equipment. [Background technology]

[0002] In recent years, in the development of embedded software, developers are increasingly writing software in "models," which are easier to create and have good visibility, instead of writing source code. This development method is called model-based development.

[0003] An example of a program development environment that supports model-based development is MATLAB (registered trademark) by MathWorks. Developers describe models using Simulink, a function of MATLAB (registered trademark). Models are described by functional units called blocks and connections that indicate input / output relationships between the blocks.

[0004] Based on this model, source code is generated using, for example, TargetLink from dSpace. This source code is called the model source. In addition, to verify the equivalence of the processing contents of the model and the model source and to check the code coverage, for example, Embedded Platform from BTC is used.

[0005] In this way, software that satisfies the entire control specification of the target device is often developed by combining multiple model sources generated from models that express partial control specifications of the target device. This makes it possible to create software more efficiently than creating software from scratch.

[0006] On the other hand, when developing new software by combining multiple model sources, it is necessary to guarantee that the control after combining the multiple model sources functions according to the required specifications. Software that is efficiently designed and developed using model-based development requires more productive testing methods for integration testing.

[0007] In Patent Document 1, by reusing unit test patterns prepared in advance for software with complex and large-scale control specifications, it is possible to automatically execute measurement tests of function call coverage and function coverage for each control component, and regression tests for each partial control model. This makes it possible to reduce the number of steps required to execute each test and output the results. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Patent Publication No. 2022-96125 Summary of the Invention [Problem to be solved by the invention]

[0009] The method of Patent Document 1 performs integration testing by sequentially executing unit test patterns prepared in advance. Therefore, if the scale of the software to be tested is large, the scale of the test patterns is expected to be large accordingly, and it is necessary to wait until the execution of all test patterns is completed before judging the test results.

[0010] If the software contains a defect, further program correction and testing will be required, so the effect of reducing software development time is not optimal. As a countermeasure, a solution using parallel distributed processing using multiple computers can be considered.

[0011] However, because the execution time of the test cases depends on the contents of the test pattern, and there is a large difference in the total execution time of each processing device, it is desirable to perform the tests in the execution order that is the critical path.In addition, it is desirable to be able to exclude from testing those whose control specifications are less important in order to prioritize the development schedule. SUMMARY OF THE PRESENT DISCLOSURE The present invention has been made in consideration of the above problems, and has an object to perform software integration tests preferentially and efficiently according to the level of importance. [Means for solving the problem]

[0012] The present invention is a software integration test execution method for performing integration testing of software components using a computer including a processor and a memory, comprising a first step of storing in a control component storage unit software components including a partial control model describing a partial control specification with a preset importance level, a model source generated from the partial control model, a unit test pattern for performing equivalence verification of the partial control model and the model source using input values ​​covering processes including branches in the model source, and a unit test execution time required for executing the unit test pattern, and a second step of storing in a control component storage unit software components including a partial control model describing a partial control specification with a preset importance level, a model source generated from the partial control model, a unit test pattern for performing equivalence verification of the partial control model and the model source using input values ​​covering processes including branches in the model source, and a unit test execution time required for executing the unit test pattern, and a second step of receiving software satisfying an overall control specification and a software component satisfying the overall control specification; a third step of collecting from the control component storage unit the unit test patterns of the partial control models included in the overall control specification model and unit test execution times required to execute the unit test patterns; a fourth step of the computer acquiring importance of the unit test patterns of the partial control models and scheduling an execution order of the tests to give priority to tests of software components with high importance; and a fifth step of the computer outputting an execution instruction for tests of the software satisfying the overall control specification in accordance with the execution order of the tests determined by the result of the scheduling. Includes. Effect of the Invention

[0013] According to the present invention, by utilizing (or reusing) unit test patterns prepared in advance for software having complex and large-scale control specifications, it becomes possible to prioritize and efficiently perform integration tests of controls that the user himself considers important.

[0014] The details of at least one implementation of the subject matter disclosed herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the disclosed subject matter will become apparent from the following disclosure, drawings, and claims. [Brief description of the drawings]

[0015] [Figure 1] FIG. 2 is a block diagram showing an example of a hardware configuration according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing an example of a software configuration according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram showing an example of a partial control model represented by partial control model data according to an embodiment of the present invention. [Figure 4] FIG. 2 is a diagram showing an example of an overall control model formed by combining partial control models according to an embodiment of the present invention. [Diagram 5] 11 is a diagram showing that information on a plurality of control parts according to an embodiment of the present invention is stored in a control part storage unit. FIG. [Figure 6] FIG. 2 is a diagram showing an example of a configuration of an overall control specification model according to an embodiment of the present invention. [Figure 7] 10 is a flowchart illustrating an example of a process performed by an integration test execution server according to an embodiment of the present invention. [Figure 8A] FIG. 11 is a diagram showing an example of an execution result of a unit test pattern according to an embodiment of the present invention. [Figure 8B] FIG. 13 is a diagram showing an example of a schedule in which tests are assigned to PCs in ascending order of IDs according to an embodiment of the present invention. [Figure 9] FIG. 13 is a diagram showing an example of a result of grouping test times of control components by importance according to an embodiment of the present invention. [Figure 10] FIG. 13 is a diagram showing an example of a result of sorting grouped control parts according to an embodiment of the present invention. [Figure 11] FIG. 11 is a diagram illustrating an example of scheduling according to an embodiment of the present invention. [Figure 12] FIG. 13 is a diagram showing an example of a result of a bonding test according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0017] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings, in which: Fig. 1 is a block diagram showing an example of the hardware configuration of an integration test execution server 10 and an integration test execution PC group 17 for executing integration tests according to an embodiment of the present invention.

[0018] The integration test execution server 10 is a computer including a processor 11, a memory 12, a storage device 13, and a network device 14.

[0019] The network device 14 is connected to a network 15 and can communicate with a user terminal 30 and an integration test execution PC group 17. The memory 12 stores, as programs Pg, a test pattern collection program Pg1, an integration test order determination program Pg2, an integration test execution program Pg3, and an integration test result aggregation program Pg4.

[0020] Furthermore, the control part storage unit 16 in the memory 12 stores control part data (software parts) that is data used in the processing of each program Pg. Furthermore, the overall control model revision management unit 18 stores revision information of a plurality of partial control models that constitute the overall control model. The overall control model and partial control models will be described later. The control part storage unit 16 can be a database that stores the control part data.

[0021] The processor 11 operates as a functional unit that provides a predetermined function by appropriately loading each program Pg in the memory 12 and executing processing according to the program Pg. The storage device 13 stores data, programs, intermediate data, and the like used by the processor 11 as appropriate.

[0022] The integration test execution PC group 17 receives instructions from the integration test execution program Pg3 of the integration test execution server 10 via the network 15 and executes the tests. The results of the test execution are collected in the integration test execution server 10 by the integration test result collection program Pg4.

[0023] In this embodiment, an example in which the integration test is executed in parallel on multiple PCs has been shown, but the present invention is not limited to this. For example, the integration test may be executed in parallel using multiple computer resources such as multiple virtual computers or multiple Docker containers.

[0024] Fig. 2 is a block diagram showing an example of the software configuration of the integration test execution server 10 according to the embodiment of the present invention, and the relationship between the user terminal 30 and the integration test execution PC group 17. Fig. 2 describes the processing contents and processing procedures in the processor 11 in Fig. 1, and also describes the input data used and the output data as a deliverable.

[0025] The functions provided by the processing in the processor 11 in the integration test execution server 10 are performed by appropriately referring to the data stored in the control component memory unit 16 and the overall control model revision management unit 18, and are composed of a test pattern collection unit 21 that performs data processing in accordance with a test pattern collection program Pg1, an integration test order determination unit 22 that performs data processing in accordance with an integration test order determination program Pg2, an integration test execution unit 23 that performs data processing in accordance with an integration test execution program Pg3, and an integration test result aggregation unit 24 that performs data processing in accordance with an integration test result aggregation program Pg4.

[0026] In the processing in the processor 11, control component data D1, overall control specification model D2, software that satisfies the overall control specification D3, integration test executable time D4, and most important part control model list D7 are input, and integration test results D5 and integration test exclusion list D6 are output as the final deliverables.

[0027] Of these, the control part data D1 is input from outside and stored in advance in the control part storage unit 16. The control part data D1 will be described in detail later with reference to the drawings, and is composed of a partial control model M (D1M) describing partial control specifications, a model source S (D1S) generated from the partial control model M (D1M), a unit test pattern Tp (D1Tp) that performs equivalence verification between the partial control model M (D1M) and the model source S (D1S) using input values ​​that cover processes including branches in the model source S (D1S), and a unit test execution time Tt (D1Tt).

[0028] In order to grasp the extent of the impact when a source such as the model source S is changed, the equivalence verification determines whether the logical operation contents match before and after the change to the model source S, and outputs an index such as the match ratio. Also, the unit test execution time Tt can be the execution result of the unit test pattern Tp.

[0029] An example of a partial control model M expressed by partial control model data (D1M) is shown in Figure 3. The partial control model M is defined by input I (input values ​​I1 and I2 in this example), output O (output values ​​O1 and O2 in this example), and processing contents OP (processes OP1, OP2, and OP3 in this example). The partial control model M is written using a modeling tool such as Simulink.

[0030] Partial control models M for other control parts are also constructed in this description format, and the combination of these partial control models M forms an overall control model MX (see Figure 4). For example, the overall control model MX forms a model that simulates an automobile electronic control unit ECU.

[0031] 4 is a diagram showing a typical example of an overall control model MX formed by combining partial control models M. The overall control model MX simulates the overall control by arranging multiple partial control models MA to ME in the order shown in the figure.

[0032] As shown in the example of Figure 4, the overall control model MX is composed of multiple partial control models M (in this embodiment, partial control models MA to ME), and the execution order of each partial control model M is determined by the calculation of common variables and reference timing.

[0033] In the illustrated example, the output of the partial control model MA is input to the partial control model MB, which performs a predetermined process on the output of the partial control model MA and outputs the result to the partial control models MC and MD.

[0034] The partial control model MC performs a predetermined process on the output of the partial control model MB and outputs it to the partial control model ME. The partial control model MD outputs the result of performing a predetermined process on the output of the partial control model MB. The partial control model ME outputs the result of performing a predetermined process on the outputs of the partial control models MC and MD.

[0035] In addition, source code written in C is generated using the autocoding function of TargetLink or the like based on the partial control model M. This source code is the model source S in the input D1 in Figure 2.

[0036] The generated model source S (D1S) needs to be verified for equivalence to check whether it has been generated according to the specifications of the partial control model M. Therefore, unit testing is performed using an Embedded Platform or the like.

[0037] The unit test specifications used at this time are input values ​​that can cover all the processes including the branches in the model source S (D1S), and this is called the unit test pattern Tp. The time taken to execute this unit test is called the unit test execution time Tt.

[0038] The partial control model M, the model source S, the unit test pattern Tp, and the unit test execution time Tt are formed in pairs and expressed as data groups. These data groups are managed by an identifier ID (hereinafter simply called ID) for each control part, and a character string (e.g., _A to _E) that indicates the importance designated by the user is added to the end of the ID.

[0039] 5 is a diagram showing that information on a plurality of control parts is stored as control part data D1 (D1A to D1E in this example) in the control part storage unit 16. The partial control model M, model source S, unit test pattern Tp, and unit test execution time Tt constituting the control part information (control part data D1) are managed with a unique ID assigned to each control part.

[0040] For example, control part information D1A is assigned an ID of M0001, control part information D1B is assigned an ID of M0002, control part information D1C is assigned an ID of M0003, control part information D1D is assigned an ID of M0004, and control part information D1E is assigned an ID of M0005.

[0041] A unique ID is also assigned to the control parts (model source S, unit test pattern Tp, unit test execution time Tt) related to the partial control model M of each control part. This makes it easy to determine which control part information each control part belongs to.

[0042] In addition, a character string (e.g., _A to _E) indicating the importance of each control part specified by the user is added to the end of the ID. On the assumption that the user embeds a character string indicating the importance in the file name of the partial control model M (D1M), the control part storage unit 16 acquires the importance specified by the user and automatically generates a unique ID.

[0043] A rule is determined in advance to indicate the priority of integration tests using five levels of importance, for example, A>B>C>D>E. The importance is assigned to the end of the ID, and in Fig. 5, ID=M0001_A for control part information D1A, ID=M0002_B for control part information D1B, ID=M0003_C for control part information D1C, ID=M0004_D for control part information D1D, and ID=M0005_E for control part information D1E.

[0044] The ID is assigned after confirming that the file name of the target partial control model M has not already been registered in the control part storage unit 16.

[0045] 2 functions as follows, using the various inputs and stored contents described above. An example of the process performed by the integration test execution server 10 is shown in the flowchart of FIG.

[0046] First, the test pattern collection unit 21 executed by the processor 11 reads the overall control specification model D2 in accordance with the test pattern collection program Pg1 and refers to the control component storage unit 16. The test pattern collection unit 21 first collects unit test patterns Tp (D1TP) of each partial control model M from the control component storage unit 16 using the overall control specification model D2 of FIG. 4 expressed by a combination of multiple partial control models M.

[0047] For example, when focusing on the partial control model MA, the test pattern collection unit 21 extracts ID=M0001_A, which is a unique ID for each control component linked to the partial control model MA (processing step S1 in FIG. 7).

[0048] The test pattern collection unit 21 extracts the unit test pattern Tp (D1Tp) and the unit test execution time Tt (D1Tt) of the corresponding partial control model MA from the control component storage unit 16 using the acquired ID as key information.

[0049] If the unit test pattern Tp and the unit test execution time Tt are not present in the control component storage unit 16, the test pattern collection unit 21 outputs the target model source name to the integration test exclusion list D6 (processing step S2 in FIG. 7). The test pattern collection unit 21 repeats processing step S2 in FIG. 7 for the number of IDs of the control components (software components).

[0050] Here, the overall control specification model D2 may be described as an overall control model MX as shown in FIG. 4, or it may be a list 100 listing the model sources S to be combined as shown in FIG. 6, or a group of model sources 110 in software that satisfies the overall control specification after combining the model sources S.

[0051] The integration test order determination unit 22 in FIG. 2 performs data processing on the acquired unit test patterns Tp and unit test execution times Tt in accordance with an integration test order determination program Pg2.

[0052] For example, if there are three PCs for executing integration tests, and you try to assign the execution of unit test patterns Tp to each PC one by one in ascending order of the acquired IDs, there is a possibility that test cases with relatively long execution times will be executed consecutively on one PC.

[0053] Fig. 8A is a diagram showing a unit test execution time Tt when a unit test pattern Tp is executed by one PC of the integration test execution PC group 17. Fig. 8B is a diagram showing an example of test execution when the unit test pattern Tp is assigned to the integration test execution PC group 17 in an order such as round robin.

[0054] In the example of FIG. 8B, the execution time of PC1 is longer than that of PC2 and PC3. On the other hand, the total execution time of PC3 is relatively short. As a result, the total execution time until all tests are executed by PC1 to PC3 is long. Also, the execution of high-importance test cases may be postponed.

[0055] In order to eliminate such a bias in the total execution time, the integration test order determination unit 22 is provided. First, the integration test order determination unit 22 obtains the importance from each ID in processing step S3 of Fig. 7, and groups the same importance. Fig. 9 is a diagram showing an example of grouping unit test patterns Tp by importance.

[0056] 9, the importance is A>B>C>D>, but this importance may be grouped according to the Automotive Safety Integrity Level (ASIL), which indicates the functional safety level of the in-vehicle device. For example, if the importance is replaced with the functional safety level, such as ASIL_D>ASIL_C>ASIL_B>ASIL_A>QM, and the importance is added to the ID in advance, it becomes possible to automatically group by functional safety level.

[0057] Furthermore, the integration test execution server 10 stores, in addition to the IDs, revision information of the multiple partial control models D1M constituting the overall control specification model D2 in the overall control model revision management unit 18 each time an integration test is performed.

[0058] This allows the integration test order determination unit 22 to identify control parts whose revisions have been changed among the control parts being used this time. The integration test order determination unit 22 can place the control parts whose revisions have been changed in a group with a higher importance than other control parts, which is effective when making small changes to the control during a running change of the equipment.

[0059] Furthermore, by registering the partial control model names in the most important partial control model list D7 on the user terminal 30 and inputting them into the integration test execution server 10, it is possible to place those control parts in the most important group and increase their priority when executing the integration test.

[0060] In addition, the most important part control model list D7 allows the user to specify the part control models for which the user wants to give higher priority during integration testing, thereby enabling integration testing of the control parts desired by the user to be executed with priority.

[0061] For example, in the case of electronic control units for automobiles, it is required to focus on the evaluation of the communication function with each device and the self-diagnosis function. Also, in the case of control units for electric vehicles, it is required that the battery charging control be evaluated as a device. This is effective when you want to check the specific control (function) according to the evaluation items of the device.

[0062] In this way, by providing three options (functional safety level, revision change control parts, and user-specified control parts), more practical grouping is possible.

[0063] Next, in processing step S4 of FIG. 7, the integration test order determination unit 22 sorts the control components for each grouped importance in descending order of unit test execution time Tt.

[0064] 10 is a diagram showing an example of control components sorted by importance in ascending order of unit test execution time Tt. Finally, the integration test order determination unit 22 schedules the allocation of each test in accordance with processing step 5 of FIG. 7 so that the total execution time of the tests on each integration test execution PC is equalized.

[0065] FIG. 11 is a diagram showing an example of a scheduling result in which unit tests are assigned to each PC of the integration test execution PC group 17. In FIG.

[0066] In FIG. 11, the integration test order determination unit 22 calculates a schedule in the above-mentioned processing step S5 from the test pattern list 120 sorted by the unit test execution time Tt for each importance of the control components in the above-mentioned processing steps S3 and S4, and outputs a schedule 130 in which the unit test patterns Tp are assigned to the integration test execution PCs.

[0067] In detail, in process step S5, the variable i indicating the number of control components used by the integration test order determination unit 22 is reset to 0 (S51), and then process steps S52 to S54 are repeated until the total number of control components TN (12 in this embodiment) is reached.

[0068] The integration test order determination unit 22 sequentially assigns the test pattern list 120 to the PC with the smallest total unit test execution time Tt to generate a schedule 130 (S53). Then, after incrementing the variable i in process step S54, the process returns to process step S52 and repeats the above process.

[0069] In this embodiment, the execution order is scheduled in ascending order of unit test execution time Tt, but it is also possible to schedule in ascending order of unit test execution time Tt so that more test patterns can be executed within the integration test execution time D4 specified by the user.

[0070] The integration test execution unit 23 instructs a plurality of integration test execution PCs to execute integration tests in accordance with the integration test execution program Pg3 and in accordance with the test execution order (schedule 130) determined by the integration test order determination unit, as shown in processing step S6 of FIG.

[0071] At that time, the integration test execution time D4 specified by the user is compared with the remaining test time (S61), and unit test patterns that cannot be completed within the integration test execution time D4 are excluded from the integration test, and the model source names are output to the integration test exclusion list D6 (S64). This is determined for each integration test execution PC based on the scheduling results of the unit test pattern Tp, and is output collectively.

[0072] The integration test execution PC group 17, in accordance with the integration test execution instruction received from the integration test execution unit 23 (S62), executes measurement tests of function call coverage and function coverage in the model source S, and regression tests of each partial control model, for the software D3 that satisfies the overall control specification (S63).

[0073] The unit test pattern Tp is an input value that comprehensively executes the processes in the model source S, so that measurement tests of function coverage and function call coverage can be performed.

[0074] In addition, since the model source S reuses the unit test pattern Tp, a regression test can be performed on the implemented microcomputer code after combining the partial control model M and the model source S for equivalence verification.

[0075] The integration test execution PC group 17 outputs the execution results to the integration test execution server 10. The integration test is performed using CoverageMaster winAMS or the like, which is capable of performing testing with a microcomputer simulator using the software D3 that satisfies the overall control specifications, that is, the implemented microcomputer code after combination, as is.

[0076] The integration test result aggregation unit 24 aggregates the integration test execution results received from each integration test execution PC in accordance with an integration test result aggregation program Pg4, generates integration test results D5, and transmits them to the user terminal 30.

[0077] 12 is a diagram showing an example of the integration test result D5. The integration test result D5 includes an ID 501, a model source name 502, a function call coverage 503, a function coverage 504, and an equivalence checking result 505 in one entry.

[0078] ID 501 is an identifier of a control component. Model source name 502 is the name of the model source S. Function call coverage 503 indicates the percentage of executed software function calls. Function coverage 504 indicates the number of functions in each software layer and the percentage of executed functions. Equivalence verification result 505 indicates the percentage of logical operation contents that match before and after a change to the model source S.

[0079] According to this embodiment, for software having complex and large-scale control specifications, not only is a pre-prepared unit test pattern Tp reused to automatically execute tests on partial control units, but tests for controls with high importance specified by the user are executed with priority, making it possible to efficiently execute tests according to the situation.

[0080] In the control component storage unit 16, by using an ID for each control component, it becomes easy and clear to associate each partial control model M, model source S, unit test pattern Tp, and unit test execution time Tt. In addition, the integration test execution server 10 automatically assigns an ID to each control component, thereby reducing the man-hours required for managing the control components.

[0081] The test pattern collection unit 21 can extract an ID from the overall control specification model D2 and automatically collect the required unit test patterns Tp based on the ID, thereby reducing the number of steps required for collection and the like.

[0082] Furthermore, when the test pattern collection unit 21 automatically collects the unit test pattern Tp, if the unit test pattern Tp is not registered in the control component storage unit 16, the test pattern collection unit 21 outputs the target model source name to the integration test exclusion list D6. This makes it possible to catch defects caused by missed integration tests in advance.

[0083] The integration test order determination unit 22 schedules the tests of control components with high importance specified by the user for each integration test execution PC evenly so as to give priority to the tests and to minimize the total execution time of the tests, thereby making it possible to most efficiently perform the target tests within the integration test execution time specified by the user.

[0084] The integration test execution unit 23 instructs each PC to automatically execute measurement tests of function call coverage and function coverage in the model source S, and regression tests of each partial control model, thereby reducing the amount of work required to execute an integration test.

[0085] The integration test result aggregation unit 24 aggregates the integration test results D5 received from each integration test execution PC and transmits them to the user terminal 30, thereby reducing the number of steps required to output the integration test results D5.

[0086] <Conclusion> As described above, the integration test execution method of the above embodiment can be configured as follows.

[0087] (1) A software integration test execution method for performing an integration test of a software component using a computer (integration test execution server 10) including a processor (11) and a memory (12), comprising: a first step in which the computer (10) stores in a control component storage unit (16) software components (control component data D1) including a partial control model (M) describing a partial control specification having a preset importance, a model source (S) generated from the partial control model (M), a unit test pattern (Tp) for performing equivalence verification of the partial control model (M) and the model source (S) using input values ​​covering processes including branches in the model source (S), and a unit test execution time (Tt) required for executing the unit test pattern (Tp); and a second step in which the computer (10) stores in a control component storage unit (16) software components (control component data D1) including an overall control specification model (D2) combining the partial control models (M) in order to provide a function of a control target. a second step of accepting software (D3) that satisfies an overall control specification; a third step of collecting, from the control component storage unit (16), the unit test patterns (Tp) of the partial control model (M) included in the overall control specification model (D2) and a unit test execution time (Tt) required to execute the unit test patterns (Tp); a fourth step of the computer (10) acquiring importance of the unit test patterns (Tp) of the partial control model (M) and scheduling an execution order of the tests in which tests of software components (D1) with high importance are executed preferentially; and a fifth step of the computer (10) outputting an execution instruction for tests of the software (D3) components that satisfy the overall control specification in accordance with the test execution order determined by the result (130) of the scheduling.

[0088] With the above configuration, by utilizing (or reusing) unit test patterns prepared in advance for software having complex and large-scale control specifications, it is possible to prioritize and efficiently perform integration tests of controls that the user considers important.

[0089] (2) The software integration test execution method according to (1) above, further comprising: a sixth step in which a plurality of external computer resources (an integration test execution PC group 17) receive an instruction to execute the test, execute a preset test, and output a result of the test to the computer (10); and a seventh step in which the computer (10) receives the execution result of the test from the plurality of computer resources (17), aggregates the execution results of the plurality of tests to generate an integration test result (D5), and outputs the integration test result to a user terminal (30).

[0090] With the above-described configuration, the integration test execution server 10 transmits to the user terminal 30 integration test results that are an aggregate of integration test execution results output from the integration test execution PC group 17, which are multiple computing resources, so that the user can quickly obtain the results of the integration test.

[0091] (3) The software integration test execution method according to (1) above, wherein in the first step, unique identifiers are assigned to the partial control model (M), the model source (S), the unit test pattern (Tp), and the unit test execution time (Tt) to manage them.

[0092] With the above configuration, by assigning a unique ID to the control components (software components) associated with the partial control model M, it becomes possible to easily determine which control component information each of the software components constituting the control component data D1 belongs to.

[0093] (4) The software integration test execution method according to (1) above, characterized in that in the first step, when storing the software component (D1) in the control component storage unit (16), if the software component (D1) is not registered, a new identifier is assigned to the software component (D1).

[0094] With the above configuration, when the control component data D1 is registered in the control component storage unit 16, a unique ID is assigned to each of the model source S, unit test pattern Tp, and unit test execution time Tt related to the partial control model M.

[0095] (5) The software integration test execution method according to (3) above, wherein in the third step, the identifiers of the software components (D1) are extracted using the overall control specification model (D2) described by combining the partial control models (M) to which the identifiers are assigned and stored in the control component storage unit (16) and a list of model sources (S) to be combined, or a group of model sources (S) after combination, and the unit test pattern (Tp) and the unit test execution time (Tt) associated with the identifiers are collected from the control component storage unit (16).

[0096] With the above configuration, the integration test execution server 10 (computer) can collect the unit test pattern Tp to be used for the integration test and the unit test execution time Tt from the control component storage unit 16 by extracting the ID of the control component data (D1) using the overall control specification model D2 and a list of the model sources S to be integrated, or the group of model sources (S) after integration.

[0097] (6) The software integration test execution method according to (5) above, wherein in the third step, if the unit test pattern (Tp) of the partial control model (M) does not exist in the control part storage unit (16), it is determined that integration testing is impossible, and the target model source (S) is output to an integration test exclusion list.

[0098] With the above configuration, when the integration test execution server 10 registers the control component data D1 in the control component storage unit 16, if the unit test pattern (Tp) of the partial control model (M) does not exist, it can output it to an integration test exclusion list, thereby notifying the user terminal 30 that the integration test was not executed.

[0099] (7) The software integration test execution method according to (5) above, wherein in the fourth step, the importance is obtained from the collected unit test patterns (Tp) and the software components (D1) are grouped by the same importance.

[0100] With the above configuration, the integration test execution server 10 can perform grouping that determines the priority order when executing the control component data D1 according to the importance given to the unit test patterns Tp.

[0101] (8) A software integration test execution method according to (7) above, characterized in that in the fourth step, revision information of the partial control model (M) constituting the overall control specification model D2 is stored in an overall control model revision management unit 18 each time an integration test is performed.

[0102] With the above configuration, the integration test execution server 10 can detect changes in the revision information by storing the revision information of the partial control model M in the overall control model revision management unit 18 every time an integration test is executed.

[0103] (9) A software integration test execution method as described in (8) above, characterized in that in the fourth step, revision information of the partial control model (M) on which a previous integration test was performed, which is stored in the overall control model revision management unit (18), is compared with revision information of the partial control model (M) on which a current integration test is to be performed, and the partial control model (M) whose revision information has been changed is set to a higher importance than other software components (D1).

[0104] With the above configuration, the integration test execution server 10 sets the importance of the partial control model M whose revision information stored in the overall control model revision management unit 18 has been changed higher than that of other control part data. This makes it possible to prioritize the partial control model M whose revision information has been updated in the integration test.

[0105] (10) A software integration test execution method as described in (7) above, characterized in that in the second step, a most important part control model (M) list storing a part control model (M) designated by a user is accepted, and in the fourth step, the part control model (M) stored in the most important part control model list D7 is set to a higher importance than other software components (D1).

[0106] With the above configuration, the integration test execution server 10 can set a high priority to the partial control model M designated according to the most important partial control model list D7. This makes it possible to change the priority of the partial control model M for which the integration test is executed according to the user's wishes.

[0107] (11) The software integration test execution method according to (7) above, characterized in that in the fourth step, the collected unit test patterns (Tp) are sorted in ascending or descending order of the unit test execution time (Tt).

[0108] With the above configuration, the integration test execution server 10 can easily determine combinations of integration tests to be assigned to multiple computer resources by sorting the unit test patterns Tp in ascending or descending order of the unit test execution time Tt.

[0109] (12) The software integration test execution method according to (7) above, characterized in that in the fourth step, the allocation and execution order of the tests are scheduled for each of a plurality of external computer resources (17) to which the integration tests are requested so that the total execution times of the tests in the plurality of computer resources (17) are uniform.

[0110] With the above configuration, the integration test execution server 10 schedules the execution order of tests assigned to the PCs so that the total execution time of the tests on the multiple computer resources (integration test execution PC group 17) is equalized.

[0111] (13) The software integration test execution method according to (7) above, characterized in that in the fifth step, parallel execution instructions for the tests are issued to a plurality of external computer resources (17) in accordance with the determined execution order of the scheduling.

[0112] With the above configuration, the integration test execution PC group 17 executes the integration test in parallel according to the schedule, thereby making it possible to reduce the time required for the test.

[0113] (14) The software integration test execution method according to (7) above, characterized in that in the second step, an integration test execution time (D4) designated by a user is accepted, and in the fifth step, a unit test pattern (Tp) that does not end within the integration test execution time is determined to be infeasible for integration testing, and the target model source (S) is output to an integration test exclusion list D6.

[0114] With the above configuration, the integration test execution server 10 stores in the integration test exclusion list D6 the unit test patterns Tp whose tests are not completed within the integration test execution time D4 specified by the user, and can notify the user terminal 30 of the fact that the integration test could not be executed.

[0115] The present invention is not limited to the above-mentioned embodiment, and various modifications are included. For example, the above-mentioned embodiment is described in detail to easily explain the present invention, and is not necessarily limited to including all the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, any addition, deletion, or replacement of other configurations can be applied to a part of the configuration of each embodiment, either alone or in combination.

[0116] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. In addition, the above-mentioned configurations and functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0117] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0118] 10 Integration test execution server 11 Processors 12. Memory 13 Storage Devices 14 Network Equipment 15 Network 16 Control parts memory section 17 PCs for executing integration tests 21 Test pattern collection unit 22 Integration test order determination part 23 Integration Test Execution Department 24 Integration test result aggregation section D1 Partial control data D1M Partially Controlled Model M D1S Model Source S D1Tp Unit test pattern Tp D1Tt Unit test execution time Tt D2 Overall Control Model Software that meets D3 overall control specifications D4 Integration test execution time D5 Integration Test Results D6 Integration Test Exclusion List D7 Most Important Part Control Model List M Partial Control Model MX Total Control Model

Claims

1. A software integration test execution method for performing an integration test of a software component using a computer including a processor and a memory, comprising the steps of: a first step in which the computer stores in a control component storage unit software components including a partial control model describing partial control specifications with preset importance, a model source generated from the partial control model, a unit test pattern for performing equivalence verification between the partial control model and the model source using input values ​​covering processes including branches in the model source, and a unit test execution time required for executing the unit test pattern; a second step in which the computer receives an overall control specification model that combines the partial control models to provide a function of a control target, and software that satisfies the overall control specification; a third step of collecting, from the control component storage unit, the unit test patterns of the partial control models included in the overall control specification model and unit test execution times required for executing the unit test patterns; a fourth step of the computer acquiring the importance of unit test patterns of the partial control model and scheduling an execution order of the tests in such a way that tests of software components having high importance are executed with priority; a fifth step of outputting an instruction to execute tests of the software that satisfies the overall control specification in accordance with the execution order of the tests determined by the result of the scheduling; 13. A method for executing software integration tests comprising:

2. 2. A software integration test execution method according to claim 1, comprising: a sixth step in which a plurality of external computer resources receive an instruction to execute the test, execute a preset test, and output a result of the execution of the test to the computer; a seventh step in which the computer receives execution results of the tests from the plurality of computer resources, aggregates the execution results of the tests to generate an integration test result, and outputs the integration test result to a user terminal; The method for executing a software integration test further comprises:

3. 2. A software integration test execution method according to claim 1, comprising: In the first step, A software integration test execution method, comprising the steps of: assigning unique identifiers to the partial control model, the model source, the unit test pattern, and the unit test execution time, and managing them.

4. 2. A software integration test execution method according to claim 1, comprising: In the first step, A software integration test execution method, comprising the steps of: when storing the software component in the control component storage unit, if the software component is not yet registered, assigning a new identifier to the software component.

5. 4. A software integration test execution method according to claim 3, comprising: In the third step, a software integration test execution method comprising: extracting the identifiers of software components using the overall control specification model described by combining the partial control models to which the identifiers are assigned and stored in the control component storage unit, and a list describing model sources to be combined, or a group of model sources after combination; and collecting the unit test patterns and unit test execution times associated with the identifiers from the control component storage unit.

6. 6. A software integration test execution method according to claim 5, comprising: In the third step, A software integration test execution method characterized in that, if the unit test pattern of the partial control model does not exist in the control component storage unit, it is determined that integration testing is not possible, and the target model source is output to an integration test exclusion list.

7. 6. A software integration test execution method according to claim 5, comprising: In the fourth step, A method for executing software integration tests, comprising the steps of: obtaining the importance from the collected unit test patterns; and grouping the software components according to the same importance.

8. 8. A method for executing a software integration test according to claim 7, comprising: In the fourth step, A method for executing a software integration test, comprising the steps of: storing revision information of partial control models constituting the overall control specification model in an overall control model revision management section every time an integration test is performed.

9. 9. A method for executing a software integration test according to claim 8, comprising: In the fourth step, A method for executing software integration testing, characterized by comparing revision information of the partial control model for which an integration test was previously performed, which is stored in the overall control model revision management unit, with revision information of the partial control model for which an integration test is to be performed this time, and setting a higher importance level for the partial control model for which the revision information has been changed than for other software components.

10. 8. A method for executing a software integration test according to claim 7, comprising: In the second step, Accept the most important partial control model list that stores the partial control models specified by the user, In the fourth step, A method for executing a software integration test, comprising the steps of: setting a level of importance of a part control model stored in the most important part control model list higher than that of other software components.

11. 8. A method for executing a software integration test according to claim 7, comprising: In the fourth step, A software integration test execution method comprising: sorting the collected unit test patterns in ascending or descending order of the unit test execution time.

12. 8. A method for executing a software integration test according to claim 7, comprising: In the fourth step, A software integration test execution method, characterized in that the allocation and execution order of the tests are scheduled for each of the multiple external computer resources to which the integration tests are requested so that the total execution time of the tests on the multiple computer resources is equal.

13. 8. A method for executing a software integration test according to claim 7, comprising: In the fifth step, A software integration test execution method, comprising the steps of: issuing parallel execution instructions for the tests to a plurality of external computer resources in accordance with the execution order of the scheduled execution.

14. 8. A method for executing a software integration test according to claim 7, comprising: In the second step, Accept the user-specified time available for integration testing, In the fifth step, A software integration test execution method, comprising: judging that a unit test pattern that is not completed within the integration test execution time is not integration testable, and outputting the target model source to an integration test exclusion list.

15. A computer including a processor and a memory for performing integration testing of a software component, a control component storage unit for storing software components including a partial control model describing a partial control specification with a preset importance, a model source generated from the partial control model, a unit test pattern for performing equivalence verification between the partial control model and the model source using input values ​​covering processes including branches in the model source, and a unit test execution time required for executing the unit test pattern; an overall control specification model that combines the partial control models to provide a function of a control target, software that satisfies the overall control specification, the unit test patterns of the partial control models included in the overall control specification model, and a test pattern collection unit that collects from the control component storage unit unit the unit test execution times required to execute the unit test patterns; an integration test order determination unit that acquires the importance of unit test patterns of the partial control model and schedules an execution order of the tests such that tests of software components with high importance are executed preferentially; an integration test execution unit that outputs an instruction to execute a test of the software components in accordance with an execution order of the tests determined by a result of the scheduling; A computer comprising:

Images