Method and system for automatically generating integration tests according to unit tests

Abstract

The present invention relates to a method and system for automatically generating integration tests based on unit tests. A method for automatically generating integration tests based on unit tests, the method comprising: automatically running unit tests for each of a plurality of models constituting a system by a processor. The method further comprises: automatically determining, by the processor, an integration of unit test cases of a set or more sets of connected models among the plurality of models. The method further comprises: automatically running integration tests for each set of connected models by the processor using the unit tests for each model of the connected models. The method further comprises: automatically detecting, by the processor, at least one of inconsistent, incomplete, and incorrect data transmitted between the connected models for each set of connected models.

Description

Technical Field

[0001] The present disclosure relates to model-based development of systems, and more particularly to automatically generating integration tests based on unit tests during model-based development of systems. Background Art

[0002] During model-based development (MBD) of a system, different entities of the system are modeled as separate model units. The model units integrated together model the entire system. The current process is to create model unit tests and integration tests as separate and independent activities. This disconnection leads to inefficiencies and repetitive work, and delays feedback on document development and system model implementation. In addition, since there is no tool to link individual model unit tests and integrate them into higher-level integration test cases representing their functionality, this work is not automated. Summary of the Invention

[0003] According to one embodiment, a method for automatically generating integration tests based on unit tests includes: automatically running unit tests for each of a plurality of models constituting a system by a processor. The method further includes: automatically running integration tests for each set of connected models of the plurality of models by using the unit tests for each of the models of the connected models by the processor. The method further includes: automatically detecting, by the processor, at least one of inconsistent, incomplete, and incorrect data transmitted between the connected models for each set of connected models.

[0004] According to another embodiment, a system for automatically generating integration tests based on unit tests includes: a processor circuit and a memory associated with the processor circuit. The memory includes computer-readable program instructions that, when executed by the processor circuit, cause the processor circuit to perform a set of functions, the set of functions including: automatically running unit tests for each of a plurality of models constituting a system. The set of functions further includes: automatically running integration tests for each set of connected models of the plurality of models by using the unit tests for each of the models of the connected models. The set of functions further includes: automatically detecting at least one of inconsistent, incomplete, and incorrect data transmitted between the connected models for each set of connected models.

[0005] According to an embodiment and any of the foregoing embodiments, the method and system further include: receiving a plurality of models constituting a system, wherein each of the plurality of models includes a low-level model that represents a specific function, a partial function, or a set of functions that the system is configured to perform.

[0006] According to an embodiment and any of the foregoing embodiments, the method and system further comprise: receiving a plurality of models that make up the system, wherein each model of the plurality of models includes a low-level model that defines a requirement, a partial requirement, or a set of requirements that the system is configured to meet.

[0007] According to an embodiment and any of the foregoing embodiments, each model includes a specific logic circuit that defines the requirement that the system is configured to meet.

[0008] According to an embodiment and any of the foregoing embodiments, each model is configured to: generate one or more expected outputs in response to one or more inputs based on the specific logic circuit associated with the specific model.

[0009] According to an embodiment and any of the foregoing embodiments, the method and system additionally comprise: generating a unit test harness for each of the plurality of models that make up the system.

[0010] According to an embodiment and any of the foregoing embodiments, the method and system additionally comprise: generating unit test cases for the unit test harness of each model, wherein automatically running the unit test for each model includes: using the unit test cases for each model.

[0011] According to an embodiment and any of the foregoing embodiments, wherein generating a unit test harness for each model includes: separating the specific model from the environment of the specific model so that the specific model is independent of other models of other system requirements that provide inputs to the specific model.

[0012] According to an embodiment and any of the foregoing embodiments, wherein automatically running the unit test for each model includes: for all combinations of possible different input values for each model, generating an expected output value for each output of one or more outputs of each model in response to the input value of each input in one or more inputs of each model.

[0013] According to an embodiment and any of the foregoing embodiments, the method and system further comprise: automatically generating a table for each model that is independent of other models in response to running the unit test for each model. The table for a specific model includes: for various combinations of possible different input values for the specific model, the expected output values for each output of one or more outputs that are respectively associated with the input value of each input in one or more inputs.

[0014] According to the embodiment and any of the foregoing embodiments, the method and system further include: automatically generating an integration test fixture for each group of connection models; and automatically generating integration test cases for each group of connection models for the integration test fixture, wherein automatically running the integration test for each group of connection models includes: using the integration test cases for each group of connection models.

[0015] According to the embodiment and any of the foregoing embodiments, wherein automatically running the integration test for each group of connection models includes: automatically generating a table for each group of connection models, the table for a particular group of connection models including: for various combinations of possible different input values for the particular group of connection models, the expected output values of each of one or more outputs of the particular group of connection models associated with the input values of each of one or more inputs of the particular group of connection models, the table being used to detect at least one of inconsistencies, incompleteness, and incorrect data among the connection models of the particular group of connection models.

[0016] The features, functions, and advantages that have been discussed can be implemented independently in various embodiments or can be combined in other embodiments. Further details thereof can be seen with reference to the following description and the drawings. Description of the Drawings

[0017] Figure 1A and Figure 1B is a flowchart of an example of a method for automatically generating an integration test according to unit tests according to an embodiment of the present disclosure.

[0018] Figure 2 is an illustration of an example of a connection model of a system according to an embodiment of the present disclosure.

[0019] Figure 3A is for Figure 2 in the example of a test fixture for model A according to an embodiment of the present disclosure.

[0020] Figure 3B is according to an embodiment of the present disclosure Figure 2 in the example of a unit-level model of model A.

[0021] Figure 3C is an example of a set of unit test cases for model A according to an embodiment of the present disclosure.

[0022] Figure 4A is for Figure 2 in the example of a test fixture for model B according to an embodiment of the present disclosure.

[0023] Figure 4B is according to an embodiment of the present disclosure Figure 2 in the example of a unit-level model of model B.

[0024] Figure 4C is an example of a set of unit test cases for Model B according to an embodiment of the present disclosure.

[0025] Figure 5A is an example of an integration test fixture for Model A and Model B according to an embodiment of the present disclosure.

[0026] Figure 5B is an example of how, according to an embodiment of the present disclosure, once Model A and Model B have each passed their respective unit tests, the method is automatically executed to generate integration test cases according to the integration test fixture, which integrates Model A and Model B based on Figure 2 the way Model A and Model B are connected to each other in.

[0027] Figure 6 is an example of what the automatically generated integration test cases for Unit Model A and Unit Model B ( Figure 2 ) look like according to an embodiment of the present disclosure.

[0028] Figure 7 is an example of a system for automatically generating integration tests using unit tests according to an embodiment of the present disclosure. Detailed Description of the Embodiment

[0029] The following detailed description of the embodiment refers to the accompanying drawings, which show specific embodiments of the present disclosure. Other embodiments with different structures and operations do not depart from the scope of the present disclosure. In the different drawings, like reference numerals may refer to the same elements or components.

[0030] The present disclosure may be a system, method, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to execute aspects of the present disclosure.

[0031] A computer-readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. For example, a computer-readable storage medium can be, but is not limited to: an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punched card or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be construed to be a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse passing through an optical fiber cable), or an electrical signal transmitted through a wire.

[0032] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network can include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the corresponding computing / processing device.

[0033] The computer-readable program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state-setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages (such as Smalltalk, C++, etc.) and conventional procedural programming languages (such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, may be executed partly on the user's computer as a stand-alone software package, may be executed partly on the user's computer and partly on a remote computer, or may be executed entirely on the remote computer or server. In the latter case, the remote computer may be connected to the user's computer through any type of network (including a local area network (LAN) or a wide area network (WAN)), or may establish a connection with an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, to perform aspects of the present disclosure, an electronic circuit may be personalized by executing the computer-readable program instructions by utilizing the state information of the computer-readable program instructions. For example, the electronic circuit includes a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA).

[0034] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0035] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine instruction, such that the instruction is executed via the processor of the computer or other programmable data processing to create a means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer-readable storage medium in which the instructions are stored includes an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0036] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other devices to produce a computer-implemented process such that the instructions executed on the computer, other programmable apparatus, or other devices implement the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0037] Figure 1A and Figure 1B FIG. 6 is a flowchart of an example of a method 100 for automatically generating integration tests based on unit tests according to an embodiment of the present disclosure. At block 102, a plurality of models 104 that make up a system 106 are received. The system 106 is any type of system. According to an example, the system 106 is a system of an aircraft or a part of a system, such as a flight control system, an engine control system, a navigation system, a communication system, a landing gear system, etc.

[0038] According to an example, each of the plurality of models 104 includes a low-level model 108 that represents a specific function, a partial function, or a set of functions that the system 106 is configured to perform. According to an embodiment, each model 104 includes specific logic circuits 202, 206 ( Figure 2 ), which define the requirements that the system 106 is configured to meet. According to an example, each of the plurality of models 104 includes a low-level model 104 that defines the requirements, partial requirements, or a set of requirements that the system 106 is configured to meet.

[0039] Reference is also made to Figure 2 , Figure 2 FIG. 7 is an illustration of an example of an interfacing model 104 (model A 104a and model B 104b) of a system 106 according to an embodiment of the present disclosure. Model A 104a includes a logic circuit 202 that defines requirement A 204. Model B 104b includes a logic circuit 206 that defines requirement B 208. Each model 104 is configured to generate one or more desired outputs 210 in response to one or more inputs 212 based on the specific logic circuit 202 or 206 associated with the specific model 104a or 104b. According to an example, the specific logic circuits 202 and 206 are each implemented as hardware, software, or a combination of hardware and software that generates one or more desired outputs 210 in response to one or more inputs 212. As shown in the example of Figure 2 FIG. 7, the output 210 of model A 104a is the input 212 of model B 104b. Model B 104b also includes an external input, i.e., input E 216, which comes from another source ( Figure 2(not shown in the figure) instead of the output from model A104a. Similarly, the output C210 from model B104b can be the input to another model 104 of system 106.

[0040] Return to reference Figure 1A , in block 110, unit test fixtures 112 are generated for each of the multiple models 104 that make up system 106. The unit test fixtures 112 are generated manually or automatically. According to an embodiment, the unit test fixtures 112 are manually generated by a user who implements a computer pointing device to specify a particular model 104 among the multiple models 104. As Figure 3A shown, Figure 3A is an example of a unit test fixture 112 for model A104a in Figure 2 according to an embodiment of the present disclosure. The main function of a test fixture such as unit test fixture 112 is to isolate the model 104 (or models) inside what can be regarded as a "black box" model 302, where this inside is not visible, that is, there is no need to worry about what the internal logic is. Additionally, the unit test fixture 112 also defines a set of inputs 304 to feed the black box model 302 according to the external interface of the particular model. The black box model 302 also processes the evaluation of the output 306 provided by the particular model 104 based on the given input 304. The value of the output 306 can be evaluated in different ways, but in any case, later when running the unit test 120 (if the black box model 302 contains only one model) or the integration test 130 (if the black box model 302 includes more than one model), the output 306 from the black box model 302 will be compared with the expected output 210. If the output 306 from the black box model 302 matches the expected output 210, the unit test 120 / integration test 130 is successful. If the output 306 from the black box model 302 does not match the expected output 210, the unit test 120 / integration test 130 fails.

[0041] Also refer to Figure 4A , Figure 4A is an example of a unit test fixture 112 for model B104b in Figure 2 according to an embodiment of the present disclosure. The unit test fixture 112 also defines a set of inputs 404 to feed the black box model 402 according to the external interface of the particular model. The black box model 402 evaluates the output 406 provided by the model 402 based on the input 404.

[0042] Thus, the unit test fixture 112 includes separating a particular model 104 from the environment of that particular model 104 in the system 106. Unit tests are performed on each model 104 independent of the model environment in the system 106. That is, unit tests are performed on each particular model 104 independent of other models 104 that provide inputs 212 to the particular model 104 and other system requirements 204 and 208.

[0043] In block 114, for each model 104, unit test cases 116 are generated according to the unit test fixture 112. Also refer to Figure 3B , Figure 3B which is an example of the unit-level model 310 of model A 104a in Figure 2 according to an embodiment of the present disclosure.

[0044] In block 118, unit tests 120 are automatically run for each model 104 among the multiple models 104 that make up the system 106. Automatically running unit tests 120 for each model 104 includes: using unit test cases 116 for each model 104. Performing unit tests 120 in the unit test cases 116. Automatically running unit tests 120 for each model 104 includes: for all combinations of possible different input values 220 for each model 104, in response to the input value 220 of each input among one or more inputs 212 of each model 104, obtaining the output value 218 of each output among one or more outputs 210 of each model 104 ( Figure 2 and Figure 3B ). As Figure 3B shown, unit tests 120 are run by applying different possible input values 220 to the unit-level model 310 of model A 104a. The logic circuit 202 implemented based on the unit-level model 310 generates the expected output value 218.

[0045] Moreover, in block 118, for each model 104 independent of other models 104, a table (e.g., Figure 3C the table 300 in Figure 3C ) that is a set 312 of unit test cases for each model 104 is generated. Also refer to Figure 3C which is an example of the set 312 of unit test cases for model A 104a according to an embodiment of the present disclosure. Figure 3C depicts an example of the table 300 generated from the unit test cases 116 of model A 104a according to an embodiment of the present disclosure. For a particular model 104 (e.g., Figure 3BTable 300 of model A 104a) in the example includes: for various combinations of possible different input values 220 of a specific model 104, the expected output values 218 of each of the one or more outputs 210 respectively associated with the input values 220 of each of the inputs in one or more inputs 212.

[0046] Reference is also made to Figure 4B , Figure 4B which is according to an embodiment of the present disclosure Figure 2 Example of the unit-level model 408 of model B 104b in. As previously described, unit tests 120 are automatically run for each model 104. For all combinations of possible different input values 220 of model B104b, in response to the input values 220 of each of the one or more inputs 212 in model B 104b, the expected output values 218 of each of the one or more outputs 210 in model B 104b are obtained. According to Figure 4B the example shown, model B 104b has an external input 216, which external input 216 comes from another source different from the output 210 of model A 104a ( Figure 2 and Figure 3B ). As Figure 4B shown, the unit tests 120 included in the unit test cases 116 are run by applying different possible input values 220 to the unit test cases 116 of model B 104b. The logic circuit 206 implemented based on the unit-level model 408 of model B 104b generates the expected output values 218.

[0047] Reference is also made to Figure 4C , Figure 4C which is an example of the set of unit test cases 410 of model B 104b according to an embodiment of the present disclosure. Figure 4C Depicts an example of table 400 as the set of unit test cases 410 of model B 104b generated according to the example shown in Figure 4B . Table 400 of model B 104b includes: for Figure 4B and Figure 4C the various combinations of possible different input values 220 of model B 104b in the example, the expected output values 218 of each of the one or more outputs 210 respectively associated with the input values 220 of each of the inputs in one or more inputs 212 and 216.

[0048] In block 122, an integration test fixture 124 is automatically generated for one or more sets of connected models 104 among multiple models 104. Reference is also made to Figure 5A , Figure 5A which is according to an embodiment of the present disclosure for Figure 2An example of the integration test fixture 124 for models A 104a and B 104b in Figure 2 In, models A 104a and B 104b illustrate a set of connected models 104. The integration of connected models 104a and 104b is determined by corresponding the input 212 of model B 104b with the output 210 of model A 104a. In other examples, more than two models 104 form a set of connected models 104. The integration test fixture 124 is generated manually or automatically. Similar to the previously described, the integration test fixture 124 is automatically generated by determining which models 104 have one or more outputs 210 (the one or more outputs 210 are provided as one or more inputs 212 to one or more other models 104 of the system 106). In another embodiment, the integration test fixture 124 is manually generated by a user who implements a computer pointing device to select groups of connected models 104.

[0049] In block 126, integration test cases 128 are automatically generated for each group of connected models 104. Automatically generating integration test cases 128 for each group of connected models 104 includes: interconnecting the input / output (I / O) data of the unit tests 120 of each model 104 from a particular group of connected models 104, similar to Figure 5B as shown. Figure 5B is an example of how method 100 automatically executes to generate integration test cases 128 according to the integration test fixture 124 ( Figure 5A ) that integrates models A 104a and B 104b once the models 104 have passed their respective unit tests 120. Different expected output values 218 of one or more outputs 210 of a first model 502 (e.g., model A 104a) are associated 504 with one or more inputs 212 of a second model 506 (e.g., model B 104b). The output value 218 of the first model 502 (model A 104a) and the corresponding input value 220 of the second model 506 (model B 104b) are implemented in the signal 508 of the connection 510 between models 502 and 506. The signal 508 is analyzed in terms of data type, size, range, unit, and any other parameters required to detect any inconsistent, incomplete, or incorrect data from the integration test of the connected models 502 / 104a and 506 / 104b. In Figure 5B the example, the output data 512 from the unit test 120 of model A 104a is interconnected with the input data 514 from the unit test 120 of model B 104b along with additional input 516 to provide the expected output 518 from the integration test 130. The integration test 130 is included in and defined by the integration test case 128.

[0050] In block 132, integration tests 130 for each connection model 104 are automatically run according to unit tests 120 for each model 104 of connection model 104. Automatically running integration tests 130 for each set of connection models 104 includes: automatically generating table 600 for each set of connection models 104 according to unit tests 120 for each model 104 ( Figure 6 ). Also refer to Figure 6 , Figure 6 is an example of what an integration test case 128 for unit model A 104a and unit model B 104b ( Figure 2 ) automatically generated according to an embodiment of the present disclosure looks like. For the example shown in Figure 5B , table 600 is generated by integration test case 128. Table 600 for a specific set of connection models 104 (such as model A 104a and model B 104b of the example described herein) includes: expected output values 602 for each of one or more outputs 604 associated with input values 606 for each of the inputs in one or more inputs 608 of the specific set of connection models 104. For various combinations 610 of possible different input values 606 for a specific set of connection models 104, the expected output values 602 for each output 604 are determined according to unit tests 120 of connection model 104. In the example shown in Figure 6 , table 600 includes additional possible input values 612, and the additional possible input values 612 come from sources other than connection models 502 and 506 in Figure 5B or the first model 502 of 104a and 104b in Figure 2 described herein ( Figure 5B ). Table 600 is used to detect at least one of inconsistencies, incompleteness, and incorrect data between connection models 502 and 506 or 104a and 104b of a specific set of connection models 104. In the examples shown in Figure 5B and Figure 6 , the input value 220 for "validity" will never be "false" ( Figure 5B ). Therefore, the expected output value 602 of "apuStatus" will never be "invalid" ( Figure 6 ). This indicates inconsistent, incomplete, or incorrect data.

[0051] As previously discussed, integration tests 130 are automatically run according to unit tests for each model 104. Referring again to Figure 3C , Figure 4C and Figure 5B , Figure 6 the input values 606 for each of one or more inputs 608 in table 600 correspond to table 300 (Figure 3C ) The input value 220 of one or more inputs 212 generated according to unit test 120 for model A 104a. Similarly, the output value 602 of each output among one or more outputs 604 in table 600 corresponds to the output value 218 of one or more outputs 210 generated according to unit test 120 for model B 104b in table 400( Figure 4C ). In the exemplary integration test case 128 shown, model B 104b is connected to model A 104a. Figure 5B

[0052] In block 134, the results 614 of integration test 130 run for each set of connected models 502 and 506 or 104a and 104b are presented. According to the example, the results 614 are presented similarly to the results shown in Figure 6 .

[0053] In block 136, for each set of connected models 104, any inconsistent, incomplete, or incorrect data transmitted between connected models 502 and 506 or 104a and 104b or generated by any model 104 is automatically detected. In the example, a message 616 identifying the inconsistent, incomplete, or incorrect data is generated in response to detecting the inconsistent, incomplete, or incorrect data.

[0054] Figure 7 is an example of a system 700 for automatically generating integration tests according to unit tests according to an embodiment of the present disclosure. According to an embodiment, Figures 1A to 1B The method 100 can be implemented in and executed by the system 700. However, any system capable of performing the operations described herein can be used. The system 700 includes a processing device 702. According to the example, the processing device 702 is a server. The processing device 702 includes a processor circuit 704 that is used to control the operation of the processing device 702 and to perform functions such as the functions described herein with respect to Figures 1A to 1B the method 100 in. The processing device 702 also includes a memory 706. An example of the memory 706 is any type of data storage device. The operating system 708, applications, and other programs are stored on the memory 706 for operation on the processor circuit 704. According to an embodiment, multiple models constituting the system (such as (for example) model 104) are stored on the memory 706. The memory 706 also includes a component 710 for automatically generating integration tests according to the unit tests described herein. According to the example, the method 100 described with reference to Figures 1A to 1B is at least partially implemented in the component 710 for performing a set of functions 711. In Figure 7In the example, unit test cases 116 and integration test cases 128 are also stored on the memory 706. In other embodiments, unit test cases 116 and integration test cases 128 are stored on another device or multiple devices.

[0055] According to the example, the processing device 702 further includes one or more input devices, output devices, or combined input / output devices, collectively referred to as I / O devices 712. Examples of I / O devices 712 include, but are not limited to, a keyboard or keypad, a pointing device (e.g., a mouse), a disk drive, and any other device to allow a user to interact with the processing device 702 and control the operation of the processing device 702, and to access and run components 710 to automatically generate integration tests 130 according to unit tests 120. In the example, one of the I / O devices 712 is a device for reading a computer program product such as the computer program product 714. The computer program product 714 may be similar to the devices described in more detail herein. Components 710 and model 104 may be loaded onto the memory 706 from a computer program product such as the computer program product 714.

[0056] According to the example, the system 700 further includes a computer system 716 to access the processing device 702 and components 710 to automatically generate integration tests 130 according to unit tests 120. The computer system 716 accesses the processing device 702 and components 710 for automatically generating integration tests according to unit tests via a network 717. Examples of the computer system 716 include, but are not limited to, any type of electronic device, a communication device including a mobile communication device. Examples of the network 717 include, but are not limited to, the Internet, an intranet, or other private or proprietary networks.

[0057] The computer system 716 includes a processor circuit 718 and a memory 720 for controlling the operation of the computer system 716. The memory 720 includes any type of data storage device. An operating system 722, applications 724, and other programs are stored on the memory 720 to run on the processor circuit 718. According to an embodiment, components 726 for automatically generating integration tests 130 according to unit tests 120 are stored on the memory 720. Figures 1A to 1B The method 100 or at least a part of the method 100 in [ ] is compiled and run on the processor circuit 718 to perform functions similar to those Figures 1A to 1B described for the method 100 in [ ].

[0058] According to an embodiment, a component 726 operating on a computer system 716 for automatically generating an integration test 130 based on a unit test 120 is connected to and / or operates together with a component 710 operating on a processing device 702 for automatically generating an integration test 130 based on the unit test 120 to perform the functions and operations described herein. Thus, the component 726 operating on the computer system 716 performs some of the function set 711 and operation set of the method 100, and the component 710 operating on the processing device 702 performs other functions of the method 100. Some embodiments of the present disclosure include only the component 710 on the processing device 702 for automatically generating an integration test 130 based on the unit test 120, while other embodiments include only the component 726 operating on the computer system 716 for automatically generating an integration test 130 based on the unit test 120.

[0059] According to an embodiment, models such as the model 104, unit test cases 116, and integration test cases 128 are stored only on one of the memories, such as the memory 720 of the computer system 716 or the memory 706 of the processing device 702. In other embodiments, the model 104, unit test cases 116, and integration test cases 128 are stored in both memories 706 and 720. In another embodiment, the model 104, unit test cases 116, and integration test cases are stored on a device other than the memory 706 or the memory 720.

[0060] According to an embodiment, the computer system 716 further includes a display 728 and one or more input devices, output devices, or combined input / output devices, collectively referred to as I / O devices 730. Examples of the I / O devices 730 are the same as the examples of the I / O devices 712.

[0061] The flowcharts and block diagrams in the figures illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified logical function. In some alternative embodiments, the functions noted in the blocks may not occur in the order noted in the figures. For example, depending on the functions involved, two consecutive blocks shown may actually be executed substantially simultaneously, or sometimes may be executed in the reverse order. It should also be noted that each block illustrated in the block diagrams and / or flowcharts, and combinations of blocks illustrated in the block diagrams and / or flowcharts, can be implemented by a system based on specific purpose hardware that performs the specified functions or actions or a combination of specific purpose hardware and computer instructions.

[0062] The present disclosure includes embodiments according to the following clauses:

[0063] Clause 1. A method (100) for automatically generating an integration test (130) based on unit tests (120), the method (100) comprising the steps of:

[0064] Automatically running (118) unit tests (120) by a processor circuit (704) for each of a plurality of models (104) constituting a system (106);

[0065] Automatically running (132) integration tests (130) by the processor circuit for each set of connected models (104a, 104b) of the plurality of models (104) by using the unit tests (120) for each of the models (104) in the connected models (104a, 104b); and

[0066] Automatically detecting (136) by the processor circuit for each set of connected models (104a, 104b) at least one of inconsistent, incomplete, and incorrect data transmitted between the connected models (104a, 104b).

[0067] Clause 2. The method (100) according to Clause 1, the method (100) further comprising the step of: receiving (102) the plurality of models (104) constituting the system (106), wherein each of the plurality of models (104) includes a low-level model (104) that represents a specific function, a partial function, or a set of functions that the system (106) is configured to perform.

[0068] Clause 3. The method (100) according to Clause 1 or 2, the method (100) further comprising the step of: receiving (102) the plurality of models constituting the system (106), wherein each of the plurality of models (104) includes a low-level model (108) that defines requirements (204, 208), partial requirements, or a set of requirements that the system (106) is configured to meet.

[0069] Clause 4. The method (100) according to Clause 1 or 2, wherein each model (104) includes specific logic circuits (202, 206) that define requirements (204, 208) that the system (106) is configured to meet.

[0070] Clause 5. The method (100) according to Clause 4, wherein each model (104) is configured to: generate one or more expected outputs (210) in response to one or more inputs (212) based on the specific logic circuits (202, 206) associated with the specific model (104).

[0071] Clause 6. The method (100) according to Clause 1 or 2, the method (100) further comprising the step of: generating (110) unit test fixtures (112) for each of the plurality of models (104) constituting the system (106).

[0072] Clause 7. The method (100) according to Clause 6, the method (100) further comprising the step of: generating (114) unit test cases (116) for the unit test fixtures (112) of each model (104), wherein automatically running (118) the unit test (120) for each model (104) includes: using the unit test cases (116) for each model (104).

[0073] Clause 8. The method (100) according to Clause 6, wherein generating (110) the unit test fixtures (112) for each model (104) includes: separating (110) a particular model (104) from the environment of the particular model (104) so that the particular model (104) is independent of other models (104) of other system requirements (204, 208) that provide inputs (212) to the particular model (104).

[0074] Clause 9. The method (100) according to Clause 1 or 2, wherein automatically running (118) the unit test (120) for each model (104) includes: for all combinations of possible different input values (220) of each model (104), generating, in response to the input value (220) of each input in one or more inputs (212) of each model (104), the expected output value (218) of each output in one or more outputs (210) of each model (104).

[0075] Clause 10. The method (100) according to Clause 9, the method (100) further comprising the step of: automatically generating (118) tables (300, 400) for each model (104) independent of other models (104) in response to running the unit test (120) for each model (104), the tables (300, 400) for a particular model (104) including: for various combinations of possible different input values (220) of the particular model (104), the expected output value (218) of each output in one or more outputs (210) respectively associated with the input value (220) of each input in one or more inputs (212).

[0076] Clause 11. The method (100) according to Clause 1 or 2, the method (100) further comprising the step of:

[0077] Automatically generate (122) an integration test fixture (124) for each set of connection models (104a, 104b); and

[0078] Automatically generate (126) integration test cases (128) for each set of connection models (104a, 104b) for the integration test fixture (124), wherein automatically running (132) the integration test (130) for each set of connection models (104a, 104b) includes: using the integration test cases (128) for each set of connection models (104a, 104b).

[0079] Clause 12. The method (100) according to Clause 1 or 2, wherein automatically running (132) the integration test (130) for each set of connection models (104a, 104b) includes: automatically generating a table (600) for each set of connection models (104a, 104b), the table (600) for a particular set of connection models (104a, 104b) including: various combinations (610) of possible different input values (606) for the particular set of connection models (104a, 104b), and for each input of one or more inputs (608) of the particular set of connection models (104a, 104b), an expected output value (602) of each output of one or more outputs (604) of the particular set of connection models (104a, 104b) associated with the input value (606) of the input, the table (600) being used to detect (136) at least one of inconsistencies, incompleteness, and incorrect data between the connection models (104a, 104b) of the particular set of connection models (104a, 104b).

[0080] Clause 13. A method (100) for automatically generating an integration test (130) based on unit tests (120), the method (100) comprising the steps of:

[0081] Receiving (102), by a processor circuit (704), a plurality of models (104) constituting a system (106), wherein each model of the plurality of models (104) includes a low-level model (108) that represents a particular function, partial function, or set of functions that the system (106) is configured to perform;

[0082] Generating (110), by the processor circuit, a unit test fixture (112) for each model (104) of the plurality of models constituting the system (106);

[0083] Generating (114), by the processor circuit, unit test cases (116) for the unit test fixture (112) of each model (104);

[0084] The unit tests (120) are automatically run (118) by the processor circuitry for each model (104) using the unit test cases (116) for each model (104).

[0085] The processor circuitry automatically generates (122) integration test fixtures (124) for each set of connected models (104a, 104b) among the multiple models (104).

[0086] The processor circuitry automatically generates (126) integration test cases (128) for each set of connected models (104a, 104b) for the integration test fixtures (124).

[0087] The processor circuitry automatically runs (132) the integration tests (130) for each set of connected models (104a, 104b) for each model (104) of the connected models (104a, 104b) by using information related to combinations of inputs (212) / outputs (210) and data collected from the unit tests (120); and

[0088] The processor circuitry automatically detects (136) at least one of inconsistent, incomplete, and incorrect data transmitted between the connected models (104a, 104b) for each set of connected models (104a, 104b).

[0089] Clause 14. The method (100) according to clause 13, wherein each model (104) includes specific logic circuitry (202, 206) that defines requirements (204, 208), partial requirements, or a set of requirements that the system (106) is configured to meet, and wherein each model (104) is configured to: generate one or more expected outputs (210) in response to one or more inputs (212) based on the specific logic circuitry (202, 206) associated with the specific model (104).

[0090] Clause 15. The method (100) according to clause 13, wherein automatically running (118) the unit tests (120) for each model (104) includes: for all combinations of possible different input values (220) for each model (104), generating expected output values (218) for each output of one or more outputs (210) of each model (104) in response to the input value (220) of each input among the one or more inputs (212) of each model (104).

[0091] Clause 16. The method (100) according to Clause 15, the method further comprising the steps of: in response to running the unit test (120) for each model (104), automatically generating (118) tables (300, 400) for each model (104) independent of other models (104), the tables (300, 400) for a particular model (104) including: for various combinations of possible different input values (220) of the particular model (104), the expected output values (218) of each output of the one or more outputs (210) respectively associated with the input values (220) of each input in the one or more inputs (212).

[0092] Clause 17. The method (100) according to Clause 13, wherein automatically running (132) the integration test (130) for each group of connected models (104a, 104b) includes: automatically generating a table (600) for each group of connected models (104a, 104b), the table (600) for a particular group of connected models (104a, 104b) including: for various combinations (610) of possible different input values (606) of the particular group of connected models (104a, 104b), the expected output values (602) of each output of the one or more outputs (604) of the particular group of connected models (104a, 104b) associated with the input values (606) of each input in the one or more inputs (608) of the particular group of connected models (104a, 104b), the table (600) for detecting (136) at least one of inconsistencies, incompleteness, and incorrect data between the connected models (104a, 104b) of the particular group of connected models (104a, 104b).

[0093] Clause 18. A system (700) for automatically generating an integration test (130) according to a unit test (120), the system (700) comprising:

[0094] Processor circuits (704, 718); and

[0095] A memory (706, 720) associated with the processor circuits (704, 718), the memory (706, 720) including computer-readable program instructions that, when executed by the processor circuits (704, 718), cause the processor circuits (704, 718) to perform a set of functions (711), the set of functions including:

[0096] Automatically running (118) unit tests (120) for each of a plurality of models (104) constituting a system (106);

[0097] For each set of connected models (104a, 104b) of the plurality of models (104), an integration test (130) is automatically run (132) by using the unit tests (120) for each of the models (104) in the connected models (104a, 104b); and

[0098] For each set of connected models (104a, 104b), at least one of inconsistent, incomplete, and incorrect data transmitted between the connected models (104a, 104b) is automatically detected (136).

[0099] Clause 19. The system (700) according to Clause 18, wherein automatically running (118) the unit tests (120) for each model (104) includes: for all combinations of possible different input values (220) of each model (104), in response to the input value (220) of each input in one or more inputs (212) of each model (104), generating an expected output value (218) of each output in one or more outputs (210) of each model (104), and the function set (711) further includes: in response to running (118) the unit tests (120) for each model (104), automatically generating (118) tables (300, 400) for each model (104) independent of other models (104), and the tables (300, 400) for a specific model (104) include: for various combinations of possible different input values (220) of the specific model (104), the expected output value (218) of each output in one or more outputs (210) associated with the input value (220) of each input in one or more inputs (212).

[0100] Clause 20. The system (700) according to Clause 18 or 19, wherein automatically running (132) the integration test (130) for each group of connection models (104a, 104b) includes: automatically generating a table (600) for each group of connection models (104a, 104b), the table (600) for a specific group of connection models (104a, 104b) including: various combinations (610) of possible different input values (606) for the specific group of connection models (104a, 104b), and for each input value (606) of each input in one or more inputs (608) of the specific group of connection models (104a, 104b), an expected output value (602) of each output in one or more outputs (604) of the specific group of connection models (104a, 104b), the table (600) being used to detect (136) at least one of inconsistencies, incompleteness, and incorrect data among the connection models (104a, 104b) of the specific group of connection models (104a, 104b).

[0101] The terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the embodiments of the present disclosure. Unless the context clearly dictates otherwise, the singular forms "a", "an", and "the" as used herein are also intended to include the plural forms. It will be further understood that when used in this specification, the terms "comprises", "comprising", "includes", and / or "including" specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0102] All structural, material, acts, and equivalents of the corresponding means or steps plus function elements in the appended claims are intended to include any structure, material, or act for performing the function in combination with other claimed elements that are specifically claimed. The description of the present embodiments has been given for purposes of illustration and description, but these descriptions are not intended to be exhaustive or limited to the disclosed forms of the embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the embodiments.

[0103] Although specific embodiments have been illustrated and described herein, those of ordinary skill in the art will appreciate that any structure designed to achieve the same purpose may be used in place of the specific embodiments shown, and these embodiments have other applications in other environments. This application is intended to cover any alterations or variations. The appended claims are in no way intended to limit the scope of the embodiments of the present disclosure to the specific embodiments described herein.

Claims

1. A method for automatically generating integration tests based on unit tests during model-based development, the method comprising the following steps: receiving, by a processor circuit, a plurality of models constituting a system, wherein each model includes a logic circuit that defines requirements, partial requirements, or a set of requirements that the system is configured to meet, and wherein each model is configured to: generate one or more expected outputs in response to one or more inputs based on the logic circuit associated with the specific model; automatically running, by the processor circuit, unit tests for each of the plurality of models constituting the system by: for all combinations of possible different input values for each model, generating an expected output value for each output in one or more outputs of each model in response to the input value of each input in one or more inputs of each model, wherein each specific model is unit tested independently of other models for other system requirements that provide inputs to the specific model; determining multiple sets of connected models of the plurality of models, wherein each set of connected models includes two or more models connected to each other; automatically generating, by the processor circuit, a table for each set of connected models from the unit tests of each model and automatically running integration tests for each set of connected models of the plurality of models by using the unit tests for each model in the connected models, and for various combinations of possible different input values for a specific set of connected models, the table for the specific set of connected models includes an expected output value for each output in one or more outputs of the specific set of connected models associated with the input value of each input in one or more inputs of the specific set of connected models; automatically detecting, by the processor circuit, at least one of inconsistent, incomplete, and incorrect data transmitted between the connected models for each set of connected models, wherein the table for the specific set of connected models is used to detect at least one of inconsistent, incomplete, and incorrect data between the connected models of the specific set of connected models.

2. The method according to claim 1, wherein each of the plurality of models includes a low-level model that represents a specific function, partial function, or set of functions that the system is configured to perform.

3. The method according to claim 1 or 2, wherein each of the plurality of models includes a low-level model that defines requirements, partial requirements, or a set of requirements that the system is configured to meet.

4. The method according to claim 1 or 2, the method further comprising the following steps: generating, for each of the plurality of models constituting the system, a unit test fixture, wherein the unit test fixture is used to isolate the model.

5. The method according to claim 4, the method further comprising the following steps: generating unit test cases for the unit test fixture for each model, wherein the step of automatically running unit tests for each model includes: using the unit test cases for each model.

6. The method according to claim 4, wherein The steps for generating unit test fixtures for each model include: separating a specific model from the environment of the specific model so that the specific model is independent of other models of other system requirements that provide inputs to the specific model.

7. The method according to claim 1, the method further comprises the following steps: In response to running the unit test for each model, automatically generating a table for each model independent of other models, the table for a specific model including: for various combinations of possible different input values of the specific model, the expected output values of each output in the one or more outputs respectively associated with the input values of each input in the one or more inputs.

8. The method according to claim 1 or 2, the method further comprises the following steps: Automatically generating an integration test fixture for each group of connected models to isolate the models; and Automatically generating integration test cases for each group of connected models for the integration test fixture, wherein the step of automatically running the integration test for each group of connected models includes: using the integration test cases for each group of connected models.

9. A system for automatically generating integration tests based on unit tests during model-based development, the system comprises: A processor circuit; and A memory associated with the processor circuit, the memory including computer-readable program instructions that, when executed by the processor circuit, cause the processor circuit to perform a set of functions, the set of functions including: Receiving a plurality of models constituting a system, wherein each model includes a logic circuit that defines the requirements, partial requirements, or a set of requirements that the system is configured to meet, and wherein each model is configured to: based on the logic circuit associated with a specific model, generate one or more expected outputs in response to one or more inputs; Automatically running unit tests for each of the plurality of models constituting the system by: for all combinations of possible different input values of each model, generating the expected output values of each output in the one or more outputs of each model in response to the input values of each input in the one or more inputs of each model, wherein each specific model is unit tested independent of other models of other system requirements that provide inputs to the specific model; Determining multiple groups of connected models of the plurality of models, wherein each group of connected models includes two or more models connected to each other; Automatically running an integration test for each group of connected models of the plurality of models by automatically generating a table for each group of connected models from the unit tests of each model, for various combinations of possible different input values of a specific group of connected models, the table for the specific group of connected models including the expected output values of each output in the one or more outputs of the specific group of connected models respectively associated with the input values of each input in the one or more inputs of the specific group of connected models; Automatically detect at least one of inconsistent, incomplete, and incorrect data transmitted between the connection models for each group of connection models, wherein the table for the specific group of connection models is used to detect at least one of inconsistent, incomplete, and incorrect data between the connection models of the specific group of connection models.

10. The system according to claim 9, wherein, the set of functions further includes: in response to running the unit test for each model, automatically generating a table for each model independent of other models, and the table for a specific model includes: for various combinations of possible different input values for the specific model, the expected output value of each output among the one or more outputs associated with the input value of each input among the one or more inputs.

Images