System and method for testing and validating an application

Abstract

A method for testing a physical system including software and hardware includes: developing a software application arranged to run at least a portion of the physical system; and building a simulation of the physical system within a test ecosystem, the simulation including an input to simulate a control input to the physical system and an output to simulate a control and information output. The method also includes: connecting the software application to the simulation of the physical system to test the operation of the software application; and simulating the operation of the physical system and the software application within the test ecosystem, the test ecosystem being operable to simulate the operation of the software application as each of a cloud application, a network application, and a local application.

Description

Technical Field

[0001] The present invention generally relates to a system and method for testing software applications, and more particularly to a system and method for testing the operation of a software application using a physical system. Background Art

[0002] Developing software solutions for running and automating products such as turbines, trains, factories, etc. can be challenging because the software must be tested under different operating conditions that are often unpredictable. An application may run satisfactorily on the cloud in one case, or in a fog-based platform in another, yet the same application may not run unless it is hosted locally in an embedded controller connected to the process or machine it controls. Since operation in large industrial systems varies greatly, significant testing must be performed under widely varying conditions. Summary of the invention

[0003] A method for testing a physical system including software and hardware includes: developing a software application arranged to run at least a portion of the physical system; and building a simulation of the physical system within a test ecosystem, the simulation including an input to simulate a control input to the physical system and an output to simulate a control and information output. The method also includes connecting the software application to the simulation of the physical system to test the operation of the software application, and simulating the operation of the physical system and the software application within the test ecosystem, the test ecosystem being operable to simulate the operation of the software application as each of a cloud application, a network application, and a local application.

[0004] In another configuration, a method for testing a physical system including software and hardware includes: establishing a cloud-based test ecosystem that simulates an operating system and system arrangement; and constructing a simulation of the physical system within the test ecosystem by selecting elements from a library of simulated elements, each element simulating a physical component and including a delay value that simulates a delay inherent in each physical component. The method also includes: placing a software application within the test ecosystem, the software application connected to the simulation of the physical system to transmit input to the simulation of the physical system and receive output from the simulation of the physical system; running the simulation of the physical system; and outputting performance data related to the running of the software application and the simulation of the physical system.

[0005] The technical features of the present application have been summarized quite broadly above so that those skilled in the art can better understand the following detailed description. The additional features and advantages of the present application that form the subject matter of the claims will be described subsequently. Those skilled in the art will appreciate that they can easily use the disclosed schemes and specific embodiments as the basis for modifying or designing other structures to achieve the same purpose as the present application. Those skilled in the art will also recognize that such equivalent constructions do not depart from the spirit and scope of the present application in its broadest form.

[0006] In addition, before the detailed description below, it should be understood that various definitions of certain words and phrases are provided in this specification, and those skilled in the art will understand that such definitions apply in many, if not most, cases to prior and future uses of such defined words and phrases. Although some terms may include a variety of embodiments, the appended claims may expressly limit these terms to specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 A schematic diagram showing a testing ecosystem for testing new applications.

[0008] Figure 2 A flow chart illustrating the application testing process is shown.

[0009] Before explaining any embodiment of the present invention in detail, it should be understood that the present invention is not limited in its application to the details of the construction and the arrangement of parts set forth in the following description or shown in the following drawings. The present invention is capable of other embodiments and can be practiced or performed in various ways. It should also be understood that the words and terms used herein are for descriptive purposes and should not be considered restrictive. DETAILED DESCRIPTION

[0010] Various techniques related to systems and methods are now described with reference to the accompanying drawings, wherein the same reference numerals always represent the same elements. The various embodiments used to describe the principles of the present application in the accompanying drawings discussed below and this patent text are only for illustration and should not be interpreted as limiting the scope of the present application in any way. It will be appreciated by those skilled in the art that the principles of the present application can be implemented in any appropriately arranged device. It will be appreciated that the functions described as being performed by a specific system element can be performed by multiple elements. Similarly, for example, an element can be configured to perform the functions described as being performed by multiple elements. The numerous creative teachings of the present application will be described with reference to exemplary, non-limiting embodiments.

[0011] It should also be understood that, unless explicitly limited in some examples, the words or phrases used herein should be interpreted broadly. For example, the terms "include", "have" and "include" and their derivatives are meant to include but are not limited to. The singular form is also intended to include the plural form, unless the context clearly states otherwise. In addition, the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more associated listed items. Unless the context clearly states otherwise, the term "or" is inclusive, i.e., means and / or. The phrases "associated with..." and "associated with" and their derivatives may mean including, included therein, interconnected, including, contained within, connected to or connected with it, coupled to or coupled with it, communicable with it, cooperate, interlace, parallel, close to, bound to or bound with it, have, have... attributes, etc.

[0012] In addition, although the terms "first", "second", "third", etc. can be used in this article to refer to various elements, information, functions or actions, these elements, information, functions or actions should not be limited by these terms. To be precise, these digital adjectives are used to distinguish different elements, information, functions or actions. For example, the first element, information, function or action can be referred to as the second element, information, function or action, and similarly, the second element, information, function or action can be referred to as the first element, information, function or action, without departing from the scope of the present application.

[0013] In addition, the term "adjacent" may mean that one element is relatively close to but not in contact with another element, or that the element is in contact with another part, unless the context clearly indicates otherwise. In addition, unless otherwise clearly indicated, the phrase "based on" is intended to mean "based at least in part on". The terms "approximately" or "substantially" or similar terms are intended to encompass variations in values ​​within normal industrial manufacturing tolerances for that dimension. If no industry standard is available, variations of 20% are included in the meaning of these terms unless otherwise indicated.

[0014] Figure 1 A system or environment 10 for testing software applications 15 and, in particular, control applications for use with a hardware system 20 is schematically illustrated. The illustrated system 10 is particularly suitable for testing applications on hardware 20 that are not easily tested in actual operation. For example, power generation turbines are high energy machines that can be severely damaged if operated outside of normal operating parameters. Testing the software 15 on an actual machine may result in such undesirable operation and is therefore difficult and potentially costly to perform.

[0015] Figure 1A test environment or test ecosystem 10 is shown that is well suited for testing new software applications 15 using existing hardware 20. The test ecosystem 10 shown is preferably cloud-based, but can also be network-based or maintained locally on a user's computer. The test ecosystem 10 provides an environment for users to build a virtual model 25 of a physical system 30, which includes a hardware model 35 and a software model 40. A database or library 45 of common physical properties or components (such as valves, hydraulic devices, turbines, motors, etc.) is provided to allow users to build virtual models 25. In addition, as users develop simulations of new components, these simulations can be added to the library 45. Each virtual model 25 or component simulates the actual operation of the physical system 30 or component, including the expected delay or delay in the operation of the system 30 or component. For example, a simulation of a physical hydraulic valve may receive a signal indicating that it is desired to open the valve. The simulation can move the valve to the desired position immediately, but this is not realistic. The delay value as part of each simulation accounts for the real-world physical delay by delaying the movement of the valve in the simulation. The simulation can include machine learning and artificial techniques that provide similar or identical outputs based on historical data of the operation of such a system in the real world.

[0016] The test environment 10 also provides access to a library of containers 50 for use with the software application 15 to be tested. The container 50 is a software package that receives the software application 15 to allow the software application 15 to run on any operating system including Linux, Windows, iOS, QHX, BSD, etc. In simple terms, the container 50 includes an internal connection 65 that connects the code segment 70 or the input / output module 75 as needed. The container 50 also includes an external connection for any operating system. When outputting from the code segment 70 or the input / output module 75, the container 50 therefore obtains an internal signal from the code segment 70 or the input / output module 75 and adjusts the signal according to the requirements of any available operating system. Subsequently, the appropriate output signal can be used for any operating system with the following signal, which is a unique external connection for the operating system used. When the signal is input, the process is reversed. The signal comes from the operating system used and enters the container 50 through the appropriate external connection. Subsequently, the signal is adjusted and transmitted to the code segment 70 or the input / output module 75.

[0017] The simulation module 55 provides a connection between the application model 40, more specifically the container 50 or multiple containers and the hardware model 35. The simulation module 55 can be configured to simulate a variety of different arrangements of the application model 40 and the hardware model 35. For example, the simulation module 55 can be arranged to simulate the application 15 as a cloud-based application interacting with the hardware system 20, or can simulate the application 15 running on a server away from the hardware 20 or at the same physical location. The simulation module 55 simulates the expected delay between the hardware system 20 and the application 15 according to the selected architecture between the two. The virtual model 25 can then simulate the application 15 in a cloud-based configuration, a network-based configuration, as a locally stored application, or any other arrangement that may be beneficial. The generated data can be sent to the virtualization management system 60 and collected for further analysis.

[0018] like Figure 1 As shown, the virtual model 25 or digital twin of the physical system 30 is built in a cloud-based or offline ecosystem 10 that provides a scalable development and pure digital deployment environment. A library 45 can be provided based on the system required to allow the creation of a digital twin 25 of the target physical system 30 before implementing the physical system 30 and the control application 15. The digital twin 25 can be composed of small and large simulations of the real physical system 30 and digital entity models (mock-up). The digital twin 25 runs in a virtualized infrastructure in a cloud or other environment as needed. The digital twin 25 includes exposed interfaces that allow programming and testing of simple to complex control systems (such as SCADA, MES, etc.). Subsequently, the application 15 is deployed in the test ecosystem 10 to test the application 15 and the physical system 20 together in the container 50 and the virtual machine that recreates the physical system 30. Application resources and loads can be measured and quantified while running in close collaboration with the simulated digital twin 25 of the physical system 30.

[0019] After the test run, the application's requirements in terms of computing power and other resources are known. The system 10 also tests and measures the latency bounds of the application 15 interacting with the simulated physical system 35. These latency and performance bounds can be provided to a virtualization management system 60 (e.g., DMTF Redfish), which can redeploy the same container 55 and virtual model 25 to the correct type of computing element (e.g., edge cloud, server rack, or number of PLCs) while meeting the described requirements. The same applies to communication requirements that most affect latency, which can be derived from the system 10 before describing the configuration requirements of the physical system 30. Such information can be provided for technologies such as software-defined networks. TOSCA, provided by Tricentis, is an exemplary declarative language for describing the requirements of an application 15 when deployed in any computing and network environment.

[0020] The virtual model 25 and container 50 should be replicated as is and placed in a computing element that meets the latency and resource requirements determined using the testing ecosystem 10 .

[0021] The testing ecosystem 10 described herein greatly improves the reliability of software 15 implemented with industrial equipment such as turbines, trains, etc. In addition, the system 10 reduces the likelihood of damage or improper operation of the equipment 20 that often occurs during testing or initial operation of new software components 15.

[0022] Figure 2 Shows the use of Figure 1 The process of testing an ecosystem 10 of a physical system. The initial step in the process is to build a model of the physical system within the test environment (step 110). The build can utilize previous components built and stored in the component library 45. New applications can be developed while, before, or after building a digital twin or virtual model of the physical system (step 115). Subsequently, the application is placed in the test ecosystem together with the digital twin 35 of the hardware part of the physical system 30 (step 120). Subsequently, the application 40 is typically connected to the system model 35 through the simulation module 55 in step 125. Before testing, any test parameters, such as operating system, application environment (e.g., cloud-based, local, etc.) and any other deferred parameters are typically applied to the system through the simulation module 55 (step 130). Subsequently, the simulation is run in step 135 and the results are output in step 140. If you want to change the test parameters and rerun the test, then change these parameters and repeat the process from step 125 (step 145). Once the desired data is collected, the simulation ends.

[0023] A control system can (independently of what we want to control) be decomposed into sensors, actuators, and controller logic. The software representing the controller logic typically reads the state of the actuators and sensors and infers the next change in the process state, which is converted and sent as input to the various actuators.

[0024] The simulation allows the software developer to write the correct control logic, but to test the behavior of the control software running under different conditions, for example adding delays between the control application and its process (actuators / sensors) to find out the optimal execution and communication conditions. If the sensor value is not transmitted on time, or the command sent to the actuator ("open the valve") is received too late, then a given delay threshold may cause the process to be in an incorrect state or fail. Simulation can capture such operating ranges during the development of the operation.

[0025] Subsequently, it is also used to describe requirements on how the application should be deployed. In terms of latency, both the execution environment and the network end-to-end QoS play a role in achieving the minimum latency levels described in the simulation steps above.

[0026] Deployment management of containerized applications or applications running in virtual machines includes: allocating and guaranteeing compute and memory for a given container or virtual machine; and also being able to predict network latency through, for example, a software-defined network controller or some network management system that requires certain requirements about latency and bandwidth as input; and responding based on whether the network path to the cloud or edge server or local embedded controller is optimal.

[0027] Containers and virtual machines can be deployed anywhere as long as the requirements are met.

[0028] There may be a cost function that determines that delay-tolerant applications should not utilize the precious and limited resources on edge and localized controller devices. Therefore, it is beneficial to shift the load to the cloud.

[0029] The deployment can also be tested again against a simulation of the process after it has occurred before allowing the control logic to run the real system.

[0030] This becomes even more important as controller software moves toward more modern programming and execution foundations, such as event-driven and publish / subscribe types of communication between processes and applications, and also for AI-based control logic, which requires more computing and storage resources that only edge servers or cloud environments can provide.

[0031] Although exemplary embodiments of the present application have been described in detail, those skilled in the art should understand that various changes, substitutions, variations, and modifications may be made to the present application without departing from the spirit and scope of the present application in its broadest form.

[0032] Nothing in this application should be construed as implying that any particular element, step, act, or function is essential to the claims: the scope of patented subject matter is limited solely by the claims that are issued. Furthermore, none of the claims are intended to involve means-plus-function claim constructions unless the exact words "means for..." are followed by a verb.

Claims

1. A method for testing a physical system including software and hardware, the method comprising: developing a software application arranged to operate at least a portion of the physical system; constructing a simulation of the physical system within the test ecosystem, the simulation including an input to simulate control inputs to the physical system and an output to simulate control and information outputs; Connecting software applications to simulations of physical systems to test the operation of the software applications; and simulating the operation of the physical system and the software application within the test ecosystem, the test ecosystem being operable to simulate the operation of the software application as each of a cloud application, a web application, and a local application, The method includes providing a library of simulation components, each simulation component being selectively used to simulate a component of a physical system to construct a simulation of the physical system, and Wherein, the simulation component library includes a simulated fluid power actuator, and wherein, the simulated fluid power actuator includes an input valve position and an input fluid pressure, and wherein, the simulated fluid power actuator outputs a current position and a time to reach a desired position.

2. The method of claim 1 , further comprising providing one of a container and a virtual machine, and placing the software application in one of the container and a virtual machine (VM), the one of the containers providing input and output connections that are independent of an operating system on which the container and the software application run.

3. The method according to claim 1, wherein: Each simulation element includes a delay value that simulates the delay inherent in each component of the physical system.

4. The method according to claim 3, wherein: The analog component library includes simulated sensors, and the simulated sensors include output terminals for outputting measured values.

5. The method according to claim 4, wherein: The simulated sensor simulates a thermocouple, and wherein the holdoff value is related to a thermal inertia of the thermocouple.

6. The method according to claim 1, wherein: The physical system includes one of an engine, a building, and a factory.

7. A method for testing a physical system including software and hardware, the method comprising: Establishing a cloud-based testing ecosystem that simulates operating systems and system arrangements; constructing a simulation of a physical system within the test ecosystem by selecting elements from a library of simulation elements, each element simulating a physical component and including a delay value simulating a delay inherent in each physical component; placing a software application within the test ecosystem, the software application connected to the simulation of the physical system to transmit inputs to the simulation of the physical system and receive outputs from the simulation of the physical system; Run simulations of physical systems; and Output performance data related to the operation of software applications and simulation of physical systems, Wherein, the simulation component library includes a simulated fluid power actuator, and wherein, the simulated fluid power actuator includes an input valve position and an input fluid pressure, and wherein, the simulated fluid power actuator outputs a current position and a time to reach a desired position.

8. The method of claim 7, further comprising providing a container and placing the software application within the container, the container providing input and output connections that are independent of an operating system on which the container and the software application run.

9. The method according to claim 7, wherein: Each simulation element includes a delay value that simulates the delay inherent in each component of the physical system.

10. The method according to claim 9, wherein: The analog component library includes simulated sensors, and the simulated sensors include output terminals for outputting measured values.

11. The method according to claim 10, wherein: The simulated sensor simulates a thermocouple, and wherein the holdoff value is related to a thermal inertia of the thermocouple.

12. The method according to claim 7, wherein: The physical system includes one of an engine, a building, and a factory.

13. The method according to claim 7, wherein: The test ecosystem simulates each of the Linux, Windows, iOS, QHX, and BSD operating systems and outputs performance data for each operating system.

14. The method according to claim 7, wherein: The test ecosystem simulates system arrangements including execution of software applications as cloud-based applications, web-based applications, local applications, and PLC-based applications, and outputs performance data for each system arrangement.

15. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method of claim 7.

Images