System and method for provisioning cross-domain test

Abstract

To provide a method, system, and device for facilitating provisioning of a cross-domain test for testing software of an embedded system.SOLUTION: A method may be implemented by at least one processor and may include: detecting a change in the software; obtaining a test configuration file associated with the software; obtaining, based on the test configuration file, a test bench; determining a plurality of test environments associated with the test bench; and performing the cross-domain test based on the plurality of test environments. The software of an embedded system may include an in-vehicle electronic control unit (ECU). The plurality of tests environments may include at least one software-in-the-loop (SIL) test environment, and at least one hardware-in-the-loop (HIL) test environment.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] Systems and methods consistent with exemplary embodiments of the present disclosure relate to test provisioning, and more particularly, to systems and methods for provisioning different simulations and / or test environments for cross-domain testing.

Background Art

[0002] Conventionally, various test environments have been introduced to test the functions and software performance of systems. As an example, an in-vehicle system such as an electronic control unit (ECU) can be tested, among other things, by utilizing at least one software-in-the-loop (SIL) test environment and at least one hardware-in-the-loop (HIL) test environment.

[0003] To develop advanced or complex functions within a system, testing of the functions and performance of multiple software and hardware components across multiple systems or domains is required. In a vehicle system, cross-domain testing may be required during the development of advanced functions in vehicle systems such as lane change assistance, mobile smart keys, etc., and cross-domain testing may include testing the interactions between different ECUs and related software and hardware components.

[0004] Simply put, cross-domain testing can refer to testing that involves testing the functions of components across different domains or environments, such as different operating environments, different system configurations, different hardware and / or software configurations, and / or the like. The purpose of cross-domain testing is to ensure that system components function correctly across all of these domains under different situations.

[0005] Therefore, cross-domain testing can have the following characteristics. (1) Cross - domain testing often presents integration challenges because a minor change in an ECU can significantly impact the functionality and stability of other related ECUs and associated components, and may require careful integration management among the components of the system. (2) Cross - domain testing may require specific hardware, software, and / or system configurations, and it can be difficult or time - consuming to set up and maintain a test environment that accurately meets the test requirements. Therefore, cross - domain testing is often subject to test environment constraints. (3) Cross - domain testing often involves multiple users (e.g., teams, stakeholders, vehicle manufacturers, suppliers, vendors, etc.) who may be located in different places and have different priorities, goals, backgrounds, communication styles, etc. Thus, effective communication and collaboration are required to perform accurate cross - domain testing, and cross - domain testing often has complex user - to - user communication and collaboration.

[0006] Considering the above, in the prior art, every time cross - domain testing is required, multiple ECUs and related software / hardware components need to be aggregated and placed in a unified test facility (e.g., a local simulation / test facility, etc.), and users / test personnel need to physically visit there to perform cross - domain testing within it. Nevertheless, the approach for performing cross - domain testing in the prior art has at least the following drawbacks.

[0007] First, it is burdensome and time-consuming for a user to physically visit a test facility. For example, a user may be located at a location geographically different from the test facility (e.g., a different region, a different country, etc.), and thus, physically visiting the test facility requires appropriate planning and can be costly (e.g., the user may need to take a long flight to the test facility, or may need to appropriately cooperate with other relevant users to reserve the test facility to schedule the test, or may need to apply for a visit visa and / or permit, and it may take time to obtain approval). In addition, since the available devices, hardware, etc. within the test facility are limited, users usually need to wait for a long time (e.g., be put on a waiting list, etc.) until they can use the test facility to perform cross-domain tests.

[0008] Furthermore, in the prior art, it is difficult to quickly and accurately identify or discover breaking changes during cross-domain testing. Specifically, since multiple components within the local test facility may be changed after being used by different users (e.g., in addition to the components being tested, related components may also be changed), if the configuration of the components is not restored to the required state, the cross-domain test performed based on the multiple changed components may not be accurate, and the user may not be able to quickly and accurately identify the breaking changes through the cross-domain test (e.g., the problem caused by the change of the component being tested may be discovered in the cross-domain test, but the user may misunderstand the problem as being caused by the change of other components, or the problem may be corrected by the change of other components and not be discovered).

[0009] Furthermore, in the prior art, the configuration of the test facility is not flexible, and it is difficult to reconfigure the cross-domain test on-the-fly or on-demand. For example, even if a user / test person requests to add a new ECU or change the ECU involved in the test facility, if the requested ECU is not immediately available, the user / test person may need to re-reserve the test facility, wait until the next test, and then revisit the test facility for further testing. Therefore, in the prior art, even if a destructive change is discovered during the cross-domain test, it is difficult to immediately correct the destructive change on the spot and re-execute the test immediately after that.

[0010] Considering the above, in the prior art, it takes a long time to execute the cross-domain test, and most of the time is spent on arranging travel, moving, waiting in line for testing, etc. As a result, the approach for executing the cross-domain test in the prior art is inefficient. In the prior art, the cross-domain test cannot be performed on demand, performing the cross-domain test is a burden on the user, and it is difficult to discover and immediately correct the destructive change of the component during the test. Ultimately, these can increase the lead time from the system-on-chip (SoC) specification to the start of vehicle production (SOP: start of production).

Summary of the Invention

[0011] According to an embodiment, a method, a system, and a device are provided for automatically facilitating a cross-domain test for testing one or more softwares of a system. For example, the method, the system, and the device can automatically determine whether the execution of the test should be triggered, automatically determine an appropriate test environment for executing the cross-domain test, and automatically execute the cross-domain test.

[0012] According to an embodiment, a method is provided for facilitating the provisioning of cross-domain testing for testing the software of an embedded system. The method may be implemented on at least one processor, and includes detecting a change in the software, obtaining a test configuration file associated with the software, obtaining a test bench based on the test configuration file, determining a plurality of test environments associated with the test bench, and executing the cross-domain testing based on the plurality of test environments. The software of the embedded system may include an in-vehicle electronic control unit (ECU), and the plurality of test environments may include at least one software-in-the-loop (SIL) test environment and at least one hardware-in-the-loop (HIL) test environment.

[0013] According to an embodiment, the test configuration file may include information on a plurality of ECUs associated with the software and information on test environments associated with each of the plurality of ECUs. At least a part of the plurality of ECUs may be associated with one or more nodes different from the software. Also, at least a part of the plurality of ECUs may be distributed and arranged at geographically different positions from the software. Further, the plurality of ECUs may include at least one virtual ECU, at least one emulated ECU, at least one physical ECU, or a combination thereof. Furthermore, the plurality of ECUs may include at least one of a central ECU (CECU), an instrument cluster (IC) ECU, an in-vehicle infotainment (IVI) ECU, and an advanced driver assistance system (ADAS) ECU.

[0014] According to an embodiment, the change of software may include a destructive change. Also, detecting a change in software includes obtaining information on the current status of the software from nodes related to the software, determining whether the software has been changed from a previous version based on the obtained status information, and determining whether the change is a destructive change based on the determination that the software has been changed.

[0015] According to an embodiment, determining a plurality of test environments may include creating a test job including a plurality of tasks based on the obtained test bench, and selecting a plurality of test environments based on one or more requirements for executing the plurality of tasks. Also, performing a cross-domain test may include allocating one or more tasks of the test job to a plurality of test environments, receiving test results related to the allocated one or more tasks from the plurality of test environments, and generating a test result of the cross-domain test based on the test results related to the allocated one or more tasks.

[0016] According to an embodiment, a system is provided for facilitating the provisioning of cross-domain testing for testing the software of an embedded system. The system may include at least one memory storage storing computer-executable instructions, and at least one processor communicatively coupled to the at least one memory storage and configured to execute computer-executable instructions for detecting changes in software, obtaining a test configuration file associated with the software, obtaining a test bench based on the test configuration file, determining a plurality of test environments associated with the test bench, and executing a cross-domain test based on the plurality of test environments. The software of the embedded system may include an in-vehicle electronic control unit (ECU), and the plurality of test environments may include at least one software-in-the-loop (SIL) test environment and at least one hardware-in-the-loop (HIL) test environment.

[0017] According to an embodiment, the test configuration file may include information on a plurality of ECUs associated with the software and information on test environments associated with each of the plurality of ECUs. At least a portion of the plurality of ECUs may be associated with one or more nodes different from the software. Also, at least a portion of the plurality of ECUs may be distributed and arranged at geographically different locations from the software. Further, the plurality of ECUs may include at least one virtual ECU, at least one emulated ECU, at least one physical ECU, or a combination thereof. Still further, the plurality of ECUs may include at least one of a central ECU (CECU), an instrument cluster (IC) ECU, an in-vehicle infotainment (IVI) ECU, and an advanced driver assistance system (ADAS) ECU.

[0018] According to an embodiment, a change to software may include a breaking change. Also, at least one processor may obtain information on the current status of the software from a node related to the software, determine whether the software has been changed from a previous version based on the obtained status information, and determine whether the change is a breaking change based on determining that the software has been changed, thereby executing computer-executable instructions for detecting a change to the software.

[0019] According to an embodiment, at least one processor may create a test job including a plurality of tasks based on the obtained test bench, and select a plurality of test environments based on one or more requirements for executing the plurality of tasks, thereby executing computer-executable instructions for determining a plurality of test environments. Also, at least one processor may allocate one or more tasks of the test job to a plurality of test environments, receive test results related to the allocated one or more tasks from the plurality of test environments, and generate a test result of the cross-domain test based on the test results related to the allocated one or more tasks, thereby executing computer-executable instructions for performing a cross-domain test.

[0020] Additional aspects are described in part in the following description, are partially apparent from the description, or may be realized by practicing the presented embodiments of the disclosure.

Brief Description of the Drawings

[0021] The features, advantages, and significance of exemplary embodiments of the present disclosure will be described below with reference to the accompanying drawings in which like reference numerals indicate like elements.

[0022]

Figure 1

[0023]

Figure 2

[0024]

Figure 3

[0025]

Figure 4

[0026]

Figure 5

[0027]

Figure 6

[0028] The following detailed description of the embodiments as examples refers to the accompanying drawings. The foregoing disclosure provides illustrations and descriptions, but is not intended to be exhaustive, nor is it intended to limit the embodiments to the exact forms disclosed. Modifications and variations are possible in light of the above disclosure, or can be obtained through the practice of the embodiments. Additionally, one or more features or components of one or more embodiments can be incorporated into, or combined with, other embodiments (or one or more features of other embodiments). Further, in the description of the operations provided below, it is understood that one or more operations can be omitted, one or more operations can be added, one or more operations can be performed simultaneously (at least in part), and the order of one or more operations can be switched.

[0029] Even if a particular combination of features is recited in the claims and / or disclosed herein, that combination is not intended to limit the disclosure of possible implementations. In fact, many of the features can be combined in ways not specifically recited in the claims and / or not disclosed herein. Each of the dependent claims listed below can depend directly on only one claim, but the disclosure of possible implementations includes each dependent claim in combination with all other claims in the claim set.

[0030] Any element, operation, or instruction used in this specification should not be construed as decisive or essential unless explicitly described as such. Also, as used in this specification, the articles "a" and "an" are intended to include one or more items and can be used interchangeably with "one or more." When only one item is intended, the term "one" or similar language is used. Also, as used in this specification, terms such as "have," "having," "include," "including," etc. are intended to be open-ended terms without limitation. Further, the phrase "based on" is intended to mean "at least partially based on" unless explicitly described otherwise. Further, expressions such as "at least one of [A] and [B]" or "at least one of [A] or [B]" should be understood to include only A, only B, or both A and B.

[0031] References throughout this specification to "one embodiment," "an embodiment," "a non-limiting preferred embodiment," or similar terms mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the solution. Thus, the phrases "in one embodiment," "in an embodiment," "in a non-limiting preferred one embodiment," and similar terms throughout this specification may all refer to the same embodiment, but not necessarily so.

[0032] Furthermore, the features, advantages, and characteristics of the present disclosure described may be combined in any suitable manner in one or more embodiments. One skilled in the art will recognize, in view of the description of this specification, that the present disclosure may be practiced without using one or more of the particular features or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in particular embodiments that may not be present in all embodiments of the present disclosure.

[0033] Exemplary embodiments consistent with the present disclosure provide a method, system, and apparatus for facilitating the provisioning of cross - domain testing for one or more softwares of an embedded system without geographical or location restrictions.

[0034] Specifically, methods, systems, apparatuses, etc. of exemplary embodiments can automatically detect one or more changes to one or more softwares of an embedded system, and accordingly can automatically configure and execute cross - domain testing for one or more softwares. According to embodiments, methods, systems, apparatuses, etc. of the embodiments can automatically determine a plurality of test environments and execute cross - domain testing based on the plurality of test environments. In some implementations, when one or more changes are detected, one or more configuration files associated with one or more softwares are obtained, and a test bench can be obtained based on the one or more configuration files. Thus, a plurality of test environments can be determined based on the test bench, and methods, systems, apparatuses, etc. of exemplary embodiments can automatically assign one or more tasks to a plurality of test environments for executing cross - domain testing regardless of where the plurality of test environments and the components involved in the testing are located.

[0035] For this purpose, methods, systems, and apparatuses consistent with exemplary embodiments of the present disclosure automatically facilitate the provisioning of an appropriate test environment for cross - domain testing when needed without being subject to geographical restrictions. A user can remotely configure one or more conditions for triggering cross - domain testing, and the execution of cross - domain testing can be automatically triggered based thereon without requiring the user to physically visit a test facility. Further, breaking changes can be quickly and easily discovered and corrected, and the reconfigured cross - domain testing can be resumed or re - executed immediately when needed.

[0036] Ultimately, the exemplary embodiments of the present disclosure can enable more efficient software development, significantly reduce the user's burden, significantly reduce the development time, and significantly reduce the costs and labor for planning physical visits to test facilities and business trips.

[0037] It is intended that the features, advantages, and significance of the above exemplary embodiments are only part of the present disclosure, and are not intended to be comprehensive or to limit the technical scope of the present disclosure. Further descriptions of the features, components, configurations, operations, and implementations of the exemplary embodiments of the present disclosure are provided below.

[0038] FIG. 1 is a block diagram of an exemplary system configuration 100 for facilitating the provisioning of cross-domain testing according to one or more embodiments. As shown in FIG. 1, the system configuration 100 may include a test management system 110, a plurality of nodes 120-1 to 120-N, a network 130, and a plurality of test environments 140-1 to 140-N.

[0039] Generally, the test management system 110 is communicatively coupled to the plurality of nodes 120-1 to 120-N (via the network 130) and to the plurality of test environments 140-1 to 140-N, and is configured to provide cross-domain testing for one or more components (e.g., virtual ECUs, physical ECUs, etc.) associated with the plurality of nodes using the plurality of test environments. An explanation of exemplary components that may be included in the test management system 110 is provided below with reference to FIG. 2, and one or more operations executable by the test management system 110, as well as related use cases, are provided below with reference to FIGS. 3 to 6.

[0040] Each of the plurality of nodes 120-1 to 120-N may include one or more devices, apparatuses, systems, or any other suitable components capable of performing reception, hosting, storage, deployment, processing, and / or provision of one or more devices, apparatuses, systems, or one or more components constituting the system.

[0041] As an example, node 120-1 may include a device or apparatus (such as a personal computer, a server or server cluster, a workstation, etc.) that can be used for building, storing, executing, or simulating one or more computer-executable software applications, such as one or more virtualized ECUs, one or more emulated ECUs, and / or any other suitable software-based components (such as a vehicle model, a data communication module (DCM) model, a heating, ventilation, and air conditioning (HVAC) model, etc.) of a vehicle system. As another example, node 120-1 may include or be associated with one or more hardware components, such as one or more fully developed physical ECUs, one or more partially developed physical ECUs, one or more vehicle hardwares (such as a powertrain, etc.). Additionally or alternatively, the node may include one or more retarget equipment.

[0042] According to an embodiment, one or more of the plurality of nodes 120-1 to 120-N may include one or more interfaces, each of which may be configured to communicatively couple the associated node to the test management system 110. For example, one or more of the plurality of nodes may include a program interface, a hardware interface, a software interface (such as an application program interface (API), etc.).

[0043] According to an embodiment, at least a part of the plurality of nodes 120-1 to 120-N is located at one or more geographical locations that are different from the test management system 110, different from another part of the plurality of nodes, and / or different from the plurality of test environments 140-1 to 140-N.

[0044] The network 130 may include one or more wired and / or wireless networks configured to couple the plurality of nodes 120-1 to 120-N to the test management system 110. For example, the network 130 may include a cellular network (e.g., a fifth generation (5G) network, a long-term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone line network (e.g., a public switched telephone network (PSTN)), a private network, an ad hoc network, an intranet, the Internet, an optical fiber-based network, etc., and / or a combination of these or other types of networks.

[0045] According to an embodiment, network 130 may include a virtual network that includes one or more physical network components (such as Ethernet (registered trademark), a WiFi module, telecommunications network hardware, etc.) on which one or more virtual network functions (such as a controller area network (CAN) bus, etc.) are implemented. Additionally or alternatively, network 130 may include at least one parameter network. According to an embodiment, network 130 may also be configured to couple test management system 110 (or one or more components included therein) to other components such as a plurality of test environments 140-1 to 140-N, one or more devices or apparatuses of a user (such as a tester, etc.).

[0046] The plurality of test environments 140-1 to 140-N may include at least one software-based test environment and / or at least one hardware-based test environment. According to an embodiment, at least one software-based test environment may include at least one software-in-the-loop (SIL) test environment, at least one virtual ECU (V-ECU) test environment, at least one model-in-the-loop (MIL) test environment, at least one processor-in-the-loop (PIL) test environment, etc. According to an embodiment, at least one hardware-based test environment may include at least one hardware-in-the-loop (HIL) test environment. Each of the test environments 140-1 to 140-N is communicatively coupled to test management system 110 and may provide one or more signals, information, data, etc. to test management system 110 or receive the same from test management system 110.

[0047] Generally, at least one hardware-based test environment can be configured to manage one or more tasks related to one or more physical hardware components, such as, for example, executing or simulating tests involving one or more physical hardware components, scheduling test executions, etc. (but not limited to these). For example, at least one hardware-based test environment can be communicatively coupled to one or more developed (or partially developed) hardware / physical components and simulate a real environment or an actual use case, thereby testing or evaluating one or more hardware / physical components.

[0048] As an example, before being incorporated into a vehicle, a physical engine ECU may be developed and tested. In such an example, instead of testing the engine ECU with an actual engine, at least one hardware-based test environment can execute a simulation of the engine that interacts with the engine ECU. As another example, in-vehicle functions can include the functions of software-based ECUs (such as, for example, virtual ECUs, emulated ECUs, etc.) and hardware-based ECUs. Thus, at least one hardware-based test environment can be configured to interoperate with at least one software-based test environment via a test management system 110, thereby providing cross-domain testing (for example, at least one hardware-based test environment can execute tasks associated with a hardware-based ECU, at least one software-based test environment can execute tasks associated with a software-based ECU, and the associated results can be aggregated and provided to the test management system 110 and then further processed).

[0049] On the one hand, at least one software-based test environment can be configured to manage one or more tasks related to one or more software components, such as execution or simulation of tests, scheduling of test executions, etc. (but not limited thereto). While a hardware-based test environment is communicatively coupled to one or more physical / hardware components and performs tests / simulations thereon, a software-based test environment can simply obtain or receive one or more software components and perform software-based tests / simulations thereon. Simply put, a software-based test environment, like a hardware-based test environment, can be generated, deployed, and executed in any suitable computing device or environment without the need for a connection to the physical / hardware components to be tested.

[0050] According to an embodiment, one or more software components that can be tested in at least one software-based test environment may include at least one virtual ECU and at least one emulated (or simulated) ECU. In this regard, at least one virtual ECU can include application software, programming code, functional algorithms, etc. that define a final ECU (e.g., in terms of design, development, manufacturing, etc.), while an emulated ECU can be different from at least one emulated ECU in that it can include application software, programming code, functional algorithms, etc. that define a general-purpose or non-final ECU. Further, at least one virtual ECU and / or at least one emulated ECU can include at least one of a central ECU (CECU), an instrument cluster (IC) ECU, an in-vehicle infotainment (IVI) ECU, an advanced driver assistance systems (ADAS) ECU, a chassis ECU, a powertrain ECU, a body ECU, and any other suitable type of ECU related to the vehicle.

[0051] Additionally or alternatively, one or more software components may include one or more software models, such as at least one simulated environmental condition model (e.g., road condition model, traffic condition model, weather condition model, etc.), at least one vehicle-related model (e.g., DCM model, HVAC model, etc.).

[0052] According to an embodiment, a software-based test environment can execute a plurality of tasks (e.g., a plurality of tests, a plurality of simulations, etc.) simultaneously. For example, the software-based test environment can execute a plurality of tests / simulations in parallel for one software component (e.g., one virtual ECU, etc.), execute one test / simulation in parallel for a plurality of software components, and / or execute the like. Further, the software-based test environment and the hardware-based test environment can execute a plurality of tasks (e.g., a plurality of tests, a plurality of simulations, etc.) simultaneously. For example, the software-based test environment can execute one or more tests / simulations for one or more software components, and at the same time, the hardware-based test environment can execute one or more tests / simulations for one or more hardware components (e.g., physical ECUs, etc.).

[0053] The plurality of test environments 140-1 to 140-N may be configured to be executed on one or more test servers, or may be communicatively coupled to one or more test servers. According to an embodiment, one or more of the plurality of test environments 140-1 to 140-N may be deployed or made executable on one or more of the plurality of nodes 120-1 to 120-N. For example, a software-based test environment may be deployed and executed on one or more of the plurality of nodes 120-1 to 120-N. Alternatively or additionally, one or more of the plurality of test environments 140-1 to 140-N may be communicatively coupled to one or more of the plurality of nodes 120-1 to 120-N (e.g., via a wired connection, a wireless connection, etc.). For example, a hardware-based test environment may be communicatively coupled to a node associated with a physical ECU (e.g., a fully developed hardware ECU, a partially developed hardware ECU, etc.). In that case, one or more of the plurality of test environments may be communicatively coupled to the test management system 110 via the network 130 as described above.

[0054] For this purpose, it can be understood that one or more of the software components and / or one or more of the hardware components described above (i.e., the components to be tested in the plurality of test environments 140-1 to 140-N) are associated with (e.g., deployed on, communicatively coupled to, etc.) one or more of the plurality of nodes 120-1 to 120-N, and the test management system 110 may be configured to utilize the plurality of nodes and the plurality of test environments to facilitate cross-domain test provisioning.

[0055] Next, refer to FIG. 2, which shows a block diagram of exemplary components of a test management system 200 according to one or more embodiments. The test management system 200 may correspond to the test management system 110 described above with reference to FIG. 1, and thus, the features described herein with reference to systems 110 and 200 may be applicable to each other, unless explicitly stated otherwise.

[0056] As shown in FIG. 2, the test management system 200 can include at least one communication interface 210, at least one storage 220, and at least one processor 230, although the test management system 200 can include more or fewer components than those shown, and / or the components included therein can be arranged in any manner different from that shown without departing from the technical scope of the present disclosure.

[0057] The communication interface 210 may include a component such as a transceiver (e.g., a transceiver, a separate receiver and transmitter, etc.) that enables the test management system 200 (or one or more components included therein) to communicate with one or more components external to the test management system 200 via, for example, a wired connection, a wireless connection, or a combination of a wired connection and a wireless connection. For example, the communication interface 210 may couple the test management system 200 (or one or more components included therein) to a plurality of test environments (e.g., test environments 140-1 to 140-N in FIG. 1, etc.), thereby enabling them to communicate and interoperate with each other. As another example, the communication interface 210 may couple the test management system 200 (or one or more components included therein) to a plurality of nodes (e.g., nodes 120-1 to 120-N in FIG. 1, etc.), thereby enabling them to communicate with each other and interoperate. Similarly, the communication interface 210 may enable the components of the test management system 200 to communicate with each other. For example, the communication interface 210 may couple the storage 220 to the processor 230, thereby enabling them to communicate and interoperate with each other.

[0058] According to an embodiment, the communication interface 210 may include a hardware-based interface such as a bus interface, an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, a software interface, etc. According to an embodiment, the communication interface 210 may include at least one controller area network (CAN) bus configured to communicatively couple components of the test management system 200 (e.g., the storage 220, the processor 230, etc.) to a plurality of nodes (e.g., nodes 120-1 to 120-N) and / or a plurality of test environments (e.g., test environments 140-1 to 140-N). Additionally or alternatively, the communication interface 210 may include a software-based interface such as an application programming interface (API), a virtual network interface (e.g., a virtual CAN bus, etc.).

[0059] At least one storage 220 may include one or more storage media suitable for storing data, information, and / or computer-readable / computer-executable instructions. According to an embodiment, the storage 220 may include a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, and / or optical memory) for storing information and / or instructions used by the processor 230. Additionally or alternatively, the storage 220 may include, together with a corresponding drive, a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, and / or a solid-state disk), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cartridge, a magnetic tape, and / or another type of non-transitory computer-readable medium.

[0060] According to an embodiment, the storage 220 can operate as a centralized library and can be configured to store information used by the processor 230 to manage one or more tests (e.g., cross-domain tests, etc.). For example, the storage 220 can be configured to store one or more parameters or configurations predefined or predetermined by one or more users (e.g., testers associated with one or more nodes), such as test cycles, one or more test conditions, one or more configuration files, and / or the like. Further, the storage 220 can store information related to the capabilities and / or availability of each of a plurality of test environments communicatively coupled to the test management system, such as the capability information of the test server (or any other suitable node) on which the test environment is deployed / hosted, the processing capabilities of the simulator of the test environment (e.g., HIL simulator, SIL simulator, etc.), the history of test results, the usage cost, the processing / test speed (or parameters that can determine the processing / test speed), and the like. Further, the storage 220 can store computer-readable instructions that, when executed by one or more processors (e.g., the processor 230), cause the one or more processors to perform one or more operations described herein.

[0061] The at least one processor 230 can include one or more processors that can be programmed or configured to perform functions or operations to facilitate the provisioning of cross-domain tests. For example, the processor 230 can be configured to perform one or more actions or one or more operations described herein by executing computer-readable instructions stored in a storage medium (e.g., the storage 220, etc.).

[0062] According to an embodiment, the processor 230 may be configured to receive one or more signals (e.g., via the communication interface 210, etc.) that define one or more instructions for performing one or more operations. Further, the processor 230 may be implemented in hardware, firmware, or a combination of hardware and software. The processor 230 may include a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), and / or another type of processing or computing component.

[0063] According to an embodiment, the processor 230 may be configured to execute computer-executable instructions stored in at least one memory storage (e.g., storage 220), thereby performing one or more operations for managing one or more tests of one or more components (e.g., software and / or hardware components deployed or associated with one or more of a plurality of nodes).

[0064] Refer to FIG. 3, which shows a flowchart of an exemplary method 300 for facilitating cross-domain test provisioning according to one or more embodiments. The method 300 may be executed by at least one processor (e.g., the processor 230) of a test management system to test one or more software related to an in-vehicle system (e.g., an in-vehicle ECU, etc.) of a vehicle.

[0065] In operation S310, at least one processor of the test management system can be configured to determine whether the execution of a test should be triggered. For example, the at least one processor can determine whether one or more conditions for executing the test are satisfied. The one or more conditions can be predefined by one or more users (such as users associated with one or more of a plurality of nodes) and stored in one or more storage media (such as storage 220). As an example, the one or more conditions can include that one or more test requirements are met / violated, one or more thresholds are achieved, one or more changes in software and / or one or more related components (such as software / hardware components within a node) are detected, and the like.

[0066] According to an embodiment, the at least one processor can determine whether one or more conditions for executing the test are satisfied by detecting a change in software (such as a virtual ECU). For example, the at least one processor can determine whether the software has been changed from a previous version and whether the change is a breaking change. An exemplary use case description related to operation S310 is provided below with reference to FIG. 4.

[0067] Therefore, based on determining that one or more conditions for executing the test are satisfied, the at least one processor can determine that the execution of the test should be triggered, and method 300 can proceed to operation S320.

[0068] In operation S320, at least one processor may be configured to obtain a test bench. Generally, a test bench in software testing may refer to a series of procedures, information or parameters such as setting of test scenarios, which provides simulation or configuration of necessary test inputs. For example, a test bench for testing an ECU may include information or parameters for simulating a load for testing the ECU, simulating one or more ECUs associated with the ECU, simulating one or more test conditions (such as road conditions, weather conditions, etc.), and the like. According to an embodiment, the test bench may include a general test bench that defines a configuration of a test environment for normal operation, and may include a general test bench that defines configurations, conditions and / or services (in addition to the general test bench) for testing specific functions or situations.

[0069] According to an embodiment, in operation S320, at least one processor may obtain one or more test configuration files related to a component to be tested (such as a software component, a hardware component, etc.), and may obtain a test bench based on the one or more test configuration files.

[0070] The one or more test configuration files may include information or parameters that define one or more test configurations, such as mapping of component functionality and related types of test environments (for example, function A of the first ECU should be tested in a software-based test environment, and function B of the first ECU should be tested in a hardware-based test environment, etc.), resources required to test component functionality (such as computing power, memory, etc.), load information related to functionality, and the like.

[0071] According to an embodiment, one or more test configuration files may include information on a plurality of ECUs (e.g., virtual ECUs, emulated ECUs, physical ECUs, etc.) related to a component to be tested (e.g., software). The information may include source code, algorithms, functionality, etc. of the plurality of ECUs, and may include information on a test environment associated with each of the plurality of ECUs. In some implementations, at least some of the plurality of ECUs may be distributed and arranged at geographically different locations from the component to be tested. Alternatively or additionally, some of the plurality of ECUs may be associated with one or more nodes different from the nodes of the component to be tested (e.g., enabled / deployed to be executable, communicatively coupled, etc.). Also, the plurality of ECUs may include at least one of a central ECU (CECU), an instrument cluster (IC) ECU, an in-vehicle infotainment (IVI) ECU, an advanced driver assistance systems (ADAS) ECU, and any other suitable type of ECU.

[0072] Accordingly, at least one processor may obtain a test bench based on one or more test configuration files. According to an embodiment, the test bench may be pre-generated and pre-stored in one or more storage media (e.g., storage 220, an external server, etc.), and at least one processor may obtain the test bench from the one or more storage media (based on the one or more test configuration files) in operation S320. Alternatively, at least one processor may generate the test bench in real time or near real time based on the one or more test configuration files in operation S320. According to an embodiment, at least one processor may obtain or generate a replica of a component related to the component under test (e.g., a cloned version of an ECU related to the component under test) and include the replica in the test bench.

[0073] Upon obtaining the test bench, method 300 proceeds to operation S330, where at least one processor of the test management system may be configured to determine a plurality of test environments associated with the test bench. The plurality of test environments may include at least one software-based test environment (e.g., a software-in-the-loop (SIL) test environment) and at least one hardware-based test environment (e.g., a hardware-in-the-loop (HIL) test environment), similar to those described above with reference to the plurality of test environments 140-1 to 140-N in FIG. 1. Subsequently, upon determining the plurality of test environments, method 300 proceeds to operation S340, where at least one processor may be configured to execute a cross-domain test based on the plurality of test environments. Exemplary operations related to operations S330 and S340, as well as exemplary use cases related thereto, are provided below with reference to FIGS. 5 and 6.

[0074] Next, referring to FIG. 4, FIG. 4 shows a flowchart of an exemplary use case related to operation S310 of FIG. 3 according to one or more embodiments. In this exemplary use case, the test management system 410 is configured to continuously (or periodically) monitor the status of components deployed within or associated with the first node 420-1 and the second node 420-2 according to test cycles associated with each of the first node 420-1 and the second node 420-2, and determine whether to initiate the execution of a test to test the component(s).

[0075] Here, the test management system 410 of FIG. 4 corresponds to the test management system 110 of FIG. 1 and the test management system 200 of FIG. 2, and the first node 420-1 and the second node 420-2 of FIG. 4 can correspond to a part of the plurality of nodes 120-1 to 120-N of FIG. 1. Further, it can be understood that the test management system 410 can include at least one processor (e.g., processor 230) as described above with reference to FIG. 2, and one or more operations in FIG. 4 can be executed by at least one processor. Therefore, related redundant descriptions may be omitted hereinafter for the sake of brevity.

[0076] As shown in FIG. 4, in operations S410-1 and S410-2, at least one processor of the test management system 410 may be configured to communicate with the first node 420-1. According to an embodiment, at least one processor may send (e.g., via an API call, etc.) a query or request to the first node 420-1 regarding information on the current status of one or more components associated with the first node 420-1 (e.g., software components deployed on or made executable (hosted) within the first node 420-1, hardware components communicatively coupled to the first node 420-1, etc.). Thus, the first node 420-1 can provide the requested information to at least one processor of the test management system 410.

[0077] According to an embodiment, at least one processor may continuously (or periodically) execute operations S410-1 and S410-2 according to a first test cycle related to and / or associated with one or more components related to the first node 420-1. The first test cycle may be pre-determined by one or more users associated with the first node 420-1 (e.g., the manager of the first node 420-1, the developer of software components deployed on the first node 420-1, etc.), and the first test cycle may be pre-stored in one or more storage media (e.g., the storage of the test management system 410, a server external to the test management system 410, etc.).

[0078] As an example, the first test cycle may define that the first component associated with the first node 420-1 should be checked at time intervals such as every 5 minutes, every hour, every day, etc. Thus, at least one processor of the test management system 410 can execute operations S410-1 and S410-2 every 5 minutes, every hour, every day, etc. to obtain the latest status information of the first component from the first node 420-1. Further, the at least one processor may execute operation S410-2 5 minutes after the execution of operation S410-1 (or after other appropriate time intervals defined by the first test cycle).

[0079] In operations S430-1 and S430-2, at least one processor of the test management system 410 is assumed to be configured to communicate with the second node 420-2 to request information on the current status of one or more components associated with the second node 420-2 according to a second test cycle predetermined by one or more users associated with the second node 420-2 in a manner similar to that described above with reference to operations S410-1 and S410-2.

[0080] Furthermore, it can be understood that the second test cycle, in which (at least one processor executes operations S430-1 and S430-2), may be the same as or different from the first test cycle, in which (at least one processor executes operations S410-1 and S410-2). Further, it can be understood that at least one processor may be configured to execute the communication with the first node 420-1 (operations S410-1 and S410-2) and the communication with the second node 420-2 (operations S430-1 and S430-2) in any suitable sequence. For example, without departing from the technical scope of the present disclosure, at least one processor may execute the communication with the first node 420-1 and the second node 420-2 simultaneously (for example, operations S410-1 and S430-1 are executed in parallel, or operations S410-2 and S430-2 are executed in parallel), or execute the communication with the second node 420-2 before executing the communication with the first node 420-1 (for example, operation S430-1 is executed before operation S410-1, etc.).

[0081] Referring further to FIG. 4, when communicating with the first node 420-1 and the second node 420-2, at least one processor of the test management system 410 may be configured to determine whether a test execution should be triggered on the components related thereto based on the information or data obtained from the first node 420-1 and the second node 420-2.

[0082] For example, in operation S420-1, at least one processor may determine whether one or more conditions for executing a test are satisfied. As an example, at least one processor may determine whether a change (from a previous version) has occurred in a component within the first node 420-1, and based on the determination that a change in the component has occurred, may determine whether a destructive change has occurred.

[0083] In this regard, a "breaking change" may refer to a modification to software (or one or more components related thereto) that causes an existing function of the software to no longer function as intended or to become unstable. Further, a breaking change may be a change that affects other parts related to the software (such as other software components, other hardware components, etc.) and changes the behavior of the software in a way that leads to software failures. For this purpose, the conditions for determining a breaking change may be predefined by a user associated with the component under test, or may be predefined by other users associated with the component under test (such as users associated with other ECUs associated with the component under test).

[0084] Therefore, in operation S420-1, when at least one processor detects a change in the component under test, it may determine whether the change is a breaking change (based on one or more predefined conditions), and based on determining that the change is a breaking change, it may determine that it is necessary to execute a test for the component under test.

[0085] It can be understood that in addition to or instead of determining a breaking change, at least one processor may determine whether any other conditions are satisfied to determine whether a test should be triggered. For example, at least one processor may determine whether the performance of an associated node (such as the first node 420-1) has deteriorated, whether a preset test execution schedule has been reached, etc., and based on this, determine whether the execution of the test should be triggered. It can also be understood that at least one processor of the test management system may be configured to execute operations S440-1, S420-2, and S440-2 in the same manner as described above with reference to operation S420-1.

[0086] For this purpose, at least one processor of the test management system 410 can continuously (or periodically) determine whether test execution should be triggered to test one or more components associated with one or more nodes in an automated manner.

[0087] Next, refer to FIG. 5, which shows a flowchart of an exemplary method 500 for selecting a plurality of test environments and performing cross-domain testing therein according to one or more embodiments. One or more operations of method 500 can be part of operations S330 and S340 of FIG. 3 and can be performed by at least one processor (e.g., processor 230) of the test management system.

[0088] In operation S510, at least one processor of the test management system can be configured to create a test job. For example, the at least one processor can create a test job including a plurality of tasks based on a test bench (e.g., obtained in operation S320), and each of the plurality of tasks can include information (e.g., requirements, etc.) related to the test to be performed.

[0089] As an example, assuming that cross-domain testing needs to be performed to test modified software (e.g., a modified virtual ECU) under a software-based test environment (e.g., a SIL test environment) and a hardware-based test environment (e.g., a HIL test environment), at least one processor can create a cross-domain test job that includes multiple tasks each containing information such as runtime information, information on required hardware resources (e.g., the types of required physical ECUs, etc.), information on required software resources (e.g., CPU power, memory, etc.), and information on replicas of components. This cross-domain test job can be utilized by at least one processor to determine an optimal software-based test environment and an optimal hardware-based test environment for executing the cross-domain test.

[0090] Upon creating the test job, method 500 proceeds to operation S520, where at least one processor of the test management system can be configured to select multiple test environments for executing the test based on the created test job.

[0091] As an example, assuming that a cross-domain test is created (as described above with reference to operation S510), at least one processor can determine test environment requirements for executing each task of the test based on one or more requirements for executing the tasks of the cross-domain test. Subsequently, at least one processor can select an appropriate or optimal test environment for executing the tasks of the cross-domain test job from among multiple test environments (e.g., multiple test environments 140-1 to 140-N) communicatively coupled to the test management system.

[0092] For example, at least one processor can select one or more software-based test environments that are communicably coupled to the test management system and meet the necessary requirements (such as software resource requirements, etc.) for executing software-based tests from among a plurality of software-based test environments, and at least one processor can select one or more hardware-based test environments that are communicably coupled to the test management system and meet the necessary requirements (such as hardware resource requirements, etc.) from among a plurality of hardware-based test environments.

[0093] Upon selecting a test environment, method 500 proceeds to operation S530, where at least one processor of the test management system can be configured to assign one or more tasks to the selected test environment. Accordingly, the test environment can execute the assigned tasks and then provide the execution results to at least one processor. Thereafter, at least one processor can generate (e.g., compile, aggregate, etc.) the test results of the cross-domain test based on the test results provided by a plurality of test environments and associated with the assigned tasks.

[0094] Next, refer to FIG. 6, which shows a flowchart of an exemplary use case related to one or more operations of the method of FIG. 5 according to one or more embodiments. In this exemplary use case, a test management system 610 is communicably coupled to a plurality of test servers 620-1 to 620-N and is configured to select one or more test environments associated with the plurality of test servers to facilitate the provisioning of cross-domain tests.

[0095] In this regard, the test management system 610 of FIG. 6 corresponds to the test management system 110 of FIG. 1, the test management system 200 of FIG. 2, or the test management system 410 of FIG. 4, and it can be understood that each of the plurality of test servers 620-1 to 620-N can be associated with one or more of the plurality of test environments 140-1 to 140-N of FIG. 1.

[0096] Furthermore, the test management system 610 may include at least one processor (e.g., processor 230), and it can be understood that one or more operations in FIG. 6 can be executed by at least one processor. Furthermore, it can also be understood that one or more operations in FIG. 6 can be executed after one or more operations in FIGS. 3 and 4.

[0097] Referring to FIG. 6, in operation S610, at least one processor of the test management system 610 can be configured to create a test job. Since the details of this operation are the same as those described in operation S510 of FIG. 5, the following may omit the overlapping description for the sake of brevity.

[0098] After creating the test job, at least one processor of the test management system 610 can be configured to communicate with the plurality of test servers 620-1 to 620-N (each of operations S620-1 to S620-N) and obtain information about the associated test environment.

[0099] In the exemplary use case of FIG. 6, at least one processor may first obtain, from a first test server 620-1, availability information, capability information, and / or any other suitable type of information related to (e.g., executable / expandable, communicatively coupled to, etc.) one or more test environments associated with the first test server 620-1 (in operation S620-1). For example, at least one processor may generate one or more API calls to request the information requested from the first test server 620-1 and provide the generated API calls to the first test server 620-1 (via a communication interface). Thus, at least one processor may obtain the necessary information from the remaining test servers 620-2 to 620-N in a similar manner.

[0100] It can be understood that at least one processor of the test management system 610 may also obtain the necessary information from the plurality of test servers 620-1 to 620-N in any suitable order. For example, at least one processor may execute operations S620-1 and S620-2 simultaneously to obtain the necessary information of a plurality of test environments associated with the first test server 620-1 and the second test server 620-2 at the same time, or execute operation S620-2 before operation S620-1 to obtain the first capability information from the second test server 620-2.

[0101] Upon receiving the necessary information from the plurality of test servers, in operation S630, at least one processor of the test management system 610 may be configured to select a plurality of test environments suitable or optimal for executing one or more tasks of the created test job from among the plurality of test environments associated with the plurality of test servers.

[0102] For example, at least one processor can select multiple test environments according to the required information (such as availability, capacity, cost, etc.) and the type of test environment required to execute one or more tasks of the test job (such as a hardware-based test environment like a HIL test environment, a software-based test environment like a SIL test environment, a V-ECU test environment, etc.).

[0103] As an example, assume that the test job is a cross-domain test job including a first task for testing virtual ECU A in a SIL test environment where the CPU power requirement is X, and a second task for testing virtual ECU A together with physical ECU B, etc. Then, at least one processor can select one or more test environments within the test server 620-1 to 620-N that have sufficient CPU power and can execute the SIL test as required in the first task. Similarly, at least one processor can select one or more test environments within the test server 620-1 to 620-N that are associated with physical ECU B and can execute the HIL test on physical ECU B as required in the second task. It should be understood that at least one processor can perform any other appropriate operations for selecting an appropriate test environment for executing one or more tasks of the created test job without departing from the technical scope of the present disclosure.

[0104] When multiple test environments are selected, at least one processor may be configured to assign or allocate the tasks of the test job to each test environment. In the exemplary use case of FIG. 6, a first test environment (e.g., SIL test environment) associated with the first test server 620-1 is selected to execute the first task of the test job, and it can be assumed that a second test environment (e.g., HIL test environment) associated with the second test server 620-2 is selected by at least one processor to execute the second task of the test job. Therefore, in operations S640-1 and S640-2, at least one processor can be configured to assign or allocate the first task and the second task to the first test environment and the second test environment, respectively.

[0105] According to an embodiment, at least one processor may be configured to assign or allocate the tasks of the test job by generating signals, messages, instructions, etc. that include task information (e.g., schedule for executing the task, target completion time of the task, etc.). According to embodiments where the task includes testing other software components (e.g., other virtual ECUs, emulated ECUs, software models such as traffic models, etc.), signals, messages, instructions, etc. generated by at least one processor may further include replicas of the other software components (e.g., cloned versions of source code, algorithms, etc.). Thereafter, at least one processor can provide the generated signal / message / instruction to a test server that is capable of or has deployed the selected test environment.

[0106] In this regard, at least one processor of the test management system 610 can first assign tasks to a first test environment that is enabled or deployed to execute on the first test server 620-1 (operation S640-1), and then can assign tasks to a second test environment that is enabled or deployed to execute on the second test server 620-2 (operation S640-2). Alternatively, at least one processor can allocate multiple tasks for execution simultaneously in the first and second test environments. It can be understood that at least one processor can also allocate or assign tasks in any other suitable sequence.

[0107] When one or more tasks are allocated or assigned to a test environment, the test environment can execute one or more tests according to the assigned tasks. Thereafter, in operations S650-1 and S650-2, at least one processor of the test management system 610 can be configured to monitor the performance of the test environment, receive from the test server one or more test results of the assigned tasks, etc., and update the relevant information in the record file accordingly. For example, at least one processor can aggregate the test results provided by multiple test environments and generate or compile a complete comprehensive test result representing the test results of the cross-domain test.

[0108] As another example, whenever one or more test results indicate a failure in the test execution (such as an exception where a task is not completed within a specified period or due to a timeout), at least one processor can update the record file so as not to reassign similar tasks to the same test environment in future tests. Similarly, whenever one or more test results indicate specific parameters (such as time or speed) that are different from those estimated from or reflected in the record, at least one processor can update the record file accordingly to more accurately reflect the current capabilities of the corresponding test environment.

[0109] Also, upon receiving a report of a task failure or determining a task failure, at least one processor can be configured to reassign the failed task to a different test environment. For example, each time a task failure is reported due to a timeout, at least one processor can reassign the task to a different test environment that exhibits a higher test speed than the test environment to which the task was previously assigned, based on the associated capability information.

[0110] In view of the above, it can be understood that at least one processor can be configured to assign one or more tasks based on the test results of previous tests. For example, to test a software component (such as a virtual ECU), a first series of tests or simulations is performed, and then the source code of the software component is updated. In this regard, at least one processor can optimize the task assignment (such as shortening the test execution time, etc.) for testing the software component using the updated source code in a second series of tests or simulations, based on the results of the first series of tests or simulations.

[0111] It should be understood that the specific order or hierarchy of blocks in the processes / flowcharts disclosed in this specification is an example of an exemplary approach. Based on design preferences, it should be understood that the specific order or hierarchy of blocks in a process / flowchart can be reconfigured. Further, some blocks can be combined or omitted. The appended method claims present the elements of the various blocks in an exemplary order and are not limited to the specific order or hierarchy presented.

[0112] Some embodiments can relate to systems, methods, and / or computer-readable media at any possible technical detail level of integration. Further, one or more of the above-described components can be stored on a computer-readable medium and implemented as instructions executable by at least one processor (and / or can include at least one processor). The computer-readable medium can include a computer-readable non-transitory storage medium having computer-readable program instructions for causing a processor to perform operations.

[0113] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A list, which does not purport to exhaust all of the more specific examples of computer-readable storage media, includes portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disks (DVDs), memory sticks, floppy disks, punch cards, mechanically encoded devices such as a raised structure in a groove having recorded instructions, and any suitable combination of the foregoing. A computer-readable medium as used herein should not be construed to be a transitory signal per se, such as a radio wave, or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission media (e.g., an optical pulse passing through an optical fiber cable), or an electrical signal transmitted through a wire.

[0114] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0115] The computer-readable program code / instructions for performing the operations may be in source code or object code in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuits, or object-oriented programming languages such as Smalltalk, C++, and procedural programming languages such as the "C" programming language or similar program languages. The computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer, and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), or the connection can be made to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute the computer-readable program instructions by personalizing the electronic circuit using the state information of the computer-readable program instructions to perform the aspects or operations.

[0116] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to cause the instructions executed via the processor of the computer or other programmable data processing apparatus to create means for realizing the functions / operations specified in the blocks of a flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium having stored therein instructions for realizing the functions / operations specified in the blocks of a flowchart and / or block diagram, such that the computer-readable storage medium comprises a manufactured article that functions in a particular manner in a computer, a programmable data processing apparatus, and / or other devices.

[0117] The computer-readable program instructions can also be loaded onto a computer, other programmable apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to generate a process that is realized by the computer to realize the functions / operations specified in the blocks of a flowchart and / or block diagram.

[0118] The flowcharts and block diagrams in the drawings illustrate the structure, functions, and operations of possible realizations of systems, methods, and computer-readable media according to various embodiments. In this regard, each block in the flowchart or block diagram can represent a module, segment, or part of an instruction, and they include one or more executable instructions for implementing a specified logical function. The method, computer system, and computer-readable media can include additional blocks, fewer blocks, different blocks, or blocks arranged differently from those shown in FIG. 1. In some alternative realizations, the functions shown in the blocks can occur in a different order than that shown in the figure. For example, two blocks shown consecutively can actually be executed simultaneously or substantially simultaneously, or depending on the functions involved, the blocks can be executed in the reverse order. It will also be noted that each block in the illustration of the block diagram and / or flowchart, and combinations of blocks in the illustration of the block diagram and / or flowchart, can be realized by a system based on special-purpose hardware that performs the specified function or operation and a combination of special-purpose hardware and computer instructions.

[0119] The systems and / or methods described herein can be realized in different forms of hardware, firmware, or a combination of hardware and software. The actual specialized control hardware or software code used to realize these systems and / or methods does not limit the realizations. Therefore, the operations and behaviors of the systems and / or methods have been described herein without reference to specific software code, and it is understood that software and hardware can be designed to realize the systems and / or methods based on the descriptions herein.

Claims

1. A method implemented by at least one processor for facilitating the provisioning of cross - domain testing for testing software of an embedded system, comprising: detecting a change in the software; obtaining a test configuration file associated with the software; obtaining a test bench based on the test configuration file; determining a plurality of test environments associated with the test bench; executing the cross - domain test based on the plurality of test environments; wherein the software of the embedded system includes an in - vehicle electronic control unit (ECU); the plurality of test environments include at least one software - in - the - loop (SIL) test environment and at least one hardware - in - the - loop (HIL) test environment; method.

2. The change in the software includes a breaking change. The method according to claim 1.

3. The test configuration file includes information on a plurality of ECUs associated with the software and information on test environments associated with each of the plurality of ECUs. The method according to claim 1 or claim 2.

4. At least a part of the plurality of ECUs is associated with one or more nodes different from the software. The method according to claim 3.

5. At least a part of the plurality of ECUs is distributed and arranged at geographically different locations from the software. The method according to claim 3.

6. The plurality of ECUs include at least one virtual ECU, at least one emulated ECU, at least one physical ECU, or a combination thereof. The method according to claim 3.

7. The plurality of ECUs include at least one of a central ECU (CECU), an instrument cluster (IC) ECU, an in - vehicle infotainment (IVI) ECU, and an advanced driver assistance system (ADAS) ECU. The method according to claim 3.

8. Detecting a change in the software comprises obtaining information on the current status of the software from a node associated with the software; determining whether the software has been changed from a previous version based on the obtained status information; Based on determining that the software has been changed, determining whether the change is a destructive change; The method according to claim 1 or claim 2, including this.

9. Determining the plurality of test environments includes: Based on the obtained test bench, creating a test job including a plurality of tasks; Selecting the plurality of test environments based on one or more requirements for executing the plurality of tasks; The method according to claim 1 or claim 2, including this.

10. Executing the cross-domain test includes: Allocating one or more tasks of the test job to the plurality of test environments; Receiving test results related to the allocated one or more tasks from the plurality of test environments; Generating a test result of the cross-domain test based on the test results related to the allocated one or more tasks; The method according to claim 9, including this.

11. A system for facilitating the provisioning of cross-domain testing for testing the software of an embedded system, comprising: At least one memory storage storing computer-executable instructions; Communicatively coupled to the at least one memory storage, and Detecting changes in the software, Obtaining a test configuration file associated with the software, Obtaining a test bench based on the test configuration file, Determining a plurality of test environments associated with the test bench, Executing the cross-domain test based on the plurality of test environments, At least one processor configured to execute the computer-executable instructions for this; Including, The software of the embedded system includes an in-vehicle electronic control unit (ECU), The plurality of test environments include at least one software-in-the-loop (SIL) test environment and at least one hardware-in-the-loop (HIL) test environment; System.

12. The change in the software includes a destructive change, The system according to claim 11.

13. The test configuration file includes information on a plurality of ECUs associated with the software and information on test environments associated with each of the plurality of ECUs. The system according to claim 11 or claim 12.

14. At least a part of the plurality of ECUs is associated with one or more nodes different from the software. The system according to claim 13.

15. At least a part of the plurality of ECUs is distributed and arranged at geographically different positions from the software. The system according to claim 13.

16. The plurality of ECUs includes at least one virtual ECU, at least one emulated ECU, at least one physical ECU, or a combination thereof. The system according to claim 13.

17. The plurality of ECUs includes at least one of a central ECU (CECU), an instrument cluster (IC) ECU, an in-vehicle infotainment (IVI) ECU, and an advanced driver assistance system (ADAS) ECU. The system according to claim 13.

18. The at least one processor acquires information on the current status of the software from a node related to the software, determines whether the software has been changed from a previous version based on the acquired status information, determines whether the change is a destructive change based on determining that the software has been changed, and is configured to execute the computer-executable instructions for detecting a change in the software. The system according to claim 11 or claim 12.

19. The at least one processor creates a test job including a plurality of tasks based on the acquired test bench, selects the plurality of test environments based on one or more requirements for executing the plurality of tasks, and is configured to execute the computer-executable instructions for determining the plurality of test environments. The system according to claim 11 or claim 12.

20. The at least one processor allocates one or more tasks of the test job to the plurality of test environments, receives test results related to the allocated one or more tasks from the plurality of test environments, and generates a test result of the cross-domain test based on the test results related to the allocated one or more tasks. configured to execute the computer-executable instructions for performing the cross-domain test thereby The system according to claim 19.

Images