Systems and methods for cross-domain provisioning

Abstract

To provide a method, system, and device for facilitating provisioning of a cross-domain test for testing software of an embedded system.SOLUTION: A method may be implemented by at least one processor and may include: detecting a change in the software; obtaining a test configuration file associated with the software; obtaining, based on the test configuration file, a test bench; determining a plurality of test environments associated with the test bench; and performing the cross-domain test based on the plurality of test environments. The software of an embedded system may include an in-vehicle electronic control unit (ECU). The plurality of tests environments may include at least one software-in-the-loop (SIL) test environment, and at least one hardware-in-the-loop (HIL) test environment.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] Systems and methods consistent with example embodiments of the present disclosure relate to test provisioning, and more particularly, to systems and methods for provisioning different simulation and / or test environments for cross-domain testing. [Background technology]

[0002] Traditionally, various test environments are deployed to test system functionality and software performance. For example, a vehicle's embedded systems, such as an Electronic Control Unit (ECU), may be tested by utilizing, among others, at least one Software-in-the-Loop (SIL) test environment and at least one Hardware-in-the-Loop (HIL) test environment.

[0003] Developing advanced or complex features within a system requires testing the functionality and performance of multiple software and hardware components across multiple systems or domains. In vehicle systems, cross-domain testing may be required during the development of advanced features in vehicle systems, such as lane change assist, mobile smart key, etc., and cross-domain testing may involve testing the interactions between different ECUs and associated software and hardware components.

[0004] Simply put, cross-domain testing may refer to testing that involves testing the functionality of a component across different domains or environments, such as different operating environments, different system configurations, different hardware and / or software configurations, and / or the like. The goal of cross-domain testing is to ensure that a system component functions correctly across all of these domains under different circumstances.

[0005] Thus, a cross-domain test may have the following characteristics: (1) Cross-domain testing often presents integration challenges because small changes to an ECU can significantly affect the functionality and stability of other related ECUs and related components, requiring careful integration management between components of the system. (2) Cross-domain testing is often constrained by the test environment because cross-domain testing may require specific hardware, software, and / or system configurations, and setting up and maintaining a test environment that precisely meets the test requirements may be difficult or time-consuming. (3) Cross-domain testing often involves multiple users (e.g., teams, stakeholders, vehicle manufacturers, suppliers, vendors, etc.) who may be located in different locations and have different priorities, goals, backgrounds, communication styles, etc., and therefore effective communication and collaboration is required to perform accurate cross-domain testing, making cross-domain testing often complex in terms of communication and collaboration between users.

[0006] In view of the above, in the prior art, whenever cross-domain testing is required, multiple ECUs and related software / hardware components must be aggregated and located in a centralized test facility (e.g., a local simulation / test facility), and a user / tester must physically visit the facility to perform the cross-domain testing. Nevertheless, the prior art approach for performing cross-domain testing has at least the following drawbacks:

[0007] First, it is burdensome and time-consuming for a user to physically visit a test facility. For example, a user may be located in a different geographic location (e.g., a different region, a different country, etc.) from the test facility, and therefore, physically visiting the test facility may require proper planning and be expensive (e.g., a user may need to take a long flight to the test facility, may need to properly cooperate with other relevant users to reserve the test facility to schedule the test, may need to apply for a visit visa and / or permit, and obtaining approval may take time). In addition, because the available equipment, hardware, etc. at the test facility is limited, a user typically needs to wait a long time (e.g., be placed on a waiting list) before being able to utilize the test facility to perform cross-domain testing.

[0008] Furthermore, in the prior art, it is difficult to quickly and accurately identify or discover breaking changes during cross-domain testing. Specifically, because multiple components in a local test facility may be changed after being used by different users (e.g., in addition to the component under test, related components may also be changed), if the component configuration is not restored to a required state, cross-domain testing performed based on multiple changed components may be inaccurate, and users may not be able to quickly and accurately identify breaking changes through cross-domain testing (e.g., a problem caused by a change to the component under test may be discovered in the cross-domain test, but the problem may be mistaken by the user as a problem caused by a change to another component, or the problem may be fixed by the change to another component and not be discovered).

[0009] Furthermore, in conventional technologies, test facility configurations are inflexible, making it difficult to reconfigure cross-domain tests on-the-fly or on-demand. For example, if a user / tester requests to add a new ECU or change the ECU involved in a test facility, but the requested ECU is not immediately available, the user / tester may need to rebook the test facility, wait until the next test, and then revisit the test facility for further testing. Therefore, in conventional technologies, even if a breaking change is discovered during cross-domain testing, it is difficult to immediately correct the breaking change upon discovery and rerun the test immediately thereafter.

[0010] Considering the above, in the prior art, performing cross-domain testing is time-consuming, with most of the time being spent on travel arrangements, travel, waiting in line for testing, etc. As a result, the prior art approach to performing cross-domain testing is inefficient, cross-domain testing in the prior art cannot be performed on-demand, performing cross-domain testing is a burden for users, and breaking changes to components under test are difficult to discover and fix immediately. Ultimately, these can lengthen the lead time from system-on-chip (SoC) specifications to the start of production (SOP) of a vehicle. Summary of the Invention

[0011] According to embodiments, methods, systems, and devices are provided for automatically facilitating cross-domain testing for testing one or more software of a system. For example, the methods, systems, and devices can automatically determine whether test execution should be triggered, automatically determine an appropriate test environment for executing the cross-domain tests, and automatically execute the cross-domain tests.

[0012] According to an embodiment, a method for facilitating provisioning of cross-domain tests for testing software of an embedded system is provided. The method may be implemented in at least one processor and may include detecting changes to the software, obtaining a test configuration file associated with the software, obtaining a test bench based on the test configuration file, determining a plurality of test environments associated with the test bench, and executing the cross-domain tests based on the plurality of test environments. The software of the embedded system may include an on-board electronic control unit (ECU), and the multiple test environments may include at least one software-in-the-loop (SIL) test environment and at least one hardware-in-the-loop (HIL) test environment.

[0013] According to an embodiment, the test configuration file may include information on a plurality of ECUs associated with the software and information on a test environment associated with each of the plurality of ECUs. At least some of the plurality of ECUs may be associated with one or more nodes different from the software. At least some of the plurality of ECUs may be distributed in geographically different locations from the software. Furthermore, the plurality of ECUs may include at least one virtual ECU, at least one emulated ECU, at least one physical ECU, or a combination thereof. Furthermore, the plurality of ECUs may include at least one of a central ECU (CECU), an instrument cluster (IC) ECU, an in-vehicle infotainment (IVI) ECU, and an advanced driver assistance system (ADAS) ECU.

[0014] According to an embodiment, the change to the software may include a breaking change, and detecting the change to the software may include obtaining information about a current status of the software from a node associated with the software, determining whether the software has changed from a previous version based on the obtained status information, and determining whether the change is a breaking change based on determining that the software has changed.

[0015] According to an embodiment, determining the plurality of test environments may include creating a test job including a plurality of tasks based on the acquired test bench, and selecting the plurality of test environments based on one or more requirements for executing the plurality of tasks. Also, performing the cross-domain test may include assigning one or more tasks of the test job to the plurality of test environments, receiving test results associated with the assigned one or more tasks from the plurality of test environments, and generating test results for the cross-domain test based on the test results associated with the assigned one or more tasks.

[0016] According to an embodiment, a system for facilitating provisioning of cross-domain tests for testing software of an embedded system is provided. The system may include at least one memory storage that stores computer-executable instructions; and at least one processor communicatively coupled to the at least one memory storage and configured to execute the computer-executable instructions to detect changes in the software, obtain a test configuration file associated with the software, obtain a test bench based on the test configuration file, determine a plurality of test environments associated with the test bench, and perform cross-domain tests based on the plurality of test environments, where the software of the embedded system may include an on-board electronic control unit (ECU), and the plurality of test environments may include at least one software-in-the-loop (SIL) test environment and at least one hardware-in-the-loop (HIL) test environment.

[0017] According to an embodiment, the test configuration file may include information on a plurality of ECUs associated with the software and information on a test environment associated with each of the plurality of ECUs. At least some of the plurality of ECUs may be associated with one or more nodes different from the software. At least some of the plurality of ECUs may be distributed in geographically different locations from the software. Furthermore, the plurality of ECUs may include at least one virtual ECU, at least one emulated ECU, at least one physical ECU, or a combination thereof. Furthermore, the plurality of ECUs may include at least one of a central ECU (CECU), an instrument cluster (IC) ECU, an in-vehicle infotainment (IVI) ECU, and an advanced driver assistance system (ADAS) ECU.

[0018] According to an embodiment, the change to the software may include a breaking change, and the at least one processor may be configured to execute computer-executable instructions to detect the change to the software by obtaining information about a current status of the software from a node associated with the software, determining whether the software has changed from a previous version based on the obtained status information, and determining whether the change is a breaking change based on determining that the software has changed.

[0019] According to an embodiment, at least one processor may be configured to execute computer-executable instructions for determining a plurality of test environments by creating a test job including a plurality of tasks based on the obtained test bench, and selecting a plurality of test environments based on one or more requirements for executing the plurality of tasks. The at least one processor may also be configured to execute computer-executable instructions for performing a cross-domain test by assigning one or more tasks of the test job to the plurality of test environments, receiving test results associated with the assigned one or more tasks from the plurality of test environments, and generating test results for the cross-domain test based on the test results associated with the assigned one or more tasks.

[0020] Additional aspects will be set forth in part in the description that follows, and in part will be obvious from the description, or may be realized by practice of the presented embodiments of the present disclosure. [Brief explanation of the drawings]

[0021] The features, advantages, and benefits of exemplary embodiments of the present disclosure will be described below with reference to the accompanying drawings, in which like numerals refer to like elements and in which:

[0022] [Figure 1] FIG. 1 is a block diagram of an exemplary system configuration for facilitating provisioning of cross-domain tests, according to one or more embodiments.

[0023] [Figure 2] FIG. 2 is a block diagram of example components of a test management system according to one or more embodiments.

[0024] [Figure 3] FIG. 3 is a flow diagram of an example method for facilitating provisioning of cross-domain tests, according to one or more embodiments.

[0025] [Figure 4] FIG. 4 is a flow diagram of an example use case associated with operation S310 of FIG. 3, according to one or more embodiments.

[0026] [Figure 5] FIG. 5 is a flow diagram of an example method for selecting multiple test environments and running cross-domain tests thereon, according to one or more embodiments.

[0027] [Figure 6] FIG. 6 is a flow diagram of an example use case associated with one or more operations of the method of FIG. 5, according to one or more embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0028] The following detailed description of example embodiments refers to the accompanying drawings. The foregoing disclosure provides illustration and description, but is not intended to be exhaustive or to limit implementations to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired by practice of the implementations. Furthermore, one or more features or components of one or more embodiments may be incorporated into or combined with other embodiments (or one or more features of other embodiments). Additionally, in the descriptions of operations provided below, it is understood that one or more operations may be omitted, one or more operations may be added, one or more operations may be performed concurrently (at least in part), and the order of one or more operations may be switched.

[0029] Although particular combinations of features are recited in the claims and / or disclosed herein, such combinations are not intended to limit the disclosure of possible implementations. Indeed, many of the features may be combined in ways not specifically recited in the claims and / or disclosed herein. Although each dependent claim listed below may depend directly on only one claim, the disclosure of possible implementations includes each dependent claim in combination with all other claims in the claim set.

[0030] No element, operation, or instruction used herein should be construed as critical or essential unless expressly stated as such. Also, as used herein, the articles "a" and "an" are intended to include one or more items and can be used interchangeably with "one or more." When only one item is intended, the term "a" or similar language is used. Also, as used herein, terms such as "have," "having," "include," "including," and the like are intended to be open-ended and unrestricted. Furthermore, the phrase "based on" is intended to mean "based at least in part on," unless expressly stated otherwise. Furthermore, phrases such as "at least one of [A] and [B]" or "at least one of [A] or [B]" should be understood to include only A, only B, or both A and B.

[0031] References throughout this specification to "one embodiment," "an embodiment," "a non-limiting preferred embodiment," or similar terms mean that a particular feature, structure, or characteristic described in connection with the illustrated embodiment is included in at least one embodiment of the solution. Thus, the phrases "in one embodiment," "in an embodiment," "a non-limiting preferred embodiment," and similar terms throughout this specification may, but do not necessarily, all refer to the same embodiment.

[0032] Furthermore, the described features, advantages, and characteristics of the present disclosure may be combined in any suitable manner in one or more embodiments. Those skilled in the art will recognize, in light of the description herein, that the present disclosure may be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments of the present disclosure.

[0033] Exemplary embodiments consistent with the present disclosure provide methods, systems, and apparatus for facilitating the provisioning of cross-domain tests for testing one or more software components of an embedded system without geographic or location restrictions.

[0034] Specifically, the method, system, apparatus, etc. of the exemplary embodiments can automatically detect one or more changes in one or more software of an embedded system and automatically configure and execute cross-domain tests for testing the one or more software accordingly. According to the embodiments, the method, system, apparatus, etc. of the exemplary embodiments can automatically determine multiple test environments and execute cross-domain tests based on the multiple test environments. In some implementations, upon detecting one or more changes, one or more configuration files associated with the one or more software may be obtained, and a test bench may be obtained based on the one or more configuration files. Thus, the multiple test environments can be determined based on the test bench, and the method, system, apparatus, etc. of the exemplary embodiments can automatically assign one or more tasks to the multiple test environments for executing cross-domain tests, regardless of where the multiple test environments and the components involved in the tests are located.

[0035] To this end, methods, systems, and apparatus consistent with example embodiments of the present disclosure automatically facilitate provisioning of an appropriate test environment for cross-domain testing when needed, without geographic limitations. A user can remotely configure one or more conditions for triggering a cross-domain test, and execution of the cross-domain test can be automatically triggered based thereon, without requiring the user to physically visit a test facility. Furthermore, breaking changes can be quickly and easily discovered and corrected, and reconfigured cross-domain tests can be immediately resumed or re-run when needed.

[0036] Ultimately, exemplary embodiments of the present disclosure enable more efficient software development, significantly reducing the burden on users, significantly reducing development time, and significantly reducing the cost and effort of planning physical visits and travel to test facilities.

[0037] The features, advantages, and benefits of the above exemplary embodiments are intended to be only a part of the present disclosure and are not intended to be exhaustive or to limit the scope of the present disclosure. Further descriptions of the features, components, configuration, operation, and implementation of exemplary embodiments of the present disclosure are provided below.

[0038] 1 is a block diagram of an example system configuration 100 for facilitating cross-domain test provisioning, according to one or more embodiments. As shown in FIG. 1, system configuration 100 may include a test management system 110, multiple nodes 120-1 through 120-N, a network 130, and multiple test environments 140-1 through 140-N.

[0039] In general, test management system 110 may be communicatively coupled to multiple nodes 120-1 through 120-N (via network 130) and to multiple test environments 140-1 through 140-N, and configured to utilize the multiple test environments to provide cross-domain testing for one or more components (e.g., virtual ECUs, physical ECUs, etc.) associated with the multiple nodes. A description of example components that may be included in test management system 110 is provided below with reference to FIG. 2, and one or more operations performable by test management system 110, as well as associated use cases, are provided below with reference to FIGS. 3 through 6.

[0040] Each of the plurality of nodes 120-1 to 120-N may include one or more devices, equipment, systems, or any other suitable components that may receive, host, store, deploy, process, and / or provide one or more components that make up the system.

[0041] As an example, node 120-1 may include a device or equipment (e.g., a personal computer, a server or server cluster, a workstation, etc.) that may be utilized to build, store, execute, or simulate one or more computer-executable software applications, such as one or more virtualized ECUs, one or more emulated ECUs, and / or any other suitable software-based components of a vehicle system (e.g., a vehicle model, a data communications module (DCM) model, a heating, ventilation, and air conditioning (HVAC) model, etc.). As another example, node 120-1 may include or be associated with one or more hardware components, such as one or more fully developed physical ECUs, one or more partially developed physical ECUs, one or more vehicle hardware (e.g., a powertrain, etc.). Additionally or alternatively, the node may include one or more retarget equipment.

[0042] According to an embodiment, one or more of the plurality of nodes 120-1 through 120-N may include one or more interfaces, each of which may be configured to communicatively couple the associated node to test management system 110. For example, one or more of the plurality of nodes may include a program interface, a hardware interface, a software interface (e.g., an application program interface (API), etc.), etc.

[0043] According to an embodiment, at least some of the plurality of nodes 120-1 through 120-N are located in one or more geographic locations that are different from the test management system 110, different from another portion of the plurality of nodes, and / or different from the plurality of test environments 140-1 through 140-N.

[0044] Network 130 may include one or more wired and / or wireless networks that may be configured to couple multiple nodes 120-1 through 120-N to test management system 110. For example, network 130 may include a cellular network (e.g., a fifth-generation (5G) network, a long-term evolution (LTE) network, a third-generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., a public switched telephone network (PSTN)), a private network, an ad hoc network, an intranet, the Internet, an optical fiber-based network, etc., and / or a combination of these or other types of networks.

[0045] According to an embodiment, network 130 may include a virtual network that may include one or more physical network components (e.g., Ethernet, WiFi modules, telecommunications network hardware, etc.) implemented with one or more virtual network functions (e.g., Controller Area Network (CAN) bus, etc.). Additionally or alternatively, network 130 may include at least one parameter network. According to an embodiment, network 130 may also be configured to couple test management system 110 (or one or more components included therein) to other components, such as multiple test environments 140-1 through 140-N, one or more user devices or equipment (e.g., testers, etc.), etc.

[0046] The plurality of test environments 140-1 through 140-N may include at least one software-based test environment and / or at least one hardware-based test environment. According to an embodiment, the at least one software-based test environment may include at least one software-in-the-loop (SIL) test environment, at least one virtual ECU (V-ECU) test environment, at least one model-in-the-loop (MIL) test environment, at least one processor-in-the-loop (PIL) test environment, etc. According to an embodiment, the at least one hardware-based test environment may include at least one hardware-in-the-loop (HIL) test environment. Each of the test environments 140-1 through 140-N may be communicatively coupled to the test management system 110 and may provide one or more signals, information, data, etc. to or receive one or more signals, information, data, etc. from the test management system 110.

[0047] In general, the at least one hardware-based test environment may be configured to manage one or more tasks related to one or more physical hardware components, such as, but not limited to, executing or simulating tests involving the one or more physical hardware components, scheduling test execution, etc. For example, the at least one hardware-based test environment may be communicatively coupled to one or more developed (or partially developed) hardware / physical components and may simulate real-world environments or actual use cases to thereby test or evaluate the one or more hardware / physical components.

[0048] As an example, a physical engine ECU may be developed and tested before being incorporated into a vehicle. In such an example, instead of testing the engine ECU on an actual engine, at least one hardware-based test environment may perform a simulation of the engine interacting with the engine ECU. As another example, the in-vehicle functionality may include functionality of a software-based ECU (e.g., a virtual ECU, an emulated ECU, etc.) and a hardware-based ECU, and thus, the at least one hardware-based test environment may be configured to interoperate with the at least one software-based test environment via the test management system 110 to provide cross-domain testing (e.g., the at least one hardware-based test environment may perform tasks associated with the hardware-based ECU, and the at least one software-based test environment may perform tasks associated with the software-based ECU, with results associated therewith provided in aggregate to the test management system 110 for further processing).

[0049] Meanwhile, at least one software-based test environment may be configured to manage one or more tasks related to one or more software components, such as, but not limited to, executing or simulating tests, scheduling test execution, etc. While a hardware-based test environment is communicatively coupled to one or more physical / hardware components and performs tests / simulations thereon, a software-based test environment may simply acquire or receive one or more software components and perform software-based tests / simulations thereon. Simply put, a software-based test environment, like a hardware-based test environment, may be generated, deployed, and executed in any suitable computing device or environment without requiring connection to the physical / hardware components to be tested.

[0050] According to embodiments, one or more software components that may be tested in the at least one software-based test environment may include at least one virtual ECU and at least one emulated (or simulated) ECU. In this regard, the at least one virtual ECU may include application software, programming code, functional algorithms, etc. that define a final ECU (e.g., in terms of design, development, manufacturing, etc.), while the emulated ECU may differ from the at least one emulated ECU in that the at least one virtual ECU may include application software, programming code, functional algorithms, etc. that define a generalized or non-final ECU. Furthermore, the at least one virtual ECU and / or the at least one emulated ECU may include at least one of a central ECU (CECU), an instrument cluster (IC) ECU, an in-vehicle infotainment (IVI) ECU, an advanced driver assistance systems (ADAS) ECU, a chassis ECU, a powertrain ECU, a body ECU, and any other suitable type of ECU associated with a vehicle.

[0051] Additionally or alternatively, the one or more software components may include one or more software models, such as at least one simulated environmental condition model (e.g., a road condition model, a traffic condition model, a weather condition model, etc.), at least one vehicle-related model (e.g., a DCM model, an HVAC model, etc.), etc.

[0052] According to embodiments, the software-based test environment can simultaneously execute multiple tasks (e.g., multiple tests, multiple simulations, etc.). For example, the software-based test environment can execute multiple tests / simulations for one software component (e.g., one virtual ECU, etc.) in parallel, execute one test / simulation for multiple software components in parallel, and / or the like. Furthermore, the software-based test environment and the hardware-based test environment can simultaneously execute multiple tasks (e.g., multiple tests, multiple simulations, etc.). For example, the software-based test environment can execute one or more tests / simulations for one or more software components, while the hardware-based test environment can execute one or more tests / simulations for one or more hardware components (e.g., physical ECUs, etc.).

[0053] The multiple test environments 140-1 through 140-N may be executable on one or more test servers or communicatively coupled to one or more test servers. According to an embodiment, one or more of the multiple test environments 140-1 through 140-N may be deployed or executable on one or more of the multiple nodes 120-1 through 120-N. For example, a software-based test environment may be deployed and executed on one or more of the multiple nodes 120-1 through 120-N. Alternatively or additionally, one or more of the multiple test environments 140-1 through 140-N may be communicatively coupled to one or more of the multiple nodes 120-1 through 120-N (e.g., via a wired coupling, a wireless coupling, etc.). For example, a hardware-based test environment may be communicatively coupled to a node associated with a physical ECU (e.g., a fully developed hardware ECU, a partially developed hardware ECU, etc.). In that case, one or more of the multiple test environments may be communicatively coupled to the test management system 110 via the network 130, as described above.

[0054] To this end, it can be understood that one or more of the software components and / or one or more hardware components described above (i.e., components to be tested in multiple test environments 140-1 through 140-N) can be associated with (e.g., deployed, communicatively coupled to, etc.) one or more of multiple nodes 120-1 through 120-N, and test management system 110 can be configured to utilize multiple nodes and multiple test environments to facilitate provisioning of cross-domain tests.

[0055] Reference is now made to Figure 2, which illustrates a block diagram of exemplary components of a test management system 200 according to one or more embodiments. Test management system 200 may correspond to test management system 110 described above with reference to Figure 1, and therefore features described herein with reference to systems 110 and 200 may be applicable to one another unless expressly stated otherwise.

[0056] As shown in FIG. 2, test management system 200 may include at least one communication interface 210, at least one storage 220, and at least one processor 230; however, it may be understood that test management system 200 may include more or fewer components than shown and / or the components included therein may be arranged in any manner different from that shown without departing from the scope of the present disclosure.

[0057] Communications interface 210 may include transceiver-like components (e.g., a transceiver, a separate receiver and transmitter, etc.) that enable test management system 200 (or one or more components included therein) to communicate with one or more components external to test management system 200 via wired connections, wireless connections, a combination of wired and wireless connections, etc. For example, communications interface 210 may couple test management system 200 (or one or more components included therein) to multiple test environments (e.g., test environments 140-1 through 140-N of FIG. 1 ), thereby enabling them to communicate and interoperate with each other. As another example, communications interface 210 may couple test management system 200 (or one or more components included therein) to multiple nodes (e.g., nodes 120-1 through 120-N of FIG. 1 ), thereby enabling them to communicate and interoperate with each other. Similarly, communications interface 210 may enable components of test management system 200 to communicate with each other. For example, communication interface 210 may couple storage 220 to processor 230, thereby allowing them to communicate and interoperate with each other.

[0058] According to embodiments, communication interface 210 may include a hardware-based interface, such as a bus interface, an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, a software interface, etc. According to embodiments, communication interface 210 may include at least one controller area network (CAN) bus configurable to communicatively couple components of test management system 200 (e.g., storage 220, processor 230, etc.) to multiple nodes (e.g., nodes 120-1 through 120-N) and / or multiple test environments (e.g., test environments 140-1 through 140-N). Additionally or alternatively, communication interface 210 may include a software-based interface, such as an application programming interface (API), a virtual network interface (e.g., a virtual CAN bus, etc.), etc.

[0059] At least one storage 220 may include one or more storage media suitable for storing data, information, and / or computer-readable / computer-executable instructions. According to an embodiment, storage 220 may include random access memory (RAM), read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, and / or optical memory) that stores information and / or instructions for use by processor 230. Additionally or alternatively, storage 220 may include a hard disk (e.g., a magnetic disk, optical disk, magneto-optical disk, and / or solid-state disk), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cartridge, a magnetic tape, and / or another type of non-transitory computer-readable medium, along with a corresponding drive.

[0060] According to an embodiment, storage 220 can operate as a centralized library and can be configured to store information utilized by processor 230 to manage one or more tests (e.g., cross-domain tests, etc.). For example, storage 220 may be configured to store one or more parameters or configurations predefined or predetermined by one or more users (e.g., testers associated with one or more nodes, etc.), such as a test cycle, one or more test conditions, one or more configuration files, and / or the like. Additionally, storage 220 can be configured to store information related to the capacity and / or availability of each of a plurality of test environments communicatively coupled to the test management system, such as capacity information of a test server (or any other suitable node) on which the test environment is deployed / hosted, the processing capacity of a simulator (e.g., HIL simulator, SIL simulator, etc.) of the test environment, historical test results, usage costs, processing / test speed (or parameters that can determine the processing / test speed), etc. Additionally, storage 220 may store computer-readable instructions that, when executed by one or more processors (e.g., processor 230), cause the one or more processors to perform one or more operations described herein.

[0061] At least one processor 230 may include one or more processors that may be programmed or configured to perform functions or operations to facilitate provisioning of cross-domain tests. For example, processor 230 may be configured to perform one or more actions or operations described herein by executing computer-readable instructions stored on a storage medium (e.g., storage 220, etc.).

[0062] According to an embodiment, processor 230 may be configured to receive one or more signals (e.g., via communications interface 210, etc.) that define one or more instructions for performing one or more operations. Furthermore, processor 230 may be implemented in hardware, firmware, or a combination of hardware and software. Processor 230 may include a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), and / or another type of processing or computing component.

[0063] According to an embodiment, processor 230 may be configured to execute computer-executable instructions stored in at least one memory storage (e.g., storage 220) to thereby perform one or more operations for managing one or more tests of one or more components (e.g., software and / or hardware components deployed on or associated with one or more of the multiple nodes).

[0064] 3, which illustrates a flow diagram of an example method 300 for facilitating cross-domain test provisioning, according to one or more embodiments. Method 300 may be performed by at least one processor (e.g., processor 230) of a test management system to test one or more software associated with an embedded system (e.g., an on-board ECU, etc.) of a vehicle.

[0065] In operation S310, at least one processor of the test management system may be configured to determine whether execution of a test should be triggered. For example, the at least one processor may determine whether one or more conditions for executing a test are met. The one or more conditions may be predefined by one or more users (e.g., users associated with one or more of the plurality of nodes) and stored in one or more storage media (e.g., storage 220, etc.). By way of example, the one or more conditions may include one or more test requirements being met / violated, one or more thresholds being achieved, one or more changes in the software and / or one or more associated components (e.g., software / hardware components within the node, etc.) being detected, etc.

[0066] According to an embodiment, at least one processor may determine whether one or more conditions for executing a test are met by detecting a change in software (e.g., a virtual ECU, etc.). For example, the at least one processor may determine whether the software has changed from a previous version and determine whether the change is a breaking change. A description of an example use case related to operation S310 is provided below with reference to FIG. 4.

[0067] Thus, based on determining that one or more conditions for running a test are met, the at least one processor may determine that running a test should be initiated, and method 300 may proceed to operation S320.

[0068] In operation S320, the at least one processor may be configured to acquire a test bench. Generally, a test bench in software testing may refer to information or parameters, such as a set of procedures, a test scenario configuration, etc., that provide a simulation or configuration of required test inputs. For example, a test bench for testing an ECU may include information or parameters for simulating a load for testing the ECU, simulating one or more ECUs associated with the ECU, simulating one or more test conditions (e.g., road conditions, weather conditions, etc.), etc. According to an embodiment, the test bench may include a general test bench that defines the configuration of a test environment for normal operation, and may include general test benches that define configurations, conditions, and / or services (in addition to the general test bench) for testing specific functions or situations.

[0069] According to an embodiment, in operation S320, at least one processor may obtain one or more test configuration files associated with components to be tested (e.g., software components, hardware components, etc.), and may obtain a test bench based on the one or more test configuration files.

[0070] The one or more test configuration files may include information or parameters that define one or more test configurations, such as a mapping of component functionality and the type of test environment associated therewith (e.g., function A of the first ECU should be tested in a software-based test environment, function B of the first ECU should be tested in a hardware-based test environment, etc.), resources required to test the component functionality (e.g., computing power, memory, etc.), and load information associated with the functionality.

[0071] According to embodiments, one or more test configuration files may include information on multiple ECUs (e.g., virtual ECUs, emulated ECUs, physical ECUs, etc.) associated with a component (e.g., software) under test. The information may include source code, algorithms, functionality, etc. of the multiple ECUs, and may include information on a test environment associated with each of the multiple ECUs. In some implementations, at least some of the multiple ECUs may be geographically distributed and located at locations different from the component under test. Alternatively or additionally, some of the multiple ECUs may be associated with (e.g., enabled / deployed, communicatively coupled, etc.) one or more nodes different from the node of the component under test. Furthermore, the multiple ECUs may include at least one of a central ECU (CECU), an instrument cluster (IC) ECU, an in-vehicle infotainment (IVI) ECU, an advanced driver assistance systems (ADAS) ECU, and any other suitable type of ECU.

[0072] Thus, the at least one processor may obtain the test bench based on the one or more test configuration files. According to an embodiment, the test bench may be pre-generated and pre-stored in one or more storage media (e.g., storage 220, an external server, etc.), and the at least one processor may obtain the test bench (based on the one or more test configuration files) from the one or more storage media in operation S320. Alternatively, the at least one processor may generate the test bench in real time or near real time based on the one or more test configuration files in operation S320. According to an embodiment, the at least one processor may obtain or generate a replica of a component related to the component under test (e.g., a cloned version of an ECU related to the component under test, etc.) and include the replica in the test bench.

[0073] Upon obtaining the test bench, method 300 proceeds to operation S330, where at least one processor of the test management system may be configured to determine a plurality of test environments associated with the test bench. The plurality of test environments may include at least one software-based test environment (e.g., a software-in-the-loop (SIL) test environment) and at least one hardware-based test environment (e.g., a hardware-in-the-loop (HIL) test environment), similar to what was described above with reference to the plurality of test environments 140-1 through 140-N of FIG. 1. Subsequently, upon determining the plurality of test environments, method 300 proceeds to operation S340, where the at least one processor may be configured to perform cross-domain testing based on the plurality of test environments. A description of example operations associated with operations S330 and S340, as well as example use cases associated therewith, is provided below with reference to FIGS. 5 and 6.

[0074] 4, which illustrates a flow diagram of an example use case associated with operation S310 of FIG. 3, in accordance with one or more embodiments. In this example use case, the test management system 410 is configured to continuously (or periodically) monitor the status of component(s) deployed in or associated with the first node 420-1 and the second node 420-2 according to a test cycle associated with each of the first node 420-1 and the second node 420-2, and determine whether to invoke test execution to test the component(s).

[0075] Here, the test management system 410 of Figure 4 corresponds to the test management system 110 of Figure 1 and the test management system 200 of Figure 2, and the first node 420-1 and the second node 420-2 of Figure 4 may correspond to some of the multiple nodes 120-1 to 120-N of Figure 1. Furthermore, it can be understood that the test management system 410 may include at least one processor (e.g., processor 230) as described above with reference to Figure 2, and one or more operations of Figure 4 may be performed by the at least one processor. Therefore, redundant descriptions related thereto may be omitted below for the sake of brevity.

[0076] 4 , in operations S410-1 and S410-2, at least one processor of the test management system 410 may be configured to communicate with the first node 420-1. According to an embodiment, the at least one processor may send a query or request (e.g., via an API call, etc.) to the first node 420-1 for information regarding the current status of one or more components associated with the first node 420-1 (e.g., software components deployed on or enabled to run (hosted) in the first node 420-1, hardware components communicatively coupled to the first node 420-1, etc.). Thus, the first node 420-1 can provide the requested information to the at least one processor of the test management system 410.

[0077] According to an embodiment, at least one processor may continuously (or periodically) perform operations S410-1 and S410-2 according to a first test cycle associated with the first node 420-1 and / or associated with one or more components associated therewith. The first test cycle may be predetermined by one or more users associated with the first node 420-1 (e.g., a manager of the first node 420-1, a developer of a software component deployed to the first node 420-1, etc.), and the first test cycle may be pre-stored in one or more storage media (e.g., storage of the test management system 410, a server external to the test management system 410, etc.).

[0078] As an example, the first test cycle may define that a first component associated with the first node 420-1 should be checked at time intervals such as every five minutes, every hour, every day, etc. Accordingly, the at least one processor of the test management system 410 may perform operations S410-1 and S410-2 every five minutes, every hour, every day, etc. to obtain up-to-date status information of the first component from the first node 420-1. Additionally, the at least one processor may perform operation S410-2, such as five minutes after performing operation S410-1 (or after another suitable time interval defined by the first test cycle).

[0079] In operations S430-1 and S430-2, it is contemplated that at least one processor of the test management system 410 may be configured to communicate with the second node 420-2 to request information on the current status of one or more components associated with the second node 420-2 according to a second test cycle predetermined by one or more users associated with the second node 420-2, in a manner similar to that described above with reference to operations S410-1 and S410-2.

[0080] Further, it may be understood that the second test cycle (in which at least one processor performs operations S430-1 and S430-2) may be similar to or different from the first test cycle (in which at least one processor performs operations S410-1 and S410-2). Furthermore, it may be understood that the at least one processor may be configured to perform communications with the first node 420-1 (operations S410-1 and S410-2) and communications with the second node 420-2 (operations S430-1 and S430-2) in any suitable sequence. For example, at least one processor may perform communications with the first node 420-1 and the second node 420-2 simultaneously (e.g., operations S410-1 and S430-1 may be performed in parallel, operations S410-2 and S430-2 may be performed in parallel, etc.), or may perform communications with the second node 420-2 before performing communications with the first node 420-1 (e.g., operation S430-1 may be performed before operation S410-1, etc.), without departing from the technical scope of the present disclosure.

[0081] Further referring to FIG. 4, upon communicating with the first node 420-1 and the second node 420-2, at least one processor of the test management system 410 may be configured to determine whether test execution should be invoked on the associated components based on information or data obtained from the first node 420-1 and the second node 420-2.

[0082] For example, in operation S420-1, the at least one processor may determine whether one or more conditions for performing a test are met. As an example, the at least one processor may determine whether a change (from a previous version) of a component in the first node 420-1 has occurred, and based on a determination that a change of the component has occurred, determine whether a breaking change has occurred.

[0083] In this regard, a "breaking change" may refer to a modification to the software (or one or more components associated therewith) that causes existing functionality of the software to no longer function as intended or to no longer be stable. Additionally, a breaking change may be a change that alters the behavior of the software in a manner that affects other parts associated with the software (e.g., other software components, other hardware components, etc.) and leads to a software failure. To this end, the conditions for determining a breaking change may be predefined by a user associated with the component under test or by another user associated with the component under test (e.g., a user associated with another ECU associated with the component under test, etc.).

[0084] Thus, in operation S420-1, upon detecting a change to the component under test, the at least one processor may determine (based on one or more predefined conditions) whether the change is a breaking change, and may determine that execution of a test for the component under test is necessary based on determining that the change is a breaking change.

[0085] It may be understood that in addition to or instead of determining a breaking change, the at least one processor may determine whether any other condition is met to determine whether test execution should be invoked. For example, the at least one processor may determine whether performance of an associated node (e.g., first node 420-1) has degraded, whether a preset test execution schedule has been reached, etc., and determine whether test execution should be invoked based thereon. It will also be understood that the at least one processor of the test management system may be configured to perform operations S440-1, S420-2, and S440-2 in a manner similar to that described above with reference to operation S420-1.

[0086] To this end, at least one processor of the test management system 410 may continuously (or periodically) determine, in an automated manner, whether test execution should be invoked to test one or more components associated with one or more nodes.

[0087] Reference is now made to Figure 5, which illustrates a flow diagram of an example method 500 for selecting multiple test environments and running cross-domain tests therein, according to one or more embodiments. One or more operations of method 500 may be part of operations S330 and S340 of Figure 3 and may be performed by at least one processor (e.g., processor 230) of a test management system.

[0088] In operation S510, at least one processor of the test management system may be configured to create a test job. For example, the at least one processor may create a test job based on the test bench (e.g., obtained in operation S320) that includes a plurality of tasks, each of which may include information (e.g., requirements, etc.) related to a test to be performed.

[0089] As an example, assuming that cross-domain testing needs to be performed to test changed / modified software (e.g., a modified virtual ECU) under a software-based test environment (e.g., a SIL test environment) and a hardware-based test environment (e.g., a HIL test environment), at least one processor can create a cross-domain test job including multiple tasks, each of which includes information such as runtime information, required hardware resource information (e.g., type of physical ECU required), required software resource information (e.g., CPU power, memory, etc.), component replica information, etc., and this cross-domain test job can be utilized by the at least one processor to determine an optimal software-based test environment and an optimal hardware-based test environment for executing the cross-domain test.

[0090] Upon creating the test job, method 500 proceeds to operation S520, where at least one processor of the test management system may be configured to select, based on the created test job, multiple test environments for executing the test.

[0091] As an example, assuming a cross-domain test is created (as described above with reference to operation S510), the at least one processor may determine test environment requirements for executing each task of the test based on one or more requirements for executing the tasks of the cross-domain test. The at least one processor may then select an appropriate or optimal test environment for executing the tasks of the cross-domain test job from among a plurality of test environments (e.g., a plurality of test environments 140-1 through 140-N) communicatively coupled to the test management system.

[0092] For example, the at least one processor may select, from among a plurality of software-based test environments communicatively coupled to the test management system, one or more software-based test environments that are available for executing software-based tests and that meet the necessary requirements (e.g., software resource requirements, etc.), and the at least one processor may select, from among a plurality of hardware-based test environments communicatively coupled to the test management system, one or more hardware-based test environments that are available and that meet the necessary requirements (e.g., hardware resource requirements, etc.).

[0093] Upon selecting the test environment, method 500 proceeds to operation S530, where at least one processor of the test management system may be configured to assign one or more tasks to the selected test environment. Accordingly, the test environment may execute the assigned tasks and then provide execution results to the at least one processor. The at least one processor may then generate (e.g., compile, aggregate, etc.) test results for the cross-domain test based on the test results associated with the assigned tasks and provided by the multiple test environments.

[0094] Reference is now made to Figure 6, which illustrates a flow diagram of an example use case associated with one or more operations of the method of Figure 5, according to one or more embodiments. In this example use case, a test management system 610 is communicatively coupled to multiple test servers 620-1 through 620-N and configured to select one or more test environments associated with the multiple test servers to facilitate provisioning of cross-domain tests.

[0095] In this regard, it may be understood that test management system 610 of FIG. 6 corresponds to test management system 110 of FIG. 1, test management system 200 of FIG. 2, or test management system 410 of FIG. 4, and that each of multiple test servers 620-1 to 620-N may be associated with one or more of multiple test environments 140-1 to 140-N of FIG. 1.

[0096] Furthermore, it may be understood that the test management system 610 may include at least one processor (e.g., processor 230), and that one or more operations in Figure 6 may be performed by at least one processor. It may also be understood that one or more operations in Figure 6 may be performed after one or more operations in Figures 3 and 4.

[0097] 6, in operation S610, at least one processor of the test management system 610 may be configured to create a test job. Details of this operation are similar to those described in operation S510 of FIG. 5, and therefore, redundant description may be omitted below for the sake of brevity.

[0098] Upon creating a test job, at least one processor of test management system 610 may be configured to communicate with a plurality of test servers 620-1 through 620-N (operations S620-1 through S620-N, respectively) to obtain information of the associated test environment.

[0099] 6, the at least one processor may first obtain (in operation S620-1) from the first test server 620-1 availability information, capability information, and / or any other suitable type of information of one or more test environments associated with (e.g., enabled / deployed on, communicatively coupled to, etc.) the first test server 620-1. For example, the at least one processor may generate one or more API calls to request the required information from the first test server 620-1 and provide the generated API calls to the first test server 620-1 (via the communication interface). Thus, the at least one processor may obtain the required information from the remaining test servers 620-2 through 620-N in a similar manner.

[0100] It may be understood that the at least one processor of the test management system 610 may also obtain the required information from the multiple test servers 620-1 through 620-N in any suitable order. For example, the at least one processor may simultaneously perform operations S620-1 and S620-2 to simultaneously obtain the required information for multiple test environments associated with the first test server 620-1 and the second test server 620-2, or may perform operation S620-2 before operation S620-2 to obtain first capability information from the second test server 620-2.

[0101] Upon receiving the necessary information from the plurality of test servers, in operation S630, at least one processor of the test management system 610 may be configured to select, from among the plurality of test environments associated with the plurality of test servers, a plurality of test environments that are suitable or optimal for performing one or more tasks of the created test job.

[0102] For example, the at least one processor may select or choose multiple test environments according to required information (e.g., availability, capacity, cost, etc.) and the type of test environment (e.g., a hardware-based test environment such as a HIL test environment, a software-based test environment such as a SIL test environment, a V-ECU test environment, etc.) required to perform one or more tasks of the test job.

[0103] As an example, assuming a test job is a cross-domain test job that includes a first task for testing virtual ECU A in a SIL test environment with CPU power requirements X, and a second task for testing virtual ECU A with physical ECU B, etc., the at least one processor may select one or more test environments within test servers 620-1 through 620-N that have sufficient CPU power and are capable of performing SIL testing as needed for the first task. Similarly, the at least one processor may select one or more test environments within test servers 620-1 through 620-N that are associated with physical ECU B and are capable of performing HIL testing on physical ECU B, as needed for the second task. It should be understood that the at least one processor may perform any other suitable operations for selecting appropriate test environments for executing one or more tasks of the created test job without departing from the scope of the present disclosure.

[0104] Upon selecting the multiple test environments, the at least one processor may be configured to assign or allocate tasks of the test job to the respective test environments. In the example use case of FIG. 6 , it may be assumed that a first test environment (e.g., a SIL test environment) associated with the first test server 620-1 is selected to perform a first task of the test job, and a second test environment (e.g., a HIL test environment) associated with the second test server 620-2 is selected by the at least one processor to perform a second task of the test job. Thus, in operations S640-1 and S640-2, the at least one processor may be configured to assign or allocate the first task and the second task to the first test environment and the second test environment, respectively.

[0105] According to embodiments, the at least one processor may be configured to assign or allocate tasks of the test job by generating signals, messages, instructions, etc. that include information about the tasks (e.g., a schedule for executing the tasks, a target time for completion of the tasks, etc.). According to embodiments in which the tasks include testing other software components (e.g., other virtual ECUs, emulated ECUs, software models such as traffic models, etc.), the signals, messages, instructions, etc. generated by the at least one processor may further include replicas of the other software components (e.g., cloned versions of source code, algorithms, etc.). The at least one processor may then provide the generated signals / messages / instructions to a test server that is enabling or deploying the selected test environment.

[0106] In this regard, the at least one processor of the test management system 610 may first assign a task to a first test environment enabled to execute or deployed on a first test server 620-1 (operation S640-1), and then assign a task to a second test environment enabled to execute or deployed on a second test server 620-2 (operation S640-2). Alternatively, the at least one processor may allocate multiple tasks for simultaneous execution on the first and second test environments. It may be understood that the at least one processor may also allocate or assign tasks in any other suitable sequence.

[0107] After allocating or assigning one or more tasks to the test environments, the test environments may execute one or more tests according to the assigned tasks. Thereafter, in operations S650-1 and S650-2, at least one processor of the test management system 610 may be configured to monitor the performance of the test environments, receive one or more test results of the assigned tasks, etc. from the test server, and update the relevant information in the record file accordingly. For example, the at least one processor may aggregate test results provided by multiple test environments and generate or compile a complete comprehensive test result representing the test results of the cross-domain tests.

[0108] As another example, whenever one or more test results indicate a test execution failure (e.g., due to an exception or timeout that causes a task to not complete within a specified period of time), the at least one processor may update the record file to avoid reassigning similar tasks to the same test environment in future tests. Similarly, whenever one or more test results indicate a particular parameter (e.g., time or speed) that differs from that estimated from or reflected in the record, the at least one processor may update the record file accordingly to more accurately reflect the current capabilities of the corresponding test environment.

[0109] Additionally, upon receiving a report of or determining a task failure, the at least one processor may be configured to reallocate the failed task to another test environment. For example, whenever a task failure is reported due to a timeout, the at least one processor may reallocate the task to another test environment whose associated capability information indicates a higher test speed than the test environment to which the task was previously assigned.

[0110] In view of the above, it can be understood that the at least one processor can be configured to assign one or more tasks based on test results of previous tests. For example, to test a software component (e.g., a virtual ECU), a first series of tests or simulations is performed, and then source code of the software component is updated. In this regard, the at least one processor can optimize task assignment (e.g., reduce test execution time, etc.) for testing the software component with the updated source code in a second series of tests or simulations based on results of the first series of tests or simulations.

[0111] It should be understood that the specific order or hierarchy of blocks in the processes / flowcharts disclosed herein is an example of an example approach. Based on design preferences, it should be understood that the specific order or hierarchy of blocks in the processes / flowcharts may be rearranged. Additionally, some blocks may be combined or omitted. The accompanying method claims present elements of the various blocks in an example order and are not limited to the specific order or hierarchy presented.

[0112] Some embodiments may relate to systems, methods, and / or computer-readable media at any possible level of technical detail of integration. Furthermore, one or more of the above components described above may be implemented as instructions stored on a computer-readable medium and executable by at least one processor (and / or may include at least one processor). The computer-readable medium may include a computer-readable non-transitory storage medium having computer-readable program instructions for causing a processor to perform operations.

[0113] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction-execution device. A computer-readable storage medium can be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves having instructions recorded thereon, and any suitable combination of the above. Computer-readable media, as used herein, should not be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted over wires.

[0114] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or can be downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0115] The computer-readable program code / instructions for performing operations may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or object-oriented programming languages ​​such as Smalltalk, C++, etc., and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, partially on the user's computer as a standalone software package, and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer-readable program instructions by utilizing state information in the computer-readable program instructions to personalize the electronic circuitry to perform aspects or operations.

[0116] These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to fabricate a machine, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram blocks. These computer-readable program instructions can also be stored on a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, and / or other device to function in a particular manner, such that the computer-readable storage medium having stored thereon instructions comprises an article of manufacture containing instructions that implement aspects of the functions / acts specified in the flowchart and / or block diagram blocks.

[0117] The computer-readable program instructions can also be loaded into a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to generate a computer-implemented process such that the instructions, executed on the computer, other programmable apparatus, or other device, implement the functions / acts specified in the flowchart and / or block diagram blocks.

[0118] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable media according to various embodiments. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction set, comprising one or more executable instructions for implementing the specified logical function(s). The methods, computer systems, and computer-readable media may include additional, fewer, different, or differently arranged blocks than shown in FIG. 1. In some alternative implementations, the functions depicted in the blocks may occur out of the order depicted in the figures. For example, two blocks shown in succession may in fact be executed concurrently or substantially concurrently, or the blocks may be executed in the reverse order, depending on the functionality involved. It will also be noted that each block in the block diagram and / or flowchart illustrations, and combinations of blocks in the block diagram and / or flowchart illustrations, may be implemented by special-purpose hardware-based systems that perform the specified functions or operations and execute a combination of special-purpose hardware and computer instructions.

[0119] The systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual specialized control hardware or software code used to implement these systems and / or methods is not intended to limit the implementation. As such, the operation and behavior of the systems and / or methods have been described herein without reference to specific software code, and it will be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

Claims

1. 1. A method, implemented by at least one processor, for facilitating cross-domain test provisioning for testing software of an embedded system, comprising: detecting modification of said software; obtaining a test configuration file associated with the software; obtaining a testbench based on the test configuration file; determining a plurality of test environments associated with the test bench; executing the cross-domain test based on the plurality of test environments; Including, the software of the embedded system includes an on-board electronic control unit (ECU); the plurality of test environments includes at least one software-in-the-loop (SIL) test environment and at least one hardware-in-the-loop (HIL) test environment; method.

2. The modification of the software includes a breaking modification. The method of claim 1.

3. The test configuration file includes information on a plurality of ECUs associated with the software and information on a test environment associated with each of the plurality of ECUs. The method according to claim 1 or claim 2.

4. At least some of the plurality of ECUs are associated with one or more nodes different from the software. The method of claim 3.

5. At least some of the plurality of ECUs are distributed and located in locations geographically different from the software. The method of claim 3.

6. The plurality of ECUs includes at least one virtual ECU, at least one emulated ECU, at least one physical ECU, or a combination thereof. The method of claim 3.

7. The plurality of ECUs include at least one of a central ECU (CECU), an instrument cluster (IC) ECU, an in-vehicle infotainment (IVI) ECU, and an advanced driver assistance system (ADAS) ECU. The method of claim 3.

8. Detecting modifications to said software obtaining information about a current status of the software from a node associated with the software; determining whether the software has been modified from a previous version based on the obtained status information; determining whether the change is the breaking change based on determining that the software has been modified; 3. The method of claim 2, comprising:

9. determining the plurality of test environments creating a test job including a plurality of tasks based on the obtained test bench; selecting the plurality of test environments based on one or more requirements for performing the plurality of tasks; 3. The method of claim 1 or claim 2, comprising:

10. performing the cross-domain test, assigning one or more tasks of the test job to the plurality of test environments; receiving test results associated with the assigned one or more tasks from the plurality of test environments; generating test results for the cross-domain test based on the test results associated with the assigned one or more tasks; 10. The method of claim 9, comprising:

11. 1. A system for facilitating cross-domain test provisioning for testing software of an embedded system, comprising: at least one memory storage device storing computer-executable instructions; communicatively coupled to the at least one memory storage; Detecting a change in the software; obtaining a test configuration file associated with the software; Obtaining a test bench based on the test configuration file; determining a plurality of test environments associated with the test bench; executing the cross-domain test based on the plurality of test environments; at least one processor configured to execute the computer-executable instructions for: Including, The embedded system software includes an on-board electronic control unit (ECU); the plurality of test environments includes at least one software-in-the-loop (SIL) test environment and at least one hardware-in-the-loop (HIL) test environment; system.

12. The modification of the software includes a breaking modification. The system of claim 11.

13. The test configuration file includes information on a plurality of ECUs associated with the software and information on a test environment associated with each of the plurality of ECUs.

13. A system according to claim 11 or claim 12.

14. At least some of the plurality of ECUs are associated with one or more nodes different from the software. The system of claim 13.

15. At least some of the plurality of ECUs are distributed and located in locations geographically different from the software. The system of claim 13.

16. The plurality of ECUs includes at least one virtual ECU, at least one emulated ECU, at least one physical ECU, or a combination thereof. The system of claim 13.

17. The plurality of ECUs include at least one of a central ECU (CECU), an instrument cluster (IC) ECU, an in-vehicle infotainment (IVI) ECU, and an advanced driver assistance system (ADAS) ECU. The system of claim 13.

18. The at least one processor obtaining information about the current status of the software from a node associated with the software; determining whether the software has been modified from a previous version based on the obtained status information; determining whether the change is the breaking change based on determining that the software has been modified; configured to execute the computer-executable instructions for detecting modification of the software by The system of claim 12.

19. The at least one processor creating a test job including a plurality of tasks based on the obtained test bench; selecting the plurality of test environments based on one or more requirements for performing the plurality of tasks; configured to execute the computer-executable instructions for determining the plurality of test environments by 13. A system according to claim 11 or claim 12.

20. The at least one processor assigning one or more tasks of the test job to the plurality of test environments; receiving test results associated with the assigned one or more tasks from the plurality of test environments; generating test results for the cross-domain test based on the test results associated with the one or more assigned tasks; configured to execute the computer-executable instructions for performing the cross-domain test by 20. The system of claim 19.

Images