Vehicle testing environment management service

Abstract

The Vehicle Software Test Environment Management System provides a virtual vehicle environment that includes virtual electronic control units (vECUs) with virtual bus connection configurations used to simulate each of the electronic control units (ECUs) of a real-world vehicle. Based on the respective configurations of each of the ECUs, the Vehicle Software Test Environment Management System determines the instance type of one or more virtual compute instances used to implement the vECU, and further determines the respective machine images to emulate the respective software environment of each of the ECUs. The Vehicle Software Test Environment Management System can also deploy a vehicle software application to be certified on one or more of the vECUs and test the deployed vehicle software application using recorded signals from one or more ECUs of a real-world vehicle.

Description

Background Art

[0001] Modern vehicles such as passenger cars, trucks, and motorcycles are often manufactured with electronic sensors, extract inputs from such electronic sensors, and include a computer system programmed with a control algorithm that determines various control actions to be performed on the vehicle or a system implemented on the vehicle. Some vehicles may include multiple electronic control units (ECUs) and sensors with various sensor modalities. Additionally, the deployment of vehicle software applications may require that the vehicle software applications be tested and certified on a vehicle equipped with multiple ECUs and sensors. However, testing vehicle software applications on production vehicles can be slow or very costly.

Brief Description of the Drawings

[0002] [Figure 1] Illustrates a vehicle software test environment management system that receives a vehicle deployment graph showing various electronic control units (ECUs) of a vehicle and showing the network configuration of the vehicle, and the vehicle software test environment management system provides instructions (or orchestrates the implementation) for implementing virtual electronic control units (vECUs) having a virtual bus connection configuration that simulates the virtual bus connection configuration of the vehicle, and the configured ECUs are used in performing software authentication of one or more software applications to be deployed on the vehicle. [Figure 2A] Illustrates a graphical diagram of a vehicle software test environment management system, its various parts, and interactions for deploying a vehicle software application for which authentication is being performed to the vECUs of a virtual test vehicle having vECUs configured according to a virtual bus connection configuration. [Figure 2B]This paper illustrates a vehicle software test environment management system that performs test planning for vehicle software applications using a vECU of a virtual test vehicle having a vECU configured according to a virtual bus connection configuration, with graphical diagrams illustrating its various parts and interactions. [Figure 2C] This paper illustrates a vehicle software test environment management system, its various components, and their interactions graphically, for providing the results of implementing a test plan and requesting / obtaining certification based on the received results, according to several embodiments. [Figure 3A] A more detailed diagram illustrating a vehicle software test environment management system, based on several embodiments, is provided, which acquires available information regarding virtual instance types and machine images and determines the virtual instance types and machine images required to simulate each ECU of a vehicle using each vECU. [Figure 3B] A more detailed diagram illustrates a vehicle software test environment management system that provisions virtual instances, each having a machine image and a virtual instance type selected to simulate the vehicle's ECU based on a vehicle deployment graph, according to several embodiments. [Figure 3C] A more detailed diagram illustrates a vehicle software test environment management system that, according to several embodiments, constitutes a simulated vehicle bus interface of virtual instances, each having a machine image and a virtual instance type selected to simulate the vehicle's ECUs in the vehicle's network configuration according to a vehicle deployment graph. [Figure 4] A more detailed diagram illustrating a vehicle software test environment management system, which provides vECUs implemented in a virtual compute instance to simulate a subset of vehicle ECUs for partial simulation, is illustrated in several embodiments. [Figure 5]This section illustrates a more detailed diagram of a vehicle software test environment management system that deploys vehicle software applications to be certified to a virtual test vehicle using an emulated over-the-air (OTA) agent of a virtual test vehicle, according to several embodiments. [Figure 6] A more detailed diagram illustrating a vehicle software test environment management system that requests / obtains certification of a vehicle software application based on test results from a virtual test vehicle and / or the results of tests conducted using one or more actual vehicles, according to several embodiments, is provided. [Figure 7] This document illustrates flowcharts of operations performed by a vehicle software test environment management system to provide a virtual electronic control unit (vECU) having a virtual bus connection configuration determined based on a vehicle deployment graph, according to several embodiments, and the vECU connected using the virtual bus connection configuration can be used to simulate the vehicle environment of a real-world vehicle. [Figure 8] This document illustrates flowcharts of operations performed by a vehicle software test environment management system for deploying a software application to a vECU and certifying the software application based on the execution of a received test plan, according to several embodiments. [Figure 9] This document illustrates flowcharts of operations performed by a vehicle software test environment management system to emulate the ECUs of real-world vehicles in several embodiments, by adding new machine images to a machine image catalog and selecting each machine image from the machine image catalog to be implemented on the vECU. [Figure 10] Block diagrams illustrating exemplary computer systems that implement some or all of the techniques described herein, according to several embodiments, are illustrated below.

[0003] Embodiments are described herein as examples of several embodiments and illustrative drawings, but those skilled in the art will recognize that embodiments are not limited to those described or shown in the drawings. It should be understood that the drawings and their detailed description are not intended to limit embodiments to any particular form disclosed, but rather to encompass all modifications, equivalents, and substitutes included in the spirit and scope defined by the appended claims. Headings used herein are for structural purposes only and are not intended to limit the scope of the description or claims. As used throughout this application, the word “may” is used in an allowable sense (e.g., having the potential to) rather than an essential sense (e.g., having the potential to). Similarly, the words “include,” “including,” and “includes” mean including, but not limited to. [Modes for carrying out the invention]

[0004] The systems and methods described herein include techniques for implementing a vehicle software test environment management system that provides instructions for implementing (or orchestrates the implementation of) a virtual vehicle environment comprising a virtual electronic control unit (vECU), the vECU simulating an electronic control unit (ECU) of a real-world vehicle, the vECU may be configured according to a virtual bus connection configuration, the virtual bus connection configuration simulating a bus and / or network configuration of a real-world vehicle. The vehicle software test environment management system may determine each instance type to be used for one or more virtual compute instances implementing the vECU based on the configuration of each virtual compute instance, such as the processor type, the configuration of each ECU being selected to approximate the configuration of each ECU of a real-world vehicle. The vehicle software test environment management system may further select each machine image to be used to emulate the software environment of each ECU. In some embodiments, the vehicle software test environment management system may deploy (or provide instructions for deploying or orchestrating deployment) vehicle software applications to be authenticated on a virtual vehicle environment, and according to some embodiments, it may use recorded signals from the ECUs of a real-world vehicle to execute (or provide instructions for executing) a test plan.

[0005] In some embodiments, vehicle software may need to undergo a testing and certification process before it may be permitted to be deployed to a vehicle. The testing and certification process may require the vehicle software to be deployed to one or more real-world vehicles, undergo various steps of a test plan, and analyze the resulting real-world vehicle signals to determine whether the signals fall within the threshold range required by the test plan. For example, the testing and certification process for automatic brake control software may include deploying the automatic brake control software to a specific real-world vehicle (or set of real-world vehicles), driving the vehicle toward an obstacle, and analyzing the force applied to the brakes of the real-world vehicle and the speed reading from the vehicle's speedometer. Based on the results of the testing against the real-world vehicle, the brake control software may be certified to that real-world vehicle (or another vehicle considered to be similar to a real-world vehicle) and then deployed to it.

[0006] The testing and certification process for real-world vehicles can be slow and resource-intensive. For example, testing automatic brake control software may require either actively driving a real-world vehicle to encounter test scenarios as required by the test plan, or passively driving the real-world vehicle until it encounters the test scenarios. Furthermore, certification of vehicle software applications is often limited to specific real-world vehicle environments, and such limitations can increase costs in both time and resources required to test and certify vehicle software on physical vehicles. For example, certification of automatic brake control software on a first vehicle of manufacturer "A" and model "B" may be limited to that specific vehicle manufacturer / model. To enable the same automatic brake control software to be deployed on another vehicle having manufacturer "C" and model "D", the automatic brake control software may need to be tested on a different manufacturer / model vehicle. Therefore, depending on the certification requirements, it may be necessary to individually test the vehicle software application on each of many different vehicle combinations. This can significantly increase the costs associated with implementing a new software application and further slow down its implementation. In some cases, certification may apply to other vehicles considered similar, but overall, the certification process for real-world vehicles can be resource-intensive, nevertheless requiring various types of physical vehicles to be used in testing.

[0007] In contrast, a virtual test vehicle environment can provide an alternative to testing performed on a real-world vehicle. For example, a vehicle software application may be tested according to a test plan on a virtual test vehicle that functions as a replica of a real-world vehicle, and the vehicle software application may be certified based on testing on the virtual test vehicle in the virtual environment. However, certification of vehicle software in a virtual test vehicle environment may require emulation of various real-world vehicle components in a specific network configuration similar to that of a real-world vehicle. For example, a vehicle may be equipped with various electronic control units (ECUs) that use various types of processors and implement different vehicle software environments. In some embodiments, the various ECUs of a real-world vehicle may be equipped with processors that implement various instruction set formats (ISAs), such as x86 or any other preferred ISA. The various ECUs may also be equipped with dedicated processors, such as graphics processing units (GPUs) and application-specific integrated circuits (ASICs). Furthermore, each software environment of the various ECUs may include different software environments, including various operating systems, deployed software, runtime environments, and so on. For example, a software environment can implement various operating systems, including various types of real-time operating systems (RTOs). The software environment can also implement various deployed applications, performing various functions, such as processing brake sensor signals or providing a gateway for transmitting signals to a connected server. For a virtual test vehicle environment to accurately simulate a real-world vehicle environment and for certification to be valid, the virtual test vehicle may need to conform to various performance standards of real-world vehicle ECUs, including the software environment and network configuration of the real-world vehicle's ECUs. As can be seen, providing such a real-world vehicle environment that simulates a real-world vehicle presents a challenging task.

[0008] In some embodiments, the vehicle software test environment management system may provide instructions for implementing (or orchestrating) a vECU that simulates each of the ECUs of a real-world vehicle. The vECU may be configured according to the virtual bus connection configuration of the real-world vehicle. The vehicle software test environment management system may receive or access a vehicle graph showing the ECUs of a real-world vehicle and their configurations. Based on the processor type or other configuration of the ECUs shown in the vehicle graph, the vehicle software test environment management system may determine the instance type of each virtual compute instance in order to implement the vECU. For example, a first vECU may be a virtual compute instance having an x86-based processor that simulates a first ECU that also has an x86-based processor, while a second vECU may be a virtual compute instance having a 64-bit Advanced RISC Machine (ARM) processor that simulates a second ECU that also has a 64-bit ARM processor. The vehicle software test environment management system may provision (or provide instructions for provisioning or orchestrating provisioning) virtual compute instances and implement vECUs on each virtual compute instance. In some embodiments, the vehicle software test environment management system may provision (or provide instructions for provisioning or orchestrating provisioning) virtual compute instances having different processor types and / or other dedicated compute resource configurations such as GPUs, where different processor types and / or other dedicated compute resource configurations correspond to real-world vehicle configurations. The vehicle software test environment management system may determine the virtual compute instance that best fits the ECU to be simulated in order to reduce the computing resource costs resulting from emulation.Furthermore, in some embodiments, the test environment management system may provide instructions for implementing (or orchestrating) various machine images to emulate on a vECU corresponding to each software environment of the ECU. The machine images may provide information for implementing the ECU's software environment on the vECU, including the ECU's configuration, metadata, permissions, and deployed software. The machine images may further provide an emulation layer for emulating the ECU's software environment.

[0009] A vehicle software test environment management system may determine the virtual bus connectivity configuration of a vECU based on the connectivity configuration of the ECUs shown in a vehicle graph, such as a vehicle deployment graph used to select the deployment location of software applications. For example, a vehicle may be arranged in a "zone architecture" such that ECUs in a given zone are connected to each other via a communication bus that is accessible only to ECUs within that given zone. Sensor signals in a particular zone may be restricted to software applications deployed within that zone. Also, sensor signals in a given zone may not be accessible by software applications deployed outside of a given zone, such as software applications deployed on another ECU connected to an ECU in the given zone via a different bus type, such as an Ethernet bus. Based on the connectivity configuration of the ECUs, the vehicle software test environment management system may provide instructions for configuring (or orchestrating) the vECUs in a virtual private cloud that implement the virtual bus connectivity configuration. In some embodiments, the software test environment management system may provide instructions to implement (or orchestrate) a virtual bus connectivity configuration by providing each vECU with its own private network address and using the provided private network address to configure a simulated vehicle bus interface for each vECU to send and receive messages via a virtual private cloud and simulate in-vehicle bus traffic.

[0010] In some embodiments, once the test environment is generated, the vehicle software test environment management system may provide instructions to deploy (or orchestrate the deployment of) the vehicle software to be certified and execute the test plan using recorded signals from one or more ECUs of a real-world vehicle. The vehicle software test environment management system may transmit (or provide instructions to transmit) recorded signals from one or more ECUs of a real-world vehicle to each vECU of the virtual test vehicle environment via virtual hooks of the vECUs and execute the test according to the test plan. In some embodiments, the vehicle software test environment management system may determine (or receive a determination) that the software application response obtained from the test is within a threshold range as provided in the test plan. Based on the response, the vehicle software test environment management system may send a request containing the response to the certification entity, which may initiate its own separate internal certification workflow to determine whether the vehicle application should be certified and allowed to be deployed.

[0011] Figure 1 illustrates a vehicle software test environment management system that receives a vehicle deployment graph showing various electronic control units (ECUs) of a vehicle and the vehicle's network configuration, according to several embodiments. The vehicle software test environment management system provides instructions for implementing (or orchestrates) a virtual electronic control unit (vECU) having a virtual bus connection configuration that simulates the vehicle's virtual bus connection configuration, and the configured ECU is used to perform software authentication for one or more software applications to be deployed to the vehicle.

[0012] Vehicle 100 may include a vehicle system that includes various electronic control units (ECUs) #1 110a to 4 110d, which are of different types, and ECUs #1 to 4 implement their respective software environments A112a to D112d. The ECUs may control various functions of the vehicle using a microprocessor or microcontroller. In some embodiments, the microprocessor may be a small, low-power chip that can be specifically designed for embedded systems such as ECUs. The various ECUs #1 110a to 4 110d may include different types of ECUs having different ECU configurations, such as different processor types. For example, the type of microprocessor used in an ECU may depend on the specific application and requirements of the ECU. Some ECUs may use a single microprocessor to control multiple functions, while others may use multiple microprocessors to control various different specific functions. For example, as considered above, different ECUs may include different processors depending on their type. In some embodiments, the microprocessor used in the ECU may be based on an architecture such as a 64-bit Advanced Reduced Instruction Set Computer (RISC) Machine (ARM) architecture, a PowerPC architecture, or an x86 architecture. These architectures can offer a wide range of processing power and functionality, allowing manufacturers to select the best option for a particular application. In addition to the microprocessor itself, ECUs #1 110a to 4 110d may also use various other types of processors, such as digital signal processors (DSPs) or field-programmable gate arrays (FPGAs), to handle specific functions such as signal processing or data storage, and may also use one or more graphics processing units (GPUs).

[0013] In some embodiments, each of the software environments A112a to D112d may be a software system designed to control and monitor various subsystems and components of each implemented ECU#1 110a to 4 110d. Each of the software environments A112a to D112d may consist of multiple software layers involved in a particular function (or set of functions). For example, software environment A112a may include multiple software layers, including an operating system (OS), middleware, and application software. In some embodiments, each of the software environments A112a to D112d may include different software layers. For example, software environment A112a may include real-time operating system (RTOS) software that provides deterministic and predictable timing behavior for applications with real-time requirements, while software environment C112c may include a Linux® operating system. Software environments A112a to D112d may further include different middleware components for different network protocols (e.g., CAN, LIN), file systems, and I / O drivers for different sensors and actuators. In some embodiments, software environments A112a to D112d may include different applications to perform various functions. For example, ECU#4 110d may be an engine control ECU, and software environment D112d may include applications to control fuel injection, ignition timing, and / or other engine parameters.

[0014] In some embodiments, one or more of the respective ECUs #1 110a to 4 110d (e.g., ECUs #1 110a to 3 110c) may be connected to a gateway ECU 102 using an Ethernet bus connection 122. The gateway ECU 102 may provide a central communication hub for various other ECUs #1 110a to 4 110d within the vehicle 100 and may also provide connectivity to a network 142. In some embodiments, the gateway ECU 102 may further function as a gateway between different communication protocols used by different vehicle ECUs #1 110a to 4 110d, enabling them to exchange information with each other. For example, the gateway ECU 102 may receive data from different ECUs #1 110a to 4 110d and convert the data into a format understandable by other subsystems, such as converting signals formatted according to the CAN protocol into a format understandable by another ECU using the LIN protocol. In some embodiments, ECUs #1 110a to #4 110d may be connected to each other using various communication buses. For example, ECUs #1 110a to #3 110c may be connected to gateway ECU 102 via Ethernet bus connection 122, ECUs #1 110a and #2 110b may be connected to each other via Controller Area Network (CAN) bus connection 120, and ECUs #3 110c and #4 110d may be connected to each other via Ethernet bus connection 122. Gateway ECU 102 may implement an application deployment orchestrator 104 and an over-the-air (OTA) agent 106. In some embodiments, the OTA agent 106 may receive signed, serialized data chunks of vehicle software to be deployed in the vehicle, along with a deployment plan, via network 142. In some embodiments, the OTA agent 106 may verify that prerequisite vehicle software components, such as images, are fully received and reconstructed before instructing the application deployment orchestrator 104 to process the deployment plan.In some embodiments, the deployment orchestrator 104 may send one or more deployment commands to the relevant ECUs among ECU#1 110a to 4 110d in order to process the deployment plan and deploy the software application. The one or more deployment commands may be one or more execution plans for implementing the vehicle software application in the vehicle based on the received deployment plan. Figure 1 illustrates a vehicle system of vehicle 100 having a gateway ECU 102 and ECU#1 110a to 4 110d connected to each other via Ethernet bus connection 122 and CAN bus connection 120, but this example is intended for illustrative purposes only and it should be understood that the vehicle system may have a variety of numbers / types of ECUs and various configurations.

[0015] In some embodiments, the gateway ECU 102 may transmit a vehicle deployment graph 130 of the vehicle 100 to a vehicle software test environment management system 140. The vehicle deployment graph may be a graphical representation of the various ECUs and communication networks within the vehicle 100. In some embodiments, the vehicle deployment graph 130 may represent ECUs #1 110a to 4 110d and network connections (e.g., a CAN bus connection 120 between ECU #1 110a and ECU #2 110b) as nodes on the graph, where the nodes represent ECUs #1 110a to 4 110d of the vehicle 100, and the edges represent the communication bus links between ECUs #1 110a to 4 110d. The vehicle deployment graph 130 may show the configuration and types of ECUs #1 110a to 4 110d (e.g., processor type), information about each software environment A112a to D112d (e.g., operating system type, deployed applications), and the connection configuration of the ECUs of vehicle 100. In some embodiments, instead of receiving the vehicle deployment graph 130 of vehicle 100 directly from the vehicle's gateway ECU 102, it may be received or accessed from the vehicle deployment graph repository by the vehicle software test environment management system 140. In some embodiments, the vehicle deployment graph may also be obtained from the vehicle's original equipment manufacturer (OEM) or other parts suppliers and / or maintained and updated by the vehicle software test environment management system 140 (or maintained and updated by a vehicle software deployment system (not shown in Figure 1)).

[0016] In some embodiments, the vehicle software test environment management system 140 may include a software deployment module 182, a software test module 184, a software authentication module 186, a virtual instance selection module 190, and a machine image selection module 192. The vehicle software test environment management system 140 may provide instructions for implementing (or orchestrating) a virtual test vehicle including a virtual vECU, the vECU simulating the ECUs of a real-world vehicle. For example, the vehicle software test environment management system 140 may provide instructions for implementing a virtual test vehicle 150, which includes virtual vECUs #1 160a to 4 160d, each simulating the ECUs #1 110a to 4 110d of vehicle 100 and the gateway ECU 102, and a virtual gateway ECU 152. Virtual vECUs #1 160a-4 160d may implement emulated software environments A162a-D162d, which emulate software environments A112a-D112d. Software environment emulation is further discussed in Figure 3B. The virtual instance selection module 190 may provide instructions for provisioning vECUs #1 160a-4 160d and the virtual gateway ECU 152 on their respective virtual compute instances. The machine image selection module 192 may provide instructions for implementing machine images to further implement emulated software environments of various vECUs in the virtual test vehicle 150 (e.g., emulated software environments A162a-D162d, emulated application deployment orchestrator 154, and emulated OTA agent 156) on each of the virtual compute instances on which the vECUs are implemented. The virtual instance module 190 and the machine image module 192 are further discussed in Figures 3A to 3C.

[0017] In some embodiments, the vehicle software test environment management system 140 may provide instructions for authenticating a vehicle software application (or instructions for obtaining authentication). For example, the software authentication module 186 may receive a vehicle software application having a request for authenticating the vehicle software application. The authentication request for the vehicle software application may include a deployment plan and / or a test plan in some embodiments. In some embodiments, the software deployment module 182 may deploy (or provide instructions for deploying) a vehicle software application that is required to be authenticated to one or more of vECU#1 160a-4 160d via the virtual gateway ECU 152. The software deployment module 182 and the deployment of the vehicle software application are further discussed in FIG. 2A. In some embodiments, the software test module 184 may transmit (or provide instructions for transmitting) the recorded signals of one or more sensor inputs to ECU#1 110a-4 110d of the vehicle 100 to the respective vECUs of vECU#1 160a-4 160d of the virtual test vehicle in order to implement a test plan. The software test module 184 and the testing of the vehicle software are further discussed in FIG. 2B. In some embodiments, the software authentication module may obtain response data of the software application and request / obtain authentication of the vehicle software based on the response data. The software authentication module 186 and the authentication of the vehicle software are further discussed in FIG. 2C.

[0018] FIG. 2A illustrates a graphical diagram of a vehicle software test environment management system, its various parts, and interactions for deploying a vehicle software application being authenticated to the vECUs of a virtual test vehicle having vECUs configured according to a virtual bus connection configuration, according to some embodiments.

[0019] In some embodiments, the software deployment module 182 may receive a vehicle software application that is requested to be authenticated 210. In some embodiments, the software deployment module 182 may receive a request sent by a deployment plan generator that generates multiple deployment plans for various vehicle models, and the deployment plan generator sends an authentication request as part of the deployment plan process. The deployment plan generator may request that the vehicle software test environment management system 140 for the vehicle software be authenticated on virtual replicas of various vehicle models.

[0020] Furthermore, upon receiving a request for vehicle software authentication, in some embodiments, the software deployment module 182 may provide instructions to an emulated OTA agent in the virtual test vehicle 150 via the network 120 to perform the transmission 222 of the vehicle application and deployment plan. The emulated OTA agent 156 may simulate the deployment process of the OTA agent 106 in the gateway ECU 102. For example, the emulated OTA agent 156 may generate chunked versions of the vehicle software and deployment plan to be sent to the vehicle via an external data transmission service (such as a third-party OTA service). In some embodiments, additional signing / encryption may be used by the external service used to transmit the data chunks. In some embodiments, the chunked vehicle software and deployment plan to the vehicle may be in a protocol-independent transmission format compatible with various external data transmission services / OTA services that utilize multiple different transmission protocols. In some embodiments, the emulated OTA agent 156 may verify that prerequisite vehicle software components, such as software packages, have been fully received and reconstructed, and may instruct the emulated application deployment orchestrator 154 to process the deployment plan. The emulated application deployment planner 154 may incrementally reconstruct the deployment plan and application package / container image to be used to implement the vehicle software. Similar to that of the application deployment orchestrator 104, the emulated application deployment orchestrator 154 may process the received deployment plan and send one or more deployment commands to the relevant vECUs among vECU#1 160a to 4 160d to deploy the software application 230 into the emulated software environment A162a. In some embodiments, the external data transmission service / OTA service may be further compatible with real-world vehicles, as further considered in Figure 5.In some embodiments, instead of (or in addition to) transmitting the vehicle application and deployment plan 222 to the emulated OTA agent 156, the vehicle software test environment management system 140 may transmit the vehicle software directly to (or request to transmit to) the virtual ECU #1 160a. The virtual ECU #1 160 may be equipped with software hooks for directly receiving the transmission of the vehicle application 230.

[0021] FIG. 2B illustrates a graphical diagram of a vehicle software test environment management system, its various parts, and interactions for performing a test plan for a vehicle software application on a vECU of a virtual test vehicle configured according to a virtual bus connection configuration, according to some embodiments.

[0022] In some embodiments, the vehicle software test environment management system 140 may receive a software test plan 240. The test plan may be associated with vehicle software that is required to be certified. In some embodiments, the test plan may detail one or more requirements and parameters for testing the vehicle software. In some embodiments, the test plan may be a detailed document cataloging the test strategies, objectives, and / or resources required when testing the vehicle software as part of the certification process, and may indicate a threshold range for the resulting response of the vehicle software to pass the certification test. In some embodiments, the test procedures described in various test plans may differ depending on the vehicle model indicated in the vehicle software. In some embodiments, a single test plan may include multiple procedures describing various different procedures for multiple vehicle models. In some embodiments, the vehicle software test environment management system 140 may access test plans associated with the requested vehicle software from a vehicle software test plan store 212. The vehicle software test plan store 212 may be a centralized repository containing test plans that may provide a standardized and structured approach to testing vehicle software across different vehicle models. In some embodiments, the test plan may be further selected from the vehicle software test plan store 212 based on the vehicle model for which the vehicle software is required to be certified.

[0023] In some embodiments, the vehicle software test environment management system 140 may configure (or provide instructions for configuring) a virtual test vehicle 150 to conform to the test environment described in the test plan by simulating data that would be generated by a real-world vehicle in the test conditions. The vehicle software test environment management system 140 may transmit recorded vehicle data that simulates data from recorded test plan scenarios 262. For example, a real-world vehicle may record signal data from one or more ECUs when it receives certain test conditions described in the test plan. The software environments A162a to D162d of vECU#1 160a to 4 160d may each include recorded vehicle data interfaces 250a to 250d, each providing a virtual hook for the recorded ECU signal data received by each virtual ECU#1 160a to 4 160d. In some embodiments, the vehicle software test environment management system 140 may apply only a portion of the received and recorded signals corresponding to a first set of temporal moments to one or more of the vECUs #1 160a to #4 160d via a virtual hook.

[0024] In some embodiments, the vehicle software test environment management system 140 may modify a second portion of the recorded signal corresponding to a second set of temporal moments based on sensor outputs received from the vECU, and apply the modified second portion of the recorded signal to each of one or more of the vECUs via virtual hooks of the second set of temporal moments. For example, when each vECU #1 160a to 4 160d receives recorded ECU signal data, the vehicle software 230 may generate a response. The vehicle software test environment management system 140 may receive the vehicle software response as part of the execution of the test plan described above 260. Based on the received response, the vehicle software test environment management system 140 may modify the recorded vehicle data, taking into account the effects of the vehicle software 230, in order to update the recorded data, taking into account the effects of the software being tested, as indicated in the response. The vehicle software test environment management system 140 may transmit the modified recorded vehicle data based on the vehicle software data instead of the unmodified recorded data 264. For example, based on the vehicle software response 260 received by the automatic braking system deployed in virtual ECU #1 160a, the vehicle software test environment management system 140 may modify the recorded speed data to be corrected, taking into account earlier deceleration resulting from the automatic braking system initiating braking earlier. The modified vehicle data is then transmitted back as modified ECU signal data and can be received by the respective virtual ECUs #1 160a to 4 160d.

[0025] In some embodiments, the response of a software application during the execution of a test plan may be non-deterministic. The vehicle software test environment management system 140 may execute the test plan multiple times to obtain a range of execution levels for the vehicle software application, in response to an indication that the response of the software application during the execution of the test plan is non-deterministic. Based on the range of execution levels, the vehicle software test environment management system 140 may determine whether a threshold range for authentication of the non-deterministic software application is met.

[0026] Figure 2C illustrates a graphical diagram of a vehicle software test environment management system, its various parts, and their interactions, for providing the results of conducting a test plan and requesting / obtaining certification based on the received results, according to several embodiments.

[0027] In some embodiments, the vehicle software test environment management system 140 may receive a vehicle software response from the execution of a test plan 270. The vehicle response 270 received from the execution of a test plan may, in some embodiments, be a subsequently generated vehicle response based on the transmission of modified vehicle software data 264 (as considered above in Figure 2B). In some embodiments, after receiving the vehicle software response from the execution of a test plan 270, the vehicle software test environment management system 140 may request and / or obtain certification based on the test results of the test plan, as considered above in Figures 2A and 2B 272. In some embodiments, the deployment plan generator 104 may request certification of the vehicle software 230 by sending a request for certification to a certification destination, the request may include one or more of the deployment plan to be used to deploy the vehicle software 230, the vehicle software 230 itself, vehicle software dependencies, vehicle metadata, and / or the results of the test plan. The certification destination may initiate its own separate internal certification workflow to determine whether the deployment plan should be certified and permitted to be used to deploy a software application or multiple software applications. In some embodiments, the certifying body may reject the certification request. In other embodiments, the certifying body may approve the certification request and send an indication of certification approval to the vehicle software test environment management system 140. In some embodiments, the environment management system 140 may initiate its own certification workflow to determine whether the deployment plan should be certified and approved.

[0028] Figure 3A illustrates a more detailed diagram of a vehicle software test environment management system in several embodiments, which retrieves available virtual instance types and machine images and determines the virtual instance types and machine images that need to be used to simulate each ECU of the vehicle using each vECU.

[0029] In some embodiments, the vehicle software test environment management system 140 may obtain information about available virtual instance types from the virtual compute instance catalog 310 320. The virtual compute instance catalog 310 may include one or more virtual compute instance types (e.g., virtual instance type A312a and virtual instance type B312b) that may be available for provisioning to implement a vECU. For example, a virtual compute instance type may be a specific type of virtual machine that operates on a cloud computing infrastructure with a specific physical computer system and provides a specific configuration of computing resources such as CPU, memory, storage, and network connectivity. For example, virtual compute instance type A312a may be a virtual machine that operates on a physical computer system having four 64-bit ARM processors. In some embodiments, a virtual compute instance type may indicate an instance type that operates on a host operating system and shares physical resources with other virtual machines. In some embodiments, a compute instance type may be a bare-metal compute instance that operates directly on physical hardware without requiring the installation of a hypervisor or operating system (e.g., a non-virtualized compute instance). As discussed above, compute instance types may include various compute instance configurations, including having various processor types, or other compute resource configurations, including various numbers of CPUs, amounts of memory, and various types and / or sizes of storage.

[0030] In some embodiments, the vehicle software test environment management system 140 may determine (or receive a determination) the virtual instance type that should be used to implement each virtual compute instance of vECU#1 160a to 160d 322. For example, based on an unfolded graph showing that ECU#1 110a has a type "A" ECU configuration, the vehicle software test environment management system 140 may determine that the virtual compute instance of vECU#1 314a (to be used to simulate ECU#1 110a) should be equipped with processor type A316a. In some embodiments, the vehicle software test environment management system 140 may receive a command to determine virtual instance type B312b for implementing vECU#2 314b. In some embodiments, the determination of the virtual instance type may be based on emulation cost, as further considered in Figure 3B.

[0031] In some embodiments, the vehicle software test environment management system 140 may further obtain information about available machine images from the machine image catalog 330 324. The machine image catalog 330 may include various machine images (e.g., machine image A332a, machine image B332b). In some embodiments, a machine image may be a snapshot of a virtual machine or server at a specific point in time, which can be used to create a new instance that is identical (or nearly identical) to the original instance with a particular configuration implemented, such as a particular vECU configuration. A machine image may include an image of the operating system, software applications, data, and other necessary components required to run a particular workload or application on the ECU. For example, machine image A332a may be an image of software environment A112a, which may be a snapshot representing the state and / or configuration of ECU#1 110a at a particular moment in time. In some embodiments, machine image A332a may be used to launch a virtual compute instance that has all the same (or similar) configuration, software, and data as software environment A112a. Similar to determining the virtual instance type, in some embodiments, the vehicle software test environment management system 140 may determine (or receive such determination) a machine image to implement each of the software environments of vECU#1 160a to 4 160d 328. For example, based on an expansion graph indicating that software environment A112a has a Linux operating system (and / or other software environment configurations such as the type of middleware or applications installed), the vehicle software test environment management system 140 may determine that machine image #1 340a, which has an emulated software environment A162a, should be implemented on the virtual compute instance of vECU#1 314a. In some embodiments, the vehicle software test environment management system 140 may receive a command determining that machine image #2 should be implemented on the virtual compute instance of vECU#2 314b.In some embodiments, the vehicle software test environment management system 140 may first determine a machine image for a vECU, and then, based on the determined machine image, determine the virtual compute instance type to use for the vECU. For example, the vehicle software test environment management system 140 may determine that machine image A332a is used to simulate vECU#1, and then, based on metadata associated with machine image A332, determine that virtual compute instance type A312a should be used, in which case the metadata indicates, for example, that virtual compute instance type A312a should be used to implement machine image A332. In some embodiments, the vehicle software test environment management system 140 may determine which virtual compute instance type to use based on an analysis of the emulation cost for running the machine image on each type of virtual compute instance.

[0032] The vehicle software test environment management system 140 may further manage the machine image catalog. In some embodiments, the vehicle software test environment management system 140 may add machine images, modify existing machine images, or delete machine images from the machine image catalog 330. For example, the vehicle software test environment management system 140 may receive ECU information about another ECU from a vehicle manufacturer or vehicle parts manufacturer, and create a new machine image that simulates the software environment of the ECU based on the received ECU information. In response to a request to create a new machine image (or to the receipt of a new machine image from a vehicle manufacturer or vehicle parts manufacturer), the vehicle software test environment management system 140 may add the new machine image to the machine image catalog 330.

[0033] Figure 3B illustrates a more detailed diagram of a vehicle software test environment management system that provisions virtual instances, each having a machine image and a virtual instance type selected to simulate the vehicle's ECU based on a vehicle deployment graph, according to several embodiments.

[0034] The vehicle software test environment management system 140 may provision (or provide instructions for provisioning) virtual compute instances determined in the virtual private cloud (as shown in Figure 3A). For example, the vehicle software test environment management system 140 may provision and boot virtual compute instances using each machine image on the virtual private cloud 350 352. The virtual private cloud 350 may enable the virtual test vehicle 150 and various virtual compute instances 314a to 314e to be isolated from other networks, including the internet, and may provide a secure and isolated environment. The vehicle software test environment management system 140 may provision (or provide instructions for provisioning) virtual compute instances 314a to 314e having various processor types A316a to E316e by allocating a portion of underlying physical resources such as CPU, memory, and storage to create a virtualization environment in which virtual compute instances can be provisioned. Additionally, the software test environment management system 140 may provision (or provide instructions to provision) virtual compute instances to boot up using the respective machine images determined for each virtual instance (e.g., machine images #1 316a to #5 316e), as considered in Figure 3A.

[0035] In some embodiments, the emulated software environments A162a to E162e may each emulate a matching software environment by providing an abstraction layer that can enable the vehicle software to run on different hardware platforms without requiring modifications. For example, the emulator for emulated software environment A162a may run on a virtual compute instance of vECU#1 314a and can translate vehicle software instructions into the native instruction set of vECU#1 314a. In some embodiments, the instruction translation may enable the software to run on vECU#1 314a as if it were running natively on the target platform of the real-world ECU corresponding to vECU#1 314a. In some embodiments, the emulation may be slower to implement because it involves additional processing overhead compared to running the software natively on the target platform. The vehicle software test environment management system 140 may decide to utilize certain virtual compute instance types to reduce implementation losses due to the emulation. In some embodiments, the vehicle software test environment management system 140 may further implement an emulated software environment that dynamically translates instructions to the native instruction set of the target platform using just-in-time (JIT) compilation techniques. The emulated software environments A162a to E162e may further include simulated vehicle bus interfaces 360a to 360e that can be used to simulate the network connectivity of ECUs within the vECU environment.

[0036] Figure 3C illustrates a more detailed diagram of a vehicle software test environment management system, comprising a virtual instance type and machine image, each selected to simulate the vehicle's ECUs in the vehicle's network configuration according to a vehicle deployment graph, in several embodiments, and constituting a simulated vehicle bus interface of virtual instances.

[0037] In some embodiments, the vehicle software test environment management system 140 may configure routing tables using simulated vehicle bus interfaces 360a-360c to replicate the network connectivity configuration of a real-world vehicle, and control IP address ranges, security groups, and network access policies. In some embodiments, the virtual private cloud 350 may be customized with IP address ranges, subnets, and routing tables, and may represent a virtual environment that simulates in-vehicle communications of a vehicle 100 (or a subset of the vehicle's ECUs). As discussed above in Figure 3B, various virtual compute instances 314a-314e may be isolated from other networks using the virtual private cloud 350. For example, the vehicle software test environment management system 140 may configure the simulated vehicle bus interfaces 360a-360c to simulate in-vehicle traffic by controlling routing tables to enable communication with vECUs according to the vehicle deployment graph 130. The ECU signals discussed above in Figure 1 can use various bus protocols, including CAN (Controller Area Network), LIN (Local Interconnect Network), and FlexRay. To simulate vehicle-based communication within a virtual test vehicle 150, simulated vehicle bus interfaces 360a-360c can convert one type of protocol to another and transmit the converted signals received from one type of protocol to another via the virtual private cloud 350. For example, simulated vehicle bus interface 360a may be configured to connect to a virtual compute instance for gateway vECU 314e using the Ethernet protocol and to connect to a virtual compute instance for vECU #2 314b using the CAN bus protocol.The simulated vehicle bus interface 360a can convert CAN bus protocol-formatted signals, which should be sent to the virtual compute instance for vECU#2 314b, to the Ethernet protocol, enabling transmission over the virtual private cloud 350 (or another network, such as the Internet, to which the virtual compute instance is connected). The simulated vehicle bus interface 360b can convert the converted messages back to CAN bus protocol-formatted signals, simulating the in-vehicle traffic of vehicle 100. As considered in Figure 1, the virtual bus connectivity configuration of the vECU can be based on the vehicle deployment graph 130. Thus, the virtual bus connectivity configuration within the virtual private cloud can emulate the network environment of a real-world vehicle implemented within the virtual private cloud by the virtual test vehicle.

[0038] Figure 4 illustrates a more detailed diagram of a vehicle software test environment management system that provides vECUs implemented in a virtual compute instance to simulate a subset of the vehicle's ECUs for partial simulation, according to several embodiments.

[0039] In some embodiments, simulating all ECUs in a vehicle via a vECU may not be necessary or practical. For example, in a vehicle with dozens of ECUs, the hardware resources required to simulate all ECUs may be very costly or unnecessarily complex to manage for the certification task to be performed. Additionally, in some cases, only certain ECUs may be involved in the development or testing of vehicle software, and it may not be necessary to simulate all of them. In some embodiments, the virtual test vehicle 150 may simulate only a subset of the set of ECUs. To simulate a subset of ECUs, the vehicle software test environment management system 140 may identify which ECUs should be added to the virtual test vehicle 150 for testing the vehicle software. In some embodiments, the vehicle software test environment management system 140 may include only the ECUs that are important for the implementation of the vehicle software. For example, the vehicle software test environment management system 140 may receive automatic braking software as a vehicle software application to be certified 210. The vehicle software test environment management system 140 may further receive a software test plan for the automatic braking software 240. Based on the received vehicle software and / or the received test plan, the vehicle software test environment management system 140 may determine that ECU#2 110b and ECU#4 110d, which are directed to the climate control system and infotainment system, should be excluded within the virtual test vehicle 150. Once the relevant ECUs are identified, the vehicle software test environment management system 140 may selectively provision (or provide instructions for provisioning) vECUs corresponding to the relevant subset of the larger set of ECUs in a given vehicle, as communicated in the vehicle deployment graph 130.

[0040] Figure 5 illustrates a more detailed diagram of a vehicle software test environment management system that deploys a vehicle software application to be certified to a virtual test vehicle using an emulated over-the-air (OTA) agent of a virtual test vehicle, according to several embodiments.

[0041] In some embodiments, the vehicle software test environment management system 140 may receive a vehicle software application 210 to be authenticated and deploy (or provide instructions for deployment) the vehicle software application to the virtual test vehicle using an emulated over-the-air (OTA) agent of the virtual test vehicle. In some embodiments, the vehicle software application 230 may be deployed to both the virtual test vehicle 150 and real-world vehicles such as vehicle 100 and vehicle 502 via an emulated OTA agent 156. In some embodiments, the vehicle software test environment management system 140 may provide instructions for transmission of the vehicle application deployment plan to both the emulated OTA agent and the vehicle OTA agent 500. For example, the vehicle application may be packaged in a format that can be sent to both the OTA agent in the ECU and the emulated OTA agent in the virtualized ECU. This can be a binary file, a compressed archive, or another preferred format. In some embodiments, a communication channel may need to be established between a sender (e.g., a vehicle software test environment management system 140) and two receivers (e.g., an OTA agent 106 in a gateway ECU 102 and an emulated OTA agent 156 in a virtualization ECU 152). The vehicle software test environment management system 140 may configure the network connection, establish security protocols, and ensure that the sender has the necessary access privileges to communicate with both agents. Once the communication channels to each agent are established, the vehicle application can be sent to both the OTA agent 106 in the gateway ECU 102 and the emulated OTA agent 156 in the virtualization ECU 152. In some embodiments, the transmission may involve sending the vehicle software as a single package or dividing it into smaller parts to optimize the transmission and ensure reliability.This process ensures that any loss or degradation in the transmission of vehicle software packages to vehicles using OTA agents is also taken into account during testing via virtual vehicles.

[0042] Figure 6 illustrates a more detailed diagram of a vehicle software test environment management system that requests / obtains certification of a vehicle software application based on test results from a virtual test vehicle and / or the results of tests conducted using one or more real-world vehicles, according to several embodiments.

[0043] In some embodiments, the vehicle software test environment management system 140 may determine that it is necessary to test the software application both on a virtual ECU and on a real ECU. For example, the vehicle software test environment management system 140 may determine that testing on a virtual ECU is more cost-effective and eliminates the need for physical hardware, but that testing on a real-world ECU is also necessary to test the compatibility of the vehicle software with the specific hardware and operating conditions of the vehicle 100. In some embodiments, real-world testing can identify problems related to physical hardware, such as communication problems or hardware failures. In some embodiments, virtual testing can complement real-world testing. For example, a limited number of real-world tests may be performed to identify problems that cannot be replicated in virtual testing, and a larger number of test runs may be performed using a virtual vehicle to explore software responses for conditions that can be reasonably well replicated in a virtual environment. In some embodiments, the vehicle software test environment management system 140 may perform testing not only on real-world vehicles 100 and 502 but also on a virtual test vehicle 150. The vehicle software test environment management system 140 may receive vehicle responses from the virtual test vehicle 150 from the execution of the test plan 270, as well as vehicle responses from the execution of the test plan for a vehicle (e.g., a real-world vehicle) 600. Based on the test results from the virtual test vehicle 150 and the vehicle, the vehicle software test environment management system 140 may request and / or obtain certification 602. The vehicle software test environment management system 140 may send a request to the certification authority that includes test information from both the virtual test vehicle 150 and the real-world vehicles 100 and 502, as discussed above in Figure 2C.

[0044] Figure 7 illustrates a flowchart of operations performed by a vehicle software test environment management system to provide a virtual electronic control unit (vECU) with a virtual bus connection configuration, based on a vehicle deployment graph that may be used to simulate a vehicle according to several embodiments.

[0045] In block 710, the vehicle software test environment management system receives or accesses a vehicle deployment graph of a given vehicle, which includes the configuration of each of the electronic control units (ECUs) of a given vehicle and the connection configuration of the ECUs of a given vehicle. In some embodiments, the vehicle deployment graph may be a graphical representation of the various ECUs and communication networks within the vehicle, where nodes on the graph represent ECUs and edges represent communication bus links between ECUs, as considered in Figure 1.

[0046] In block 720, the vehicle software test environment management system determines the respective configurations for the virtual electronic control units (vECUs) to be used to simulate each of the ECUs in the vehicle deployment graph. In some embodiments, the respective configurations determined by the vehicle software test environment management system may indicate the type of processor to be used for the vECU, as considered in Figure 3A.

[0047] In block 730, the vehicle software test environment management system determines the virtual bus connection configuration for the vECU based on the ECU connection configuration shown in the vehicle deployment graph. In some embodiments, the virtual bus connection configuration may be determined to simulate in-vehicle traffic of multiple bus protocols, as considered in Figure 1.

[0048] In block 740, the vehicle software test environment management system provides vECUs having the respective determined configurations, corresponding to each ECU in the vehicle deployment graph, which is implemented on a virtual compute instance configured based on the respective determined configurations for the vECUs. In some embodiments, only a subset of ECUs may be synchronized on the vECUs, as considered in Figure 4.

[0049] In block 750, the vehicle software test environment management system configures the vECU with a network configuration that implements the determined virtual bus connection configuration. In some embodiments, the virtual bus connection configuration may be implemented via a simulated vehicle bus interface that converts and converts signals between various protocols, as considered in Figure 3C.

[0050] Figure 8 illustrates flowcharts of operations performed by a vehicle software test environment management system in several embodiments for deploying a software application to a vECU and certifying the software application based on the results of the received test plan.

[0051] In block 810, the vehicle software test environment management system deploys the software application to at least one of the provided vECUs. In some embodiments, the vehicle software test environment management system may transmit signed serialized data chunks of the vehicle software to be deployed in the vehicle, along with a deployment plan for deploying the software application, as considered in Figure 2A.

[0052] In block 820, the vehicle software test environment management system authenticates a software application based on the results of a test plan received when it is performed on at least one of the provided vECUs, and at least one of the provided vECUs is included in a network configuration that has each of the determined configurations and implements the determined virtual bus connection configuration. In some embodiments, the vehicle software test environment management system may send a request including a response to the authentication destination, which may initiate its own separate internal authentication workflow to determine whether the vehicle application should be authenticated and allowed to be deployed, as considered in Figure 2C.

[0053] Figure 9 illustrates flowcharts illustrating the operations performed by a vehicle software test environment management system for adding new machine images to a machine image catalog and selecting each machine image from the machine image catalog to be implemented on the vECU, according to several embodiments.

[0054] In block 910, the vehicle software test environment management system receives ECU information about the ECU from the vehicle manufacturer or vehicle parts manufacturer. In some embodiments, the ECU information may be information about the software environment of the ECU, as discussed in Figures 1 and 3A.

[0055] In block 920, the vehicle software test environment management system creates a new machine image based on the received ECU information. In some embodiments, the vehicle software test environment management system creates a new machine image to emulate the software environment of the ECU, as considered in Figures 3A and 3B.

[0056] In block 930, the vehicle software test environment management system adds a new machine image to the machine image catalog. In some embodiments, the vehicle software test environment management system may modify or delete existing machine images in the machine image catalog, as considered in Figure 3A.

[0057] In block 940, the vehicle software test environment management system selects a machine image for each vECU from a machine image catalog based on the received ECU information. In some embodiments, the vehicle software test environment management system selects a machine based on the cost incurred in emulation resource costs from translating the instruction set of the vehicle software to the native instruction set of another compute environment.

[0058] Exemplary computer system Any of the various computer systems may be configured to implement processes associated with the vehicle software test environment management system or any other component of the drawings above. For example, Figure 10 illustrates a block diagram illustrating an exemplary computer system that implements some or all of the techniques described herein in several embodiments. In various embodiments, the vehicle software test environment management system, the provider network used to deploy the vehicle software test environment management system and other cloud services, the operating system of a vehicle or device, or any other component of Figures 1 to 9 above may each include one or more computer systems 1000, such as those illustrated in Figure 10.

[0059] In the illustrated embodiments, the computer system 1000 includes one or more processors 1010 coupled to system memory 1020 via an input / output (I / O) interface 1030. The computer system 1000 further includes a network interface 1040 coupled to the I / O interface 1030. In some embodiments, the computer system 1000 may be an example of a server implementing something that provides enterprise logic or downloadable applications, and in other embodiments, the server may include more, fewer, or different elements than the computer system 1000.

[0060] In various embodiments, the computing device 1000 may be a uniprocessor system including one processor, or a multiprocessor system including several (e.g., two, four, eight, or another preferred number) processors 1010a to 1010n. Processors 1010a to 1010n may include any preferred processor capable of executing instructions. For example, in various embodiments, processors 1010a to 1010n may be processors implementing any of the various instruction set formats (ISAs) such as x86, PowerPC, SPARC, or MIPS ISA, or any other preferred ISA. In some embodiments, processors 1010a to 1010n may include dedicated processors such as graphics processing units (GPUs) or application-specific integrated circuits (ASICs). In a multiprocessor system, each of processors 1010a to 1010n may, but not necessarily, implement the same ISA.

[0061] The system memory 1020 may be configured to store program instructions and data accessible by processors 1010a to 1010n. In various embodiments, the system memory 1020 may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory. In the illustrated embodiments, it is shown that program instructions and data implementing one or more desired functions, such as these methods, techniques, and data described above, are stored in the system memory 1020 as code (e.g., program instructions) 1025 and data storage 1035.

[0062] In one embodiment, the I / O interface 1030 may be configured to coordinate I / O traffic between processors 1010a-1010n, including a network interface 1040 or other peripheral interfaces, system memory 1020, and any peripheral devices within the device. In some embodiments, the I / O interface 1030 may perform any necessary protocols, timing, or other data conversions to convert data signals from one component (e.g., system memory 1020) into a format suitable for use by another component (e.g., processor 1010). In some embodiments, the I / O interface 1030 may include support for devices mounted via various types of peripheral buses, such as a PCI bus standard or a variation of the Universal Serial Bus (USB) standard. In some embodiments, the I / O interface 1030 may include support for devices mounted via an automotive CAN bus, etc. In some embodiments, the functionality of the I / O interface 1030 may be divided into two or more separate components, such as a northbridge and a southbridge. Furthermore, in some embodiments, some or all of the functions of the I / O interface 1030, such as the interface to the system memory 1020, may be directly incorporated into the processors 1010a to 1010n.

[0063] In some embodiments, the network interface 1040 may be coupled with the I / O interface 1030, as well as one or more input / output devices 1050, such as a cursor control device 1060, a keyboard 1070, and a display 1080. In some cases, embodiments are intended to be implemented using a single instance of computer system 1000, while in other embodiments, multiple such computer systems, or multiple nodes constituting computer system 1000, may be configured to host different parts or instance program instructions, as described above for various embodiments. For example, in one embodiment, some elements of a program instruction may be implemented via one or more nodes of computer system 1000, separate from those nodes that implement other elements.

[0064] The network interface 1040 may be configured to allow data to be exchanged between the computing device 1000 and other devices associated with the network. In various embodiments, the network interface 1040 may support communication over any suitable wired or wireless general-purpose data network, such as Ethernet networks, cellular networks, Bluetooth networks, Wi-Fi networks, or ultra-broadband networks. Additionally, the network interface 1040 may support communication over telecommunications / telephone networks, such as analog voice networks or digital fiber optic networks, over storage area networks, such as Fibre Channel SANs, or over any other suitable type of network and / or protocol.

[0065] In some embodiments, system memory 1020 may be an embodiment of a computer-readable (e.g., computer-accessible) medium configured to store program instructions and data as described above for implementing the corresponding methods, systems, and apparatus embodiments. However, in other embodiments, program instructions and / or data may be received, transmitted, or stored on different types of computer-readable media. Generally speaking, computer-readable media may include non-temporary storage media or memory media such as magnetic or optical media (e.g., disks or DVDs / CDs) coupled to computing device 1000 via I / O interface 1030. One or more non-temporary computer-readable storage media may also include any volatile or non-volatile media such as RAM (e.g., SDRAM, DDR SDRAM, RDRAM, SRAM, etc.), ROM, etc., which may be included in some embodiments of computing device 1000 as system memory 1020 or another type of memory. Furthermore, the computer-readable medium may include transmission media or signals such as electrical signals, electromagnetic signals, or digital signals that are transmitted via communication media such as networks and / or wireless links, which may be implemented via the network interface 1040. Some or all of a plurality of computing devices, such as those illustrated in Figure 10, may be used to implement the functions described in various embodiments; for example, software components running on various different devices and servers may work together to provide the functions. In some embodiments, parts of the described functions may be implemented using storage devices, network devices, or various types of computer systems. As used herein, the terms “computing device” and “ECU” refer to, but are not limited to, at least all of these types of devices.

[0066] The various methods illustrated in the figures and described herein represent illustrative embodiments of the method. The method may be carried out manually, in software, in hardware, or in combination thereof. The order of any method may be changed, and various elements may be added, rearranged, combined, omitted, modified, etc. For example, in one embodiment, the method may be carried out by a computer system including a processor that executes program instructions stored on a computer-readable storage medium coupled to the processor. The program instructions may be configured to implement the functions described herein (e.g., functions such as data transfer tools, various services, databases, devices and / or other communication devices).

[0067] As will be apparent to those skilled in the art, various modifications and changes can be made that have advantages of the present disclosure. It is intended to encompass all such modifications and changes, and therefore the above description is intended to be considered illustrative rather than restrictive.

[0068] Various embodiments may further include receiving, transmitting, or storing instructions and / or data implemented in accordance with the foregoing description on a computer-accessible medium. Generally speaking, computer-accessible mediums may include storage or memory mediums such as magnetic or optical media (e.g., disks or DVD / CD-ROMs), volatile or non-volatile media such as RAM (e.g., SDRAM, DDR, RDRAM, SRAM, etc.), ROM, and transmission or signal media such as electrical signals, electromagnetic signals, or digital signals transmitted over communication media such as networks and / or wireless links.

[0069] Embodiments of this disclosure may be described in consideration of the following provisions. Clause 1. A system, A system comprising one or more computing devices configured to implement a vehicle software test environment, wherein the vehicle software test environment is Upon receiving the vehicle software application to be certified, To receive or access a test plan for certifying a software application for use in a given vehicle, A vehicle deployment graph of a given vehicle, which includes the configuration of one or more electronic control units (ECUs) of the given vehicle and the connection configuration of the ECUs of the given vehicle, is received or accessed. Determine the configuration of each virtual electronic control unit (vECU) that should be used to simulate each of the ECUs in the vehicle deployment graph. Based on the ECU connection configuration shown in the vehicle deployment graph, the virtual bus connection configuration for the vECU is determined. A computing resource configured as a vECU having a determined configuration corresponding to each ECU of the vehicle deployment graph, implemented using virtual compute instances configured based on the determined configuration of each vECU, and the computing resource is further configured with a network configuration that implements the determined virtual bus connection configuration, providing a computing resource. Deploy the software application to at least one of the provided computing resources configured as a vECU, A system configured to execute received test plans in order to authenticate a software application for use in a given vehicle, wherein the received test plans are executed using provided computing resources configured as vECUs configured in a virtual bus configuration. Clause 2. In order to implement the received test plan, the vehicle software test environment shall Receive recorded signals from one or more ECUs of a given vehicle, or a vehicle having a similar configuration to the given vehicle. The received and recorded signals are applied to one or more of the vECUs via virtual hooks in one or more of the vECUs, The system described in Clause 1 is further configured to determine whether the response resulting from the software application is within a threshold range to be provided to the test plan. Clause 3. In order to implement the received test plan, the vehicle software test environment shall Only a portion of the received and recorded signal corresponding to the first set of temporal moments is applied to one or more of the vECUs via a virtual hook, Based on the application of the first set of temporal moments of the received recorded signal, the sensor output from the vECU is received, Based on the sensor output received from the vECU, a second portion of the recorded signal corresponding to a second set of temporal moments is modified. The system described in Clause 2, configured to apply a modified second portion of the recorded signal to one or more vECUs, respectively, via a virtual hook to a second set of temporal moments. Clause 4. The vehicle software test environment is During the execution of the test plan, the responses of the software application are recorded. The system is further configured to execute the received test plan multiple times using the same signals as the received recorded signals, the software application is nondeterministic, the multiple executions of the test plan indicate the range of execution levels of the nondeterministic software application, and the threshold range for authentication of the nondeterministic software application is based on the range of execution levels, as described in any of Clauses 1 to 3. Article 5. Method, Receiving or accessing a vehicle deployment graph of a given vehicle, which includes the configuration of one or more electronic control units (ECUs) of the given vehicle and the connection configuration of the ECUs of the given vehicle. To determine the respective configurations for one or more virtual electronic control units (vECUs) that should be used to simulate one or more of the ECUs in the vehicle deployment graph, A method comprising providing one or more vECUs having determined configurations for use in authenticating a software application to be implemented in a given vehicle, wherein the provided one or more vECUs are implemented on one or more virtual compute instances configured based on the determined configurations for the one or more vECUs. Clause 6. Determining the respective configuration for one or more vECUs, This involves determining the machine image for one or more vECUs, and determining whether each machine image emulates the software environment of each ECU among the ECUs shown in the vehicle deployment graph. The method according to Clause 5, which includes determining the instance type of each of the one or more virtual compute instances used to implement the vECU. Clause 7. Determining the instance type of each of the one or more virtual compute instances used to implement vECU, Based on the processor type of one or more ECUs shown in the vehicle deployment graph, the instance type of each ECU is determined, The method according to Clause 6, which includes determining each instance type based on each machine image determined to be used for one or more vECUs. Clause 8. Provide one or more vECUs, each having the determined configuration and implemented in one or more virtual compute instances. Based on the processor type of each ECU shown in the vehicle deployment graph, provision one or more virtual compute instances of the determined instance type, The method described in clause 6 or 7, which includes booting one or more virtual compute instances using each determined machine image. Clause 9. Determining the machine image for one or more vECUs, The method of any of clauses 6 to 8, which includes selecting a machine image for each ECU from a machine image catalog containing machine images for ECUs used in multiple types of vehicles. Clause 10. Receiving ECU information about the ECU from the vehicle manufacturer or vehicle parts manufacturer, Based on the received ECU information, a new machine image is created, The method described in any of clauses 6 to 9, further including adding new machine images to the machine image catalog. Clause 11. Determine the virtual bus connection configuration for the vECU based on the ECU connection configuration shown in the vehicle deployment graph, The method of any one of the clauses 5 to 10, further comprising configuring the provided vECU in the network configuration to implement the determined virtual bus connection configuration within the virtual private cloud. Clause 12. Configuring the vECU provided in accordance with the virtual bus connection configuration, Providing each vECU implemented on a virtual private cloud with its own private network address, The method according to Clause 11, which includes configuring a simulated vehicle bus interface for each vECU to simulate in-vehicle bus traffic by sending and receiving messages via a virtual private cloud using a provided private network address. Clause 13. Receiving the vehicle software application to be certified, Receiving or accessing a test plan for certifying a software application for use in a given vehicle, Deploy the software application to at least one of the provided vECUs, The method according to Clause 12, further comprising performing a received test plan to certify a software application for use in a given vehicle, wherein the received test plan is performed using provided computing resources configured as a vECU configured in a virtual bus configuration. Clause 14. The method of Clause 13, further comprising conducting a second test plan performed on a real-world vehicle to certify a software application for use in a given vehicle. Clause 15. The method of any one of Clauses 5 to 14, further comprising deploying a software application to at least one of the one or more vECUs provided via an over-the-air (OTA) service, wherein the OTA service is configured to deploy the software application to one or more vECUs and ECUs of a given vehicle. Clause 16. To receive a test plan for certifying a software application for use in a given vehicle, This includes, and further includes, implementing the received test plan. Receiving recorded signals from one or more ECUs of a given vehicle, or a vehicle having a similar configuration to a given vehicle, The received and recorded signals are applied to each of the vECUs via one or more virtual hooks among the vECUs, The method of any of clauses 5 to 15, which includes determining whether the response resulting from the software application is within a threshold range as provided for the test plan. Article 17. The received test plan shall be implemented. This includes, and is carried out, the implementation of the received test plan. The method according to Clause 16, which includes correcting recorded signals of one or more ECUs based on downstream changes from a software application. Clause 18. Recording the responses of the software application, The method of Clause 16, further comprising performing a received test plan to certify a software application for use in a given vehicle, wherein the received test plan is performed multiple times using received recorded signals, using provided computing resources configured as a vECU configured in a virtual bus configuration, and the software application is non-deterministic. Clause 19. One or more non-temporary computer-readable storage media for storing program instructions, wherein when a program instruction is executed on or across one or more processors, one or more processors... A vehicle deployment graph of a given vehicle, which includes the configuration of one or more electronic control units (ECUs) of the given vehicle and the connection configuration of the ECUs of the given vehicle, is received or accessed. Determine the respective configurations for one or more virtual electronic control units (vECUs) that should be used to simulate one or more of the ECUs in the vehicle deployment graph. One or more non-temporary computer-readable storage media, which are used to authenticate software applications to be implemented in a given vehicle, and which provide one or more vECUs, each having a determined configuration, and which are implemented on one or more virtual compute instances configured based on the determined configurations for one or more vECUs. Clause 20. When a program instruction is executed, it is performed on one or more processors. The vehicle software application to be authenticated is received, To receive a test plan for certifying a software application for use in a given vehicle, Deploy the software application to at least one of the provided vECUs. One or more non-temporary computer-readable storage media as described in Clause 19, wherein the software application is authenticated based on the performance of a test plan received by at least one of the provided vECUs, and at least one of the provided vECUs is contained within a network configuration having each of the determined configurations and implementing the determined virtual bus connection configuration.

Claims

1. It is a system, A system comprising one or more computing devices configured to implement a vehicle software test environment, wherein the vehicle software test environment is A vehicle deployment graph of a given vehicle, which includes the configuration of one or more electronic control units (ECUs) of the given vehicle and the connection configuration of the ECUs of the given vehicle, is received or accessed. Determine the respective configurations for one or more virtual electronic control units (vECUs) to be used to simulate one or more of the ECUs in the vehicle deployment graph, A system configured to provide one or more vECUs having the respective determined configurations for use in authenticating a software application to be implemented in a given vehicle, wherein the provided one or more vECUs are implemented on one or more virtual compute instances configured based on the respective determined configurations for the one or more vECUs.

2. The aforementioned vehicle software testing environment service, To receive or access a test plan for authenticating the software application for use in a given vehicle, The software application is deployed to at least one of the multiple provided computing resources, which are configured as one or more vECUs. The system according to claim 1, further configured to perform the received test plan in order to authenticate the software application for use in the given vehicle, wherein the received test plan is performed using the provided computing resources configured as the vECU configured in the virtual bus configuration.

3. The aforementioned vehicle software testing environment service, The system is further configured to determine the virtual bus connection configuration for the vECU based on the connection configuration of the ECU shown in the vehicle deployment graph, The system according to claim 1 or 2, wherein the provided computing resources are further configured in a network configuration that implements the determined virtual bus connection configuration.

4. In order to carry out the received test plan, the vehicle software test environment is configured Receiving recorded signals from one or more ECUs of the given vehicle, or a vehicle having a similar configuration to the given vehicle, The received recorded signal is applied to each of the one or more vECUs via one or more virtual hooks among the vECUs. The system according to claim 2 or 3, further configured to determine whether the response resulting from the software application is within a threshold range to be provided to the test plan.

5. In order to carry out the received test plan, the vehicle software test environment is configured Only a portion of the received recorded signal corresponding to a first set of temporal moments is applied to one or more of the vECUs, each of the vECUs, via the virtual hook. Based on the application of the received recorded signal to the first set of temporal moments, the sensor output from the vECU is received, Based on the sensor output received from the VECU, the second portion of the recorded signal corresponding to the second set of temporal moments is modified. The system according to any one of claims 2 to 4, configured to apply the modified second portion of the recorded signal to one or more of the vECUs to each of the vECUs via the virtual hook to the second set of temporal moments.

6. The aforementioned vehicle software test environment is During the execution of the aforementioned test plan, the response of the software application is recorded. The system according to any one of claims 2 to 5, further configured to perform the received test plan multiple times using the same signal as the received recorded signal, wherein the software application is nondeterministic, the multiple executions of the test plan indicate a range of execution levels of the nondeterministic software application, and the threshold range for authentication of the nondeterministic software application is based on the range of execution levels.

7. It is a method, Receiving or accessing a vehicle deployment graph of a given vehicle, which includes the configuration of one or more electronic control units (ECUs) of the given vehicle and the connection configuration of the ECUs of the given vehicle. To determine the respective configurations for one or more virtual electronic control units (vECUs) to be used to simulate one or more of the ECUs in the vehicle deployment graph, A method comprising providing one or more vECUs having the respective determined configurations for use in authenticating a software application to be implemented in a given vehicle, wherein the provided one or more vECUs are implemented on one or more virtual compute instances configured based on the respective determined configurations for the one or more vECUs.

8. Determining the respective configurations for the one or more vECUs is Determining the machine image for each of the one or more vECUs, and determining that each machine image emulates the software environment of each of the ECUs shown in the vehicle deployment graph, The method according to claim 7, comprising determining the instance type of each of the one or more virtual compute instances used to implement the vECU.

9. Determining the instance type of each of the one or more virtual compute instances used to implement the vECU is: The process involves determining the respective instance types based on the processor types of the one or more ECUs shown in the vehicle deployment graph, The method according to claim 7 or 8, further comprising determining the respective instance type based on the respective machine images determined to be used for one or more vECUs.

10. To provide one or more vECUs having the respective determined configurations and implemented in one or more of the virtual compute instances, Provisioning one or more virtual compute instances of the determined instance type based on the respective processor types of the ECUs shown in the vehicle deployment graph, The method according to claim 8 or 9, further comprising booting one or more virtual compute instances using the determined machine images.

11. Determining the machine image for each of the one or more vECUs is The method according to claim 8, further comprising selecting each of the machine images from a machine image catalog which includes machine images of ECUs used in multiple types of vehicles.

12. Receiving ECU information about the ECU from the vehicle manufacturer or vehicle parts manufacturer, Based on the received ECU information, a new machine image is created, The method according to claim 11, further comprising adding the new machine image to the machine image catalog.

13. Receiving a test plan for certifying the software application for use in the given vehicle, The implementation of the received test plan further includes, Receiving recorded signals from one or more ECUs of the given vehicle, or a vehicle having a similar configuration to the given vehicle, The received recorded signal is applied to each of the vECUs via one or more virtual hooks among the vECUs, The method according to any one of claims 7 to 12, comprising determining that the response resulting from the software application is within a threshold range so as to be provided to the test plan.

14. The received test plan is to be carried out as described above. This includes carrying out the received test plan, and the carrying out of the test plan is The method according to claim 13, comprising modifying the recorded signals of one or more ECUs based on downstream changes from the software application.

15. Recording the response of the software application, The method of claim 13 or 14, further comprising performing the received test plan to authenticate the software application for use in the given vehicle, wherein the received test plan is performed multiple times using the received recorded signals using the provided computing resources configured as the vECU configured in the virtual bus configuration, and the software application is nondeterministic.

Images