Systems, methods, and computer programs for viewing test case results

Abstract

System provision. [Solution] A system, method, and device for testing vehicle software are provided. According to an exemplary embodiment, the system may include: a memory storage that stores computer executable instructions; and at least one processor communicatively connected to the memory storage, wherein the at least one processor is configured to execute instructions to generate a graphical user interface having at least a first part and a second part, the first part of which may include one or more elements of a test environment; and the second part of which may include a log area; to receive a selection input from a user to select one of the one or more elements displayed in the first part; and, in response to receiving the selection input, to display a test log in the log area associated with only the selected one of the one or more elements.

Description

Technical Field

[0001] Systems, methods, and computer programs consistent with exemplary embodiments of the present disclosure relate to vehicle software testing, and more particularly, to viewing test case results in vehicle software testing.

Background Art

[0002] In software development, software needs to be tested to ensure that the software functions as intended, meets specified requirements, and functions reliably in various scenarios. Software testing is an important part of the software development life cycle (SDLC) and is performed to identify defects, errors, or bugs in the software before the software is deployed to an actual system.

[0003] Testing software for a vehicle can be done by designing test cases that include multiple test steps to be executed within a test environment. Then, the results of the execution of the test case within the test environment are generated and can be viewed by a user to evaluate the software.

[0004] In the related art, the results of the execution of a test case can be presented as a simple pass / fail status for the test case as a whole and / or as a general log associated with all elements of the test. In this regard, it is difficult for a user to determine which specific test steps within the test case are passing or failing, and reading through the general log associated with all elements of the test is a time-consuming process. Therefore, the process of determining the specific cause of a failure is time-consuming and error-prone. Thus, there is a need for a system that can enable a user to view the results of the execution of a test case and quickly and efficiently identify the specific cause of a failure.

Summary of the Invention

[0005] The exemplary embodiments of this disclosure test vehicle software. Accordingly, the exemplary embodiments of this disclosure provide a graphical user interface that enables a user to quickly view test results (i.e., test logs, status, graphs, and similar) for specific elements of a test case and specific test steps at a particular point in time. Accordingly, the graphical user interface facilitates the analysis and identification of errors in the test case and thus reduces the time required to perform such analysis and identification.

[0006] According to an exemplary embodiment, a system is provided. The system may include a memory storage that stores computer executable instructions, and at least one processor that is communicatively connected to the memory storage, wherein the at least one processor can be configured to execute instructions to generate a graphical user interface having at least a first part and a second part, the first part of which may include one or more elements of a test environment, and the second part of which may include a log area, to receive a selection input from a user to select one of the one or more elements displayed in the first part, and in response to receiving the selection input, to display a test log in the log area associated with only the selected one of the one or more elements.

[0007] According to an exemplary embodiment, the graphical user interface may further include a third part, which may include one or more test steps of a test case and a status associated with each of the one or more test steps.

[0008] According to exemplary embodiments, one or more elements may include one or more electronic control units associated with the type of vehicle.

[0009] According to an exemplary embodiment, the second part may further include a graph area, which may include a graph showing one or more communications between one or more elements over a certain period of time.

[0010] According to an exemplary embodiment, the graphical user interface may further include a fourth part, the fourth part of which may include a video simulation of a simulated test environment.

[0011] According to an exemplary embodiment, the graphical user interface may further include a scrub section associated with a test log in a second section and a video simulation in a fourth section, and at least one processor may further be configured to receive a slide input from the user to execute an instruction to slide the scrub section to a certain time instance, and in response to the receipt of the slide input, update the test log displayed in the log area and the video simulation displayed in the fourth section based on the time instance.

[0012] According to an exemplary embodiment, a fourth part may further include an annotation window, and at least one processor may further be configured to execute instructions to receive annotation input from a user and to display the annotation input in the annotation window.

[0013] According to exemplary embodiments, a method is provided. The method may include generating a graphical user interface comprising at least a first part and a second part, wherein the first part may include one or more elements of a test environment, and the second part may include a log area; receiving a selection input from a user to select one of the one or more elements displayed in the first part; and, in response to receiving the selection input, displaying a test log in the log area associated with only the selected one of the one or more elements.

[0014] According to an exemplary embodiment, the graphical user interface may further include a third part, which may include one or more test steps of a test case and a status associated with each of the one or more test steps.

[0015] According to exemplary embodiments, one or more elements may include one or more electronic control units associated with the type of vehicle.

[0016] According to an exemplary embodiment, the second part may further include a graph area, which may include a graph showing one or more communications between one or more elements over a certain period of time.

[0017] According to an exemplary embodiment, the graphical user interface may further include a fourth part, the fourth part of which may include a video simulation of a simulated test environment.

[0018] According to an exemplary embodiment, the graphical user interface may further include a scrub section associated with a test log in a second section and a video simulation in a fourth section, and the method may further include receiving a slide input from a user to slide the scrub section to a certain time instance, and updating the test log displayed in the log area and the video simulation displayed in the fourth section based on the time instance in response to the receipt of the slide input.

[0019] According to an exemplary embodiment, a fourth part may further include an annotation window, and the method may further include receiving annotation input from a user and displaying the annotation input in the annotation window.

[0020] According to an exemplary embodiment, a non-temporary computer-readable recording medium is provided. The non-temporary computer-readable recording medium can record instructions for causing at least one processor to perform a method, which is executable by at least one processor, the method being to generate a graphical user interface comprising at least a first part and a second part, the first part being to include one or more elements of a test environment, and the second part being to include a log area; receiving a selection input from a user to select one of the one or more elements displayed in the first part; and, in response to receiving the selection input, displaying a test log in the log area associated with only the selected one of the one or more elements.

[0021] According to an exemplary embodiment, the graphical user interface may further include a third part, which may include one or more test steps of a test case and a status associated with each of the one or more test steps.

[0022] According to exemplary embodiments, one or more elements may include one or more electronic control units associated with the type of vehicle.

[0023] According to an exemplary embodiment, the second part may further include a graph area, which may include a graph showing one or more communications between one or more elements over a certain period of time.

[0024] According to an exemplary embodiment, the graphical user interface may further include a fourth part, the fourth part of which may include a video simulation of a simulated test environment.

[0025] According to an exemplary embodiment, the graphical user interface may further include a scrubbing section associated with a test log in a second portion and a video simulation in a fourth portion, and the method may further include receiving, from a user, a slide input that slides the scrubbing section to a certain time instance, and updating, based on the time instance in response to receiving the slide input, the test log displayed in a log area and the video simulation displayed in the fourth portion.

[0026] Additional aspects may be described in part in the following description, become apparent in part from the description, or may be realized by practice of the presented embodiments of the disclosure.

Brief Description of the Drawings

[0027] The features, advantages, and significance of the preferred embodiments of the present disclosure are described below with reference to the accompanying drawings, in which like reference numerals indicate like elements.

[0028] [Figure 1] FIG. 1 shows a block diagram of exemplary components in a vehicle software test (VST) system according to one or more embodiments. [Figure 2] FIG. 2 shows exemplary components of a graphical user interface (GUI) for testing vehicle software according to one or more embodiments. [Figure 3] FIG. 3 shows exemplary components of a first portion of a graphical user interface (GUI) for testing vehicle software according to one or more embodiments. [Figure 4] FIG. 4 shows exemplary components of a third portion of a graphical user interface (GUI) for testing vehicle software according to one or more embodiments. [Figure 5] FIG. 5 shows a flowchart of an exemplary method for testing vehicle software according to one or more embodiments.

Modes for Carrying Out the Invention

[0029] A detailed description of exemplary embodiments follows with reference to the accompanying drawings. The same reference numerals in different drawings may identify the same or similar elements.

[0030] The foregoing disclosures are illustrative and illustrative, but are not intended to be exhaustive or to limit implementations to the exact forms disclosed. Modifications and variations may be possible in view of the foregoing disclosures or may be obtained from the implementations. Furthermore, one or more features or components of one embodiment may be incorporated into or combined with another embodiment (or one or more features of another embodiment). Furthermore, it should be understood that in the flowcharts and descriptions of operations provided below, one or more operations may be omitted, one or more operations may be added, one or more operations may be performed (at least partially) simultaneously, and the order of one or more operations may be changed.

[0031] It will be apparent that the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or combinations of hardware and software. The actual dedicated control hardware or software code used to implement such systems and / or methods is not a limitation of the implementation. Therefore, the operation and behavior of the systems and / or methods are described herein without reference to specific software code. It will be understood that software and hardware can be designed to implement the systems and / or methods based on the descriptions herein.

[0032] Where particular combinations of features are enumerated in the claims and / or disclosed herein, such combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features can be combined in ways not specifically enumerated in the claims and / or disclosed herein. Each of the dependent claims listed below may depend directly on only one claim, but the disclosure of possible implementations includes each dependent claim combined with all other claims in the set of claims.

[0033] Any element, action, or instruction used herein should not be construed as important or essential unless explicitly stated otherwise. Furthermore, when used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” When referring to only one item, the term “one” or similar terms should be used. Also, when used herein, the terms “has,” “have,” “having,” “include,” “including,” or similar terms are intended to be open-ended. Additionally, the phrase “based on” is intended to mean “at least partially based on” unless explicitly stated otherwise. Furthermore, expressions such as “at least one of [A] and [B],” “[A] and / or [B],” or “at least one of [A] or [B]” should be understood as including only A, only B, or both A and B.

[0034] The features, advantages, and importance of the exemplary embodiments described herein are merely a part of the disclosure and are not intended to be exhaustive or to limit the scope of the disclosure.

[0035] Further descriptions of the features, components, configuration, operation, and implementations of the threshold adjustment system of this disclosure according to one or more embodiments are provided below.

[0036] Exemplary system architecture Figure 1 shows a block diagram of exemplary components in a vehicle software test (VST) system 100 according to one or more embodiments. The VST system 100 may include devices, systems, platforms, modules, or similar items that can be configured to perform one or more operations or actions for testing vehicle software.

[0037] As shown in Figure 1, the VST system 100 may include at least one communication interface 110, at least one processor 120, at least one input / output component 130, and at least one storage 140, but it can be understood that the VST system 100 may include more or fewer components than those shown in Figure 1 and / or may be arranged in a different manner than those shown in Figure 1, without departing from the scope of this disclosure.

[0038] The communication interface 110 may include at least one transceiver-like component (e.g., transceiver, separate receiver and transmitter, bus, etc.) that enables the components of the VST system 100 to communicate with each other and / or with one or more components outside the VST system 100 via wired connections, wireless connections, or a combination of wired and wireless connections. For example, the communication interface 110 may include an Ethernet interface, optical interface, coaxial interface, infrared interface, radio frequency (RF) interface, universal serial bus (USB) interface, Wi-Fi interface, cellular network interface, or similar.

[0039] For example, the communication interface 110 may connect the processor 120 to the storage 140, thereby enabling them to communicate with each other and operate together when performing one or more operations.

[0040] According to one or more embodiments, the communication interface 110 may include one or more application programming interfaces (APIs) that enable the VST system 100 (or one or more components contained therein) to communicate with one or more software applications.

[0041] The input / output component 130 may include at least one component that enables the VST system 100 to receive and / or provide output information. In some embodiments, the input / output component 130 may include at least one input component (e.g., a touchscreen display, keyboard, keypad, mouse, button, switch, microphone, etc.) and at least one output component (e.g., a display, speaker, one or more light-emitting diodes (LEDs), etc.), each of which may be separated from each other. Furthermore or alternatively, at least one input component may include a sensor that detects information (e.g., a global positioning system (GPS) component, accelerometer, gyroscope, and / or actuator).

[0042] Storage 140 may include one or more storage media suitable for internally storing data, information, and / or computer executable instructions. According to exemplary embodiments, storage 140 may include at least one memory storage such as random access memory (RAM), read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, and / or optical memory) for storing information and / or instructions for use by the processor 120. Further or alternatively, storage 140 may include, together with a corresponding drive, a hard disk (e.g., magnetic disk, optical disk, magneto-optical disk, and / or solid-state disk), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cartridge, a magnetic tape, and / or another type of non-temporary computer-readable media. Computer-readable media are defined herein as non-temporary memory devices. A memory device may include memory space within a single physical storage device or memory space extending across multiple physical storage devices.

[0043] According to an exemplary embodiment, the storage 140 may be configured to store information such as raw data, metadata, or the like. Furthermore, or alternatively, the storage 140 may be configured to store one or more pieces of information associated with one or more operations performed by the processor 120. For example, the storage 140 may store information defining past operations performed by the processor 120 to test the vehicle software, one or more results of operations performed by the processor 120, or the like. Furthermore, the storage 140 may store data or information required when testing the vehicle software.

[0044] In some implementation, the storage 140 may include multiple storage media, and the storage 140 may be configured to store copies or duplicates of at least a portion of the information in the multiple storage media in order to provide redundancy and to back up the information or associated data. Furthermore, the storage 140 may also store computer-readable instructions or computer-executable instructions that, when executed by one or more processors (e.g., processor 120), cause one or more processors to perform one or more operations / behaviors described herein.

[0045] The processor 120 may include at least one processor that is programmable or configurable to perform functions or operations described herein. For example, the processor 120 may be configured to execute computer executable instructions stored in at least one storage medium or memory storage (e.g., storage 140) to perform one or more operations or actions described herein.

[0046] According to exemplary embodiments, the processor 120 may be configured to receive one or more signals and / or one or more user inputs (e.g., via a communication interface 110, an input / output component 130, etc.) that define one or more instructions that perform one or more operations. Furthermore, the processor 120 may be implemented in hardware, firmware, or a combination of hardware and software. For example, the processor 120 may include at least one of a central processing unit (CPU), an image processing unit (GPU), an accelerator processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), and / or another type of processing or computing component.

[0047] According to an exemplary embodiment, the processor 120 may be configured to collect, extract, and / or receive one or more pieces of information (such as signals or data), process the received pieces of information, and thereby test the vehicle software.

[0048] The number and arrangement of components shown in Figure 1 are provided as an example. In fact, VST system 100 may include additional components, fewer components, different components, or components arranged differently compared to those shown in Figure 1. Furthermore or alternatively, a set of components of VST system 100 (e.g., one or more components) may perform one or more functions that are described as being performed by another set of components of VST system 100.

[0049] A description of some exemplary operations that the processor 120 may perform is provided below with reference to Figure 5.

[0050] Examples of vehicle software testing in this disclosure According to an exemplary embodiment, one or more test cases may be designed and executed within a test environment to test vehicle software.

[0051] A test case can refer to a specific set of conditions and / or steps designed to verify the functionality or behavior of software (or the system under test). A test case may define one or more operations and expected outcomes for a particular test scenario. Conditions and / or steps designed to verify the functionality or behavior of software may be referred to as test steps included in a test case. According to exemplary embodiments, one or more test steps may be included in a test case, which may include one or more of the following: a test case ID (e.g., a set of unique identifiers or numbers assigned to a test case for tracking and reference purposes), a test objective (e.g., a description or specification of the goal or objective of the test case), at least one test condition (e.g., prerequisites or initial states required to run the test case, specific settings or configurations required to trigger the test run), at least one test operation (e.g., an operation or action performed to run the test case, required inputs or interactions with the software), at least one expected outcome (e.g., the expected result or behavior expected from the software when the test case runs successfully), and one or more of any other suitable information.

[0052] A test environment may refer to a specific set of conditions and / or elements, and a test case is performed in (against) those conditions and / or elements. In particular, a test environment may include one or more vehicle elements involved in a test case. One or more vehicle elements may refer to one or more elements associated with one or more components of a vehicle, e.g., electronic control units (ECUs), motors, and similar. For example, in a test case designed to test software related to detecting and displaying obstacles near the rear of a vehicle, the test environment may include the vehicle's (one or more vehicle elements) IVI ECU and ADAS ECU. According to exemplary embodiments, a test environment may further include one or more environmental elements involved in a test case. One or more environmental elements may refer to one or more elements associated with the vehicle's environment. For example, in a test case designed to test software related to detecting and displaying obstacles near the rear of a vehicle, the test environment may further include obstacles near the rear of the vehicle and rain (one or more environmental elements), and as a result, when the vehicle's IVI ECU and ADAS ECU detect and display obstacles near the rear of the vehicle during rain, the test case may be able to evaluate the software associated with the vehicle's IVI ECU and ADAS ECU.

[0053] Figure 2 shows exemplary components of a graphical user interface (GUI) 200 for testing vehicle software according to one or more embodiments.

[0054] As shown in Figure 2, the graphical user interface (GUI) 200 may include at least a first portion 220 and a second portion 240.

[0055] According to an exemplary embodiment, the first part 220 may include one or more elements of a test environment. One or more elements may include one or more vehicle elements and / or one or more environmental elements. One or more vehicle elements may be elements associated with one or more components of a vehicle in the test environment, e.g., electronic control units (ECUs), motors, and the like. According to an exemplary embodiment, one or more vehicle elements may be elements associated with one or more components of a particular type (model) of vehicle in the test environment. For example, one or more elements may include one or more electronic control units (ECUs) of a particular type of vehicle (which may differ from ECUs in different types of vehicles). One or more environmental elements may be elements associated with the environment of a vehicle in the test environment, e.g., amount of rain, amount of moisture, sunlight, specific weather, and the like.

[0056] Figure 3 shows exemplary components of a first portion 300 of a graphical user interface (GUI) for testing vehicle software according to one or more embodiments. The first portion 300 may correspond to the first portion 220 in Figure 2, and therefore, features associated with the first portion 300 and the first portion 220 may be similarly applicable to each other unless otherwise explicitly stated.

[0057] As shown in Figure 3, the first part 300 may include three elements: an in-vehicle infotainment (IVI) electronic control unit (ECU) 320, an advanced driver-assistance system (ADAS) electronic control unit (ECU) 340, and a transmission control module (TCM) electronic control unit (ECU) 360. The IVI ECU 320, ADAS ECU 340, and TCM ECU 360 may correspond to the IVI ECU, ADAS ECU, and TCM ECU in a vehicle, respectively. It should be understood that the configuration shown in Figure 3 is simplified for illustrative purposes and is not intended to limit the scope of this disclosure. For example, the number of elements may be arbitrary, and the elements may be any elements in the test environment described above.

[0058] Here, it can be understood that, for example, a user may want to perform a test case that is executed in a test environment equipped with the vehicle's IVI ECU, ADAS ECU, and TCM ECU (i.e., the test is performed to test the vehicle's IVI ECU, ADAS ECU, and TCM ECU). Therefore, the first part may show the IVI ECU 320, ADAS ECU 340, and TCM ECU 360 as shown in Figure 3.

[0059] Returning to Figure 2, according to an exemplary embodiment, the second portion 240 may include a log area. The log area may display test logs associated with a test case. According to an exemplary embodiment, the log area may display a general test log associated with all elements of the test environment, and / or test logs associated with a specific element of the test environment. For example, the log area may display a general test log associated with the IVI ECU320, ADAS ECU340, and TCM ECU360 in the test environment, and / or test logs associated with one of the IVI ECU320, ADAS ECU340, and TCM ECU360 in the test environment. It can be understood that a test log may refer to a log showing various information, activities, results, failures, and any other relevant data related to the execution of a test as is known in the art.

[0060] According to an exemplary embodiment, when a test case is executed in a test environment comprising one or more elements displayed in the first part, the VST system may display a general log associated with all of the one or more elements in the log area. The user may then provide a selection input to select one of the one or more elements displayed in the first part, where the VST system may then display a test log associated with only the selected element in the log area. For example, the user may select ADAS ECU340 displayed in the first part 300 in Figure 3, where the VST system may then display a test log associated only with ADAS ECU340 in the log area in the second part 240. According to an exemplary embodiment, the user may provide a selection input by clicking on an element in the first part 220.

[0061] Therefore, a GUI can enable users to quickly view specific test logs associated with specific elements of the test environment, thereby reducing the time required to analyze and identify the cause of errors within test cases.

[0062] According to an exemplary embodiment, the GUI200 may further include a third part (not shown in Figure 2). The third part may include one or more test steps of a test case, along with the status associated with each of the one or more test steps.

[0063] Figure 4 shows exemplary components of a third part 400 of a graphical user interface (GUI) for testing vehicle software according to one or more embodiments. As shown in Figure 4, the third part 400 may include five test steps, namely test step 1 410, test step 2 420, test step 3 430, test step 4 440, and test step 5 450, and their respective associated statuses 412, 422, 432, 442, and 452. It should be understood that the configuration shown in Figure 4 is simplified for illustrative purposes and is not intended in any way to limit the scope of this disclosure. For example, in fact, the number of test steps and associated statuses within the third part (and within the test case) may be any number, the specific content and format of the test steps may be arbitrary, and the specific format of the statuses may be arbitrary.

[0064] As shown in Figure 4, test step 1 410 may include the step “Assume to start REACTOR_SCENARIO” which defines the operation to start a test environment with the default conditions and / or elements on which the test case will be executed. Test step 2 420 may include the step “Turn on ADAS_ECU” which defines the operation to start an advanced driver assistance system (ADAS) electronic control unit (ECU) within the test environment. Similarly, test step 3 430 may include the step “Turn on IVI_ECU” which defines the operation to start an in-vehicle infotainment (IVI) electronic control unit (ECU) within the test environment.

[0065] Test step 440 may include the step "When the OBSTACLE is 2.0 meters away from the FRONT_RIGHT of the car" defining a conditional scenario in which the obstacle is 2.0 meters away from the front right of the car. Test step 5450 may include the step "The IVI then indicates that the OBSTACLE is at MAX_DISTANCE from the FRONT_RIGHT of the car" where the in-car infotainment indicates that the obstacle is at the maximum distance from the front right of the car.

[0066] Furthermore, status 412 may be associated with test step 1 410 and may indicate that test step 1 410 was successful (i.e., the start of "REACTOR_SCENARIO" was successful). Similar explanations apply to statuses 422, 432, and 442, which may be associated with test steps 2 420, 3 430, and 4 440, and 452

[0067] Therefore, a GUI can enable users to quickly identify exactly where within a test case (i.e., which specific test step within the test case) failed, and then quickly analyze and identify the cause of the error (failure) within the test case by viewing the specific test log in the log area.

[0068] Returning to Figure 2, according to an exemplary embodiment, the second portion 240 may further include a graph area. The graph area may include a graph showing one or more communications between one or more elements over a certain period of time. According to an exemplary embodiment, the graph may be a time-series graph. For example, the graph may show communications between the IVI ECU 320 and the ADAS ECU 340 over a certain period of time (e.g., from the start of the test case to the time when test step 5 450 fails).

[0069] Therefore, GUI200 can enable users to quickly view the interactions between elements of the test environment over a certain period of time, and then quickly analyze and identify the causes of errors in test cases by viewing specific test logs associated with specific elements in the log area.

[0070] According to an exemplary embodiment, the GUI200 may further include a fourth part (not shown in Figure 2). The fourth part may include a video simulation of a simulated test environment in which a test case is performed. For example, for a test case aimed at testing communication between the IVI ECU320 and the ADAS ECU340 during reverse parking, the fourth part may show a video simulation of a vehicle performing reverse parking.

[0071] Therefore, GUI200 can enable users to visualize real-world vehicle scenarios while testing, which improves the time available for analyzing and identifying the causes of errors within test cases.

[0072] According to an exemplary embodiment, a user may be able to add annotations to the video simulation in a fourth part. In particular, the fourth part may further include an annotation window, to which the user may provide annotation inputs. Annotation inputs may include comments, drawings, symbols, and the like that the user wishes to add to the video simulation as annotations. Thus, the user may provide annotations to the system, which may be displayed in the annotation window.

[0073] According to an exemplary embodiment, the GUI200 may further include a scrubber (not shown in Figure 2), which may be associated with one or more of the second part 240, the third part, and the fourth part.

[0074] According to an exemplary embodiment, a scrub section may be associated with a second section 240. In particular, a scrub section may be associated with a test log displayed in a log area in the second section 240, and as a result, when a user moves the scrub section to a specific time instance (i.e., provides input to slide the scrub section), the test log displayed in the log area may be updated to reflect the test log generated up to that specific time instance.

[0075] According to an exemplary embodiment, the scrub section may further be associated with the graph of the graph area in the second section 240, so that when the user moves the scrub section to a specific time instance (i.e., provides input to slide the scrub section), the graph of the graph area may be updated to reflect the communication between one or more elements up to that specific time instance.

[0076] According to an exemplary embodiment, a scrub section may be associated with a third section. In particular, a scrub section may be associated with a status displayed in the third section, and as a result, when a user moves the scrub section to a particular time instance (i.e., provides input to slide the scrub section), the status in the third section may be updated to reflect the status in that particular time instance.

[0077] According to an exemplary embodiment, a scrub section may be associated with a fourth section. In particular, a scrub section may be associated with a video simulation displayed in the fourth section, and as a result, when a user moves the scrub section to a specific time instance (i.e., provides input to slide the scrub section), the video simulation displayed in the fourth section may be updated to reflect the video simulation at that particular time instance.

[0078] Therefore, GUI200 allows users to quickly view test results at a specific point in time (i.e., test logs, test step status, graphs, and similar information), thereby facilitating the analysis of errors within test cases and reducing the time required to perform such analysis.

[0079] It should be understood that the configuration shown in Figure 2 is simplified for illustrative purposes and is not intended in any way to limit the scope of this disclosure. For example, the relative position, relative size, and / or shape of each element within GUI200 may actually be arbitrary.

[0080] Exemplary behavior for testing vehicle software in this disclosure Some exemplary operations that can be performed by the VST system of this disclosure are described below with reference to Figure 5.

[0081] Figure 5 shows a flowchart of an exemplary method 500 for testing vehicle software according to one or more embodiments. One or more operations in method 500 may be performed by at least one processor (e.g., processor 120) of the VST system.

[0082] As shown in Figure 5, in operation S510, at least one processor may be configured to generate a graphical user interface. The graphical user interface may include at least a first part and a second part, where the first part may include one or more elements of the test environment, and the second part may include a log area.

[0083] According to exemplary embodiments, one or more elements may include one or more vehicle elements and / or one or more environmental elements. One or more vehicle elements may be elements associated with one or more components of a vehicle in a test environment, e.g., electronic control units (ECUs), motors, and the like. According to exemplary embodiments, one or more vehicle elements may be elements associated with one or more components of a particular type (model) of vehicle in a test environment. For example, one or more elements may include one or more electronic control units (ECUs) of a particular type of vehicle. One or more environmental elements may be elements associated with the vehicle's environment, e.g., rainfall, moisture content, sunlight, specific weather, and the like.

[0084] According to an exemplary embodiment, the log area may display test logs associated with a test case. In particular, the log area may display a general test log associated with all elements of the test environment, and / or test logs associated with a specific element of the test environment. The method then proceeds to operation S520.

[0085] In operation S520, at least one processor may be configured to receive a selection input for selecting one of one or more elements displayed in the first part. The selection input may be received from a user. According to an exemplary embodiment, the selection input may include a click input. The method then proceeds to operation S530.

[0086] In operation S530, at least one processor may be configured to display a test log in response to the reception of a selected input. The test log may be associated with only one of more elements selected and may be displayed in the log area of ​​the second part.

[0087] When operation S530 is performed, method 500 may terminate or end. Alternatively, method 500 may return to operation S520, as a result, at least one processor may be configured to repeatedly receive selection input (in operation S520) and display the test log (in operation S530) for at least a predetermined amount of time. For example, the user may want to view test logs associated with different elements in the first part sequentially (or periodically). Therefore, at least one processor may then resume receiving selection input (in operation S520) and displaying the test log (in operation S530).

[0088] According to an exemplary embodiment, a second part of the graphical user interface may further include a graph area. The graph area may include a graph showing one or more communications between one or more elements over a certain period of time. According to an exemplary embodiment, the graph may be a time-series graph.

[0089] According to an exemplary embodiment, the graphical user interface may further include a third part. The third part may include one or more test steps of a test case and a status associated with each of the one or more test steps. According to an exemplary embodiment, when the test case is executed, each of the statuses may be updated to indicate the status of the associated one of the one or more test steps, for example, success, failure, and similar.

[0090] According to an exemplary embodiment, the graphical user interface may further include a fourth part, which may include a video simulation of a simulated test environment.

[0091] According to an exemplary embodiment, the graphical user interface may further include a scrub section, which may be associated with one or more of the second, third, and fourth sections.

[0092] According to an exemplary embodiment, the scrub section may be associated with a test log displayed in a log area in a second section. In this regard, at least one processor may be configured to receive a slide input from a user to slide the scrub section to a certain time instance. Subsequently, in response to the receipt of the slide input, at least one processor may be configured to update the test log displayed in the log area based on the time instance. For example, the test log displayed in the log area may be updated to reflect the test log generated up to the time instance.

[0093] According to an exemplary embodiment, the scrub section may be associated with a graph in a graph area in a second section. In this regard, at least one processor may be configured to receive a slide input from a user to slide the scrub section to a certain time instance. Subsequently, in response to the receipt of the slide input, at least one processor may be configured to update the graph displayed in the graph area based on the time instance. For example, the graph displayed in the graph area may be updated to reflect the graph (communication between one or more elements) generated up to the time instance.

[0094] According to an exemplary embodiment, the scrub portion may be associated with a status in a third portion. In this regard, at least one processor may be configured to receive a slide input from a user, which slides the scrub portion to a certain time instance. Subsequently, in response to the receipt of the slide input, at least one processor may be configured to update the status displayed in the third portion based on the time instance. For example, the status displayed in the third portion may be updated to reflect the status of each test step in the time instance.

[0095] According to an exemplary embodiment, the scrub section may be associated with a video simulation in a fourth section. In this regard, at least one processor may be configured to receive a slide input from a user, which slides the scrub section to a certain time instance. Subsequently, in response to the receipt of the slide input, at least one processor may be configured to update the video simulation in the fourth section based on the time instance. For example, the video simulation in the fourth section may be updated to reflect the video simulation in the time instance.

[0096] According to an exemplary embodiment, a fourth part may further include an annotation window. In this regard, at least one processor may be configured to receive annotation input from a user. The annotation input may include comments, drawings, symbols, and the like that the user wishes to add to the video simulation as annotations. Subsequently, in response to the receipt of the annotation input, at least one processor may be configured to display the annotation input in the annotation window.

[0097] Various embodiments The foregoing disclosures are illustrative and illustrative, but are not intended to be exhaustive or to limit implementations to the exact forms disclosed. Modifications and variations may be possible in view of the foregoing disclosures or may be derived from implementations.

[0098] Some embodiments may relate to systems, methods, and / or computer-readable media at any possible level of technical detail of integration. Furthermore, one or more of the above-described components may be implemented as instructions stored in a computer-readable medium and executable by at least one processor (and / or may include at least one processor). The computer-readable medium may include a computer-readable non-temporary storage medium (or multiple mediums) having computer-readable program instructions for causing a processor to perform an operation.

[0099] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction-executing device. A computer-readable storage medium may be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any preferred combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes, namely, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or grooved raised structures on which instructions are recorded, and any preferred combination thereof. The computer-readable storage media used herein should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmitting media (e.g., light pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0100] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device, or they may be downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0101] The computer-readable program code / instructions that perform the operation may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, or similar, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may be fully executed on the user's computer, partially executed on the user's computer, executed as a standalone software package, partially executed on the user's computer and partially executed on a remote computer, or fully executed on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), or a connection to an external computer may be made (for example, via the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer-readable program instructions by personalizing the electronic circuit using state information of computer-readable program instructions in order to perform a particular action or operation.

[0102] The computer-readable program instructions may be provided to a processor of a general-purpose computer, a dedicated computer, or other programmable data processing device to generate a machine, and as a result, the instructions executed via the processor of the computer or other programmable data processing device create means for implementing functions / actions specified in blocks or blocks of a flowchart and / or block diagram. The computer-readable program instructions may also be stored in a computer-readable storage medium that can instruct computers, programmable data processing devices, and / or other devices to function in a particular way, and as a result, the computer-readable storage medium in which the instructions are stored comprises a manufactured article containing instructions for implementing modes of functions / actions specified in blocks or blocks of a flowchart and / or block diagram.

[0103] Computer-readable program instructions can also be loaded onto a computer, other programmable data processing device, or other device to perform a series of operational steps on the computer, other programmable device, or other device, thereby generating a computer implementation process, the instructions executed on the computer, other programmable device, or other device, which implement the functions / actions specified in the blocks or blocks(s) of a flowchart and / or block diagram.

[0104] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable media according to various embodiments. In this regard, each block in a flowchart or block diagram may represent a microservice module, segment, or part of an instruction set comprising one or more executable instructions that implement a specified logical function. Methods, computer systems, and computer-readable media may include additional blocks, fewer blocks, different blocks, or blocks in different arrangements compared to those depicted in the figures. In some alternative implementations, the functions described in the blocks may occur regardless of the order in which they are shown in the figures. For example, two blocks shown consecutively may actually be executed simultaneously or substantially simultaneously, or blocks may be executed in reverse order depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs a specified function or action or executes a dedicated combination of hardware and computer instructions.

[0105] It will be apparent that the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or combinations of hardware and software. The actual dedicated control hardware or software code used to implement such systems and / or methods is not a limitation of the implementation. Therefore, the operation and behavior of the systems and / or methods are described herein without reference to specific software code, and it is understood that software and hardware can be designed to implement the systems and / or methods based on the descriptions herein.

[0106] It can be understood that many modifications and variations of this disclosure are possible in light of the teachings above. Within the scope of the attached sections, it will be apparent that this disclosure may be implemented in ways other than those specifically described herein.

Claims

1. It is a system, Memory storage that stores executable computer instructions, At least one processor that is communicatively connected to the memory storage, The processor comprises the above, and the at least one processor executes the instructions, To generate a graphical user interface comprising at least a first part and a second part, wherein the first part comprises one or more elements of a test environment, and the second part comprises a log area, The first part receives a selection input from the user to select one of the one or more elements displayed, In response to the receipt of the selection input, a test log associated with only the selected one of the one or more elements is displayed in the log area. A system configured to perform the following actions.

2. The graphical user interface further comprises a third part, The system according to claim 1, wherein the third part comprises one or more test steps of a test case and a status associated with each of the one or more test steps.

3. The system according to claim 1 or 2, wherein the one or more elements comprises one or more electronic control units associated with the type of vehicle.

4. The second section mentioned above further includes a graph area, The system according to claim 1 or 2, wherein the graph area comprises a graph showing one or more communications between one or more elements over a certain period of time.

5. The graphical user interface further comprises a fourth part, The system according to claim 1 or 2, wherein the fourth part comprises a video simulation of a simulated test environment.

6. The graphical user interface further comprises a scrubbing section associated with the test log in the second part and the video simulation in the fourth part, The at least one processor further executes the instruction: The user provides a slide input to slide the scrub section to a certain time instance. The system according to claim 5, configured to update the test log displayed in the log area and the video simulation displayed in the fourth portion based on the time instance in response to the reception of the slide input.

7. The fourth part described above further comprises an annotation window, The at least one processor further executes the instruction: Annotation input is received from the user, The system according to claim 5, wherein the annotation input is configured to be displayed in the annotation window.

8. A method performed by the processor, To generate a graphical user interface comprising at least a first part and a second part, wherein the first part comprises one or more elements of a test environment, and the second part comprises a log area, The first part receives a selection input from the user to select one of the one or more elements displayed, In response to the receipt of the selection input, a test log associated with only the selected one of the one or more elements is displayed in the log area. Methods that include...

9. The graphical user interface further comprises a third part, The method according to claim 8, wherein the third part comprises one or more test steps of a test case and a status associated with each of the one or more test steps.

10. The method according to claim 8 or 9, wherein the one or more elements comprises one or more electronic control units associated with the type of vehicle.

11. The second section mentioned above further includes a graph area, The method according to claim 8 or 9, wherein the graph area comprises a graph showing one or more communications between one or more elements over a certain period of time.

12. The graphical user interface further comprises a fourth part, The method according to claim 8 or 9, wherein the fourth part comprises a video simulation of a simulated test environment.

13. The graphical user interface further comprises a scrubbing section associated with the test log in the second part and the video simulation in the fourth part, The aforementioned method, Receiving a slide input from the user to slide the scrub section to a certain time instance, In response to the reception of the slide input, the test log displayed in the log area and the video simulation displayed in the fourth section are updated based on the time instance, The method according to claim 12, further comprising:

14. The fourth part described above further comprises an annotation window, The aforementioned method, Receiving annotation input from the user, Displaying the annotation input in the annotation window, The method according to claim 12, further comprising:

15. A computer program for causing at least one processor to perform a method, wherein the method is To generate a graphical user interface comprising at least a first part and a second part, wherein the first part comprises one or more elements of a test environment, and the second part comprises a log area, The first part receives a selection input from the user to select one of the one or more elements displayed, In response to the receipt of the selection input, a test log associated with only the selected one of the one or more elements is displayed in the log area. A computer program that includes [this].

16. The graphical user interface further comprises a third part, The computer program according to claim 15, wherein the third part comprises one or more test steps of a test case and a status associated with each of the one or more test steps.

17. The computer program according to claim 15 or 16, wherein the one or more elements comprises one or more electronic control units associated with a type of vehicle.

18. The second section mentioned above further includes a graph area, The computer program according to claim 15 or 16, wherein the graph area comprises a graph showing one or more communications between one or more elements over a certain period of time.

19. The graphical user interface further comprises a fourth part, The computer program according to claim 15 or 16, wherein the fourth portion comprises a video simulation of a simulated test environment.

20. The graphical user interface further comprises a scrubbing section associated with the test log in the second part and the video simulation in the fourth part, The aforementioned method, Receiving a slide input from the user to slide the scrub section to a certain time instance, In response to the reception of the slide input, the test log displayed in the log area and the video simulation displayed in the fourth section are updated based on the time instance, The computer program according to claim 19, further comprising:

Images