A system, method, and computer program for writing test steps in vehicle software testing.

Abstract

System provision. [Solution] A system, method, and device for testing vehicle software are provided. According to an exemplary embodiment, the system may include a memory storage that stores computer executable instructions, and at least one processor that is communicatively connected to the memory storage, wherein the at least one processor may be configured to execute instructions to receive one or more natural language codes from a first user, to determine whether the received one or more natural language codes include one or more new natural language codes that do not correspond to any of the one or more programming language codes stored in the memory storage, and, depending on whether it has determined that the received one or more natural language codes include one or more new natural language codes, to send one or more new natural language codes to a second user and to receive one or more new programming language codes from the second user.

Description

Technical Field

[0001] Systems, methods, and computer programs consistent with exemplary embodiments of the present disclosure relate to vehicle software testing, and more particularly, to writing test steps in vehicle software testing.

Background Art

[0002] In software development, software needs to be tested to ensure that it functions as intended, meets specified requirements, and operates reliably in various scenarios. Software testing is an important part of the software development life cycle (SDLC) and is performed to identify defects, errors, or bugs in the software before it is deployed to the actual system.

[0003] Testing software for a vehicle can be done by designing test cases that include multiple test steps operating within a test environment. Then, the results of the test case operations within the test environment are generated and can be viewed by a user to evaluate the software.

[0004] In the related art, test cases are designed by inputting and typing test steps in a programming language such as Python (registered trademark). However, such a process requires the user to have extensive knowledge of the relevant programming language, and users without such knowledge may find it difficult to design test cases, thereby increasing the risk of time and errors. Therefore, there is a need for a system that can enable users to quickly and efficiently design test cases without the need for knowledge of a programming language.

Summary of the Invention

[0005] The exemplary embodiments of this disclosure test vehicle software. Therefore, the exemplary embodiments of this disclosure enable users without extensive knowledge of low-level programming to easily design and consider test cases, while simultaneously providing such users with the flexibility to create new test steps. Furthermore, the exemplary embodiments of this disclosure enable the rapid and efficient addition of test steps to test cases, eliminating the need for users to manually type such test steps.

[0006] According to an exemplary embodiment, a system is provided. The system may relate to testing vehicle software and may include: memory storage storing computer executable instructions; and at least one processor communicatively connected to the memory storage, wherein the at least one processor may be configured to execute instructions to receive one or more natural language codes from a first user, determine whether the received one or more natural language codes include one or more new natural language codes that do not correspond to any of the one or more programming language codes stored in the memory storage, and, in response to determining that the received one or more natural language codes include one or more new natural language codes, send one or more new natural language codes to a second user and receive one or more new programming language codes from the second user, wherein the received one or more new programming language codes may correspond to one or more new natural language codes.

[0007] According to an exemplary embodiment, at least one processor may be further configured to execute instructions and store one or more new programming language codes and corresponding one or more new natural language codes in memory storage as a default test step.

[0008] According to an exemplary embodiment, at least one processor may be configured to determine whether a received natural language code includes one or more new natural language codes by executing an instruction to execute one or more received natural language codes, the received natural language codes may be executed by executing one or more corresponding programming language codes, determining that the received natural language codes cannot be executed, and identifying one or more codes from the received natural language codes that cannot be executed.

[0009] According to an exemplary embodiment, at least one processor may be further configured to execute instructions and, in response to receiving one or more new programming language codes, determine whether one or more received natural language codes are successfully executed.

[0010] According to an exemplary embodiment, at least one processor may be configured to execute instructions to generate a graphical user interface comprising at least a first part and a second part, wherein the first part may include a part for entering code, and the second part may include a search area and a test step area; to receive a search input from a first user in the search area; to display one or more test steps associated with the search input in the test step area; to receive a selection input from the first user for selecting one of the one or more test steps displayed in the test step area; and to input a natural language code associated with the selected one of the one or more test steps into the first part.

[0011] According to an exemplary embodiment, one or more natural language codes may be received in the first part.

[0012] According to an exemplary embodiment, the memory storage may include a plurality of default test steps, each of which may be associated with a natural language code and a corresponding one or more programming language codes, and one or more test steps associated with a search input may be obtained from the plurality of default test steps.

[0013] According to an exemplary embodiment, the first part may further include a pop-up window, and at least one processor is further configured to execute instructions to receive partial natural language code input from a first user in the first part, display one or more test steps associated with the partial natural language code input in the pop-up window, receive selection input from the first user to select one of the one or more test steps displayed in the pop-up window, and in the first part to replace the partial natural language code input with natural language code associated with one of the selected test steps.

[0014] According to an exemplary embodiment, one or more natural language codes may be in the Gherkin programming language format, and one or more programming language codes may be in the Python® programming language format.

[0015] According to an exemplary embodiment, a method is provided. The method may be performed by a system for testing vehicle software and may include: receiving one or more natural language codes from a first user; determining whether the received one or more natural language codes include one or more new natural language codes that do not correspond to any of the one or more programming language codes stored in the system's memory storage; sending one or more new natural language codes to a second user in response to determining that the received one or more natural language codes include one or more new natural language codes; and receiving one or more new programming language codes from the second user, the received one or more new programming language codes may correspond to one or more new natural language codes.

[0016] According to an exemplary embodiment, the method may further include storing one or more new programming language codes and corresponding one or more new natural language codes in memory storage as a default test step.

[0017] According to an exemplary embodiment, determining whether one or more received natural language codes include one or more new natural language codes may include determining whether the received one or more natural language codes can be executed by executing one or more corresponding programming language codes, determining whether the received one or more natural language codes cannot be executed, and identifying one or more codes from the received one or more natural language codes that cannot be executed.

[0018] According to an exemplary embodiment, the method may further include determining whether one or more received natural language codes are successfully executed, in response to receiving one or more new programming language codes.

[0019] According to an exemplary embodiment, the method is to generate a graphical user interface comprising at least a first part and a second part, wherein the first part may include a part for entering code, and the second part may include a search area and a test step area; to receive a search input from a first user in the search area; to display one or more test steps associated with the search input in the test step area; to receive a selection input from the first user for selecting one of the one or more test steps displayed in the test step area; and to input a natural language code associated with the selected one of the one or more test steps into the first part.

[0020] According to an exemplary embodiment, one or more natural language codes may be received in the first part.

[0021] According to an exemplary embodiment, the memory storage may include a plurality of default test steps, each of which may be associated with a natural language code and a corresponding one or more programming language codes, and one or more test steps associated with a search input may be obtained from the plurality of default test steps.

[0022] According to an exemplary embodiment, the first part may further include a popup window, and the method may further include receiving a partial natural language code input from a first user in the first part; displaying one or more test steps associated with the partial natural language code input in the popup window; receiving a selection input from the first user to select one of the one or more test steps displayed in the popup window; and in the first part replacing the partial natural language code input with a natural language code associated with one of the selected test steps.

[0023] According to an exemplary embodiment, one or more natural language codes may be in the Gherkin programming language format, and one or more programming language codes may be in the Python® programming language format.

[0024] According to an exemplary embodiment, a non-temporary computer-readable recording medium is provided. The non-temporary computer-readable recording medium may record on the non-temporary computer-readable recording medium instructions for causing at least one processor to perform a method, which are executable by at least one processor, and the method includes: receiving one or more natural language codes from a first user; determining whether the received one or more natural language codes include one or more new natural language codes that do not correspond to any of the one or more programming language codes stored in the memory storage of the system performing the method; transmitting one or more new natural language codes to a second user in response to determining that the received one or more natural language codes include one or more new natural language codes; and receiving one or more new programming language codes from the second user, wherein the received one or more new programming language codes may correspond to one or more new natural language codes.

[0025] According to an exemplary embodiment, determining whether one or more received natural language codes include one or more new natural language codes may include determining whether the received one or more natural language codes can be executed by executing one or more corresponding programming language codes, determining whether the received one or more natural language codes cannot be executed, and identifying one or more codes from the received one or more natural language codes that cannot be executed.

[0026] Additional embodiments may be partially described in the following description, partially evident from the description, or realized by implementing the embodiments presented in this disclosure. [Brief explanation of the drawing]

[0027] The features, advantages, and significance of the preferred embodiments of the present disclosure are described below with reference to the accompanying drawings, in which like reference numerals indicate like elements.

[0028] [Figure 1] FIG. 1 shows a block diagram of exemplary components in a vehicle software test (VST) system according to one or more embodiments. [Figure 2] FIG. 2 shows an exemplary test case according to one or more embodiments. [Figure 3] FIG. 3 shows exemplary components of a graphical user interface (GUI) for testing vehicle software according to one or more embodiments. [Figure 4] FIG. 4 shows a flowchart of an exemplary method for testing vehicle software according to one or more embodiments. [Figure 5] FIG. 5 shows a flowchart of an exemplary method for testing vehicle software according to one or more embodiments. [Figure 6] FIG. 6 shows a flowchart of an exemplary method for testing vehicle software according to one or more embodiments. [Figure 7] FIG. 7 shows a flowchart of an exemplary method for testing vehicle software according to one or more embodiments.

Best Mode for Carrying Out the Invention

[0029] The following detailed description of the exemplary embodiments refers to the accompanying drawings. The same reference numerals in different drawings can identify the same or similar elements.

[0030] The foregoing disclosures are illustrative and illustrative, but are not intended to be exhaustive or to limit implementations to the exact forms disclosed. Modifications and variations may be possible in view of the foregoing disclosures or may be obtained from the implementations. Furthermore, one or more features or components of one embodiment may be incorporated into or combined with another embodiment (or one or more features of another embodiment). Furthermore, it should be understood that in the flowcharts and descriptions of operations provided below, one or more operations may be omitted, one or more operations may be added, one or more operations may be performed (at least partially) simultaneously, and the order of one or more operations may be changed.

[0031] It will be apparent that the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or combinations of hardware and software. The actual dedicated control hardware or software code used to implement such systems and / or methods is not a limitation of the implementation. Therefore, the operation and behavior of the systems and / or methods are described herein without reference to specific software code. It will be understood that software and hardware can be designed to implement the systems and / or methods based on the descriptions herein.

[0032] Even if certain combinations of features are enumerated in the claims and / or disclosed herein, such combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features can be combined in ways not specifically enumerated in the claims and / or disclosed herein. Each of the dependent claims listed below may depend directly on only one claim, but the disclosure of possible implementations includes each dependent claim combined with all other claims in the set of claims.

[0033] Any element, action, or instruction used herein should not be construed as important or essential unless explicitly stated otherwise. Furthermore, when used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” When referring to only one item, the term “one” or similar terms should be used. Also, when used herein, the terms “has,” “have,” “having,” “include,” “including,” or similar terms are intended to be open-ended. Additionally, the phrase “based on” is intended to mean “at least partially based on” unless explicitly stated otherwise. Furthermore, expressions such as “at least one of [A] and [B],” “[A] and / or [B],” or “at least one of [A] or [B]” should be understood as including only A, only B, or both A and B.

[0034] The features, advantages, and importance of the exemplary embodiments described herein are merely a part of the disclosure and are not intended to be exhaustive or to limit the scope of the disclosure.

[0035] Further descriptions of the features, components, configuration, operation, and implementations of the threshold adjustment system of this disclosure according to one or more embodiments are provided below.

[0036] Exemplary system architecture Figure 1 shows a block diagram of exemplary components in a vehicle software test (VST) system 100 according to one or more embodiments. The VST system 100 may include devices, systems, platforms, modules, or similar items that can be configured to perform one or more operations or actions for testing vehicle software.

[0037] As shown in Figure 1, the VST system 100 may include at least one communication interface 110, at least one processor 120, at least one input / output unit 130, and at least one storage 140, but it can be understood that the VST system 100 may include more or fewer components than those shown in Figure 1 and / or be arranged in a different manner than those shown in Figure 1, without departing from the scope of this disclosure.

[0038] The communication interface 110 may include at least one transceiver-like component (e.g., transceiver, separate receiver and transmitter, bus, etc.) that enables the components of the VST system 100 to communicate with each other and / or with one or more components outside the VST system 100 via wired connections, wireless connections, or a combination of wired and wireless connections. For example, the communication interface 110 may include an Ethernet interface, optical interface, coaxial interface, infrared interface, radio frequency (RF) interface, universal serial bus (USB) interface, Wi-Fi interface, cellular network interface, or similar.

[0039] For example, the communication interface 110 may connect the processor 120 to the storage 140, thereby enabling them to communicate with each other and operate together when performing one or more operations.

[0040] According to one or more embodiments, the communication interface 110 may include one or more application programming interfaces (APIs) that enable the VST system 100 (or one or more components contained therein) to communicate with one or more software applications.

[0041] The input / output unit 130 may include at least one component that enables the VST system 100 to receive and / or provide output information. In some embodiments, the input / output unit 130 may include at least one input component (e.g., a touchscreen display, keyboard, keypad, mouse, button, switch, microphone, etc.) and at least one output component (e.g., a display, speaker, one or more light-emitting diodes (LEDs), etc.), each of which may be separated from each other. Furthermore or alternatively, at least one input component may include a sensor that detects information (e.g., a global positioning system (GPS) component, accelerometer, gyroscope, and / or actuator).

[0042] Storage 140 may include one or more storage media suitable for internally storing data, information, and / or computer executable instructions. According to exemplary embodiments, storage 140 may include at least one memory storage such as random access memory (RAM), read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, and / or optical memory) for storing information and / or instructions for use by the processor 120. Further or alternatively, storage 140 may include, together with a corresponding drive, a hard disk (e.g., magnetic disk, optical disk, magneto-optical disk, and / or solid-state disk), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cartridge, a magnetic tape, and / or another type of non-temporary computer-readable media. Computer-readable media are defined herein as non-temporary memory devices. A memory device may include memory space within a single physical storage device or memory space extending across multiple physical storage devices.

[0043] According to an exemplary embodiment, storage 140 may be configured to store information such as raw data, metadata, or the like. Furthermore, or alternatively, storage 140 may be configured to store one or more pieces of information associated with one or more operations performed by processor 120. For example, storage 140 may store information defining past operations performed by processor 120 to test vehicle software, one or more results of operations performed by processor 120, or the like. Furthermore, storage 140 may store data or information required when testing vehicle software. For example, storage 140 may store at least a number of default test steps (described below with reference to Figure 4).

[0044] In some implementation, the storage 140 may include multiple storage media, and the storage 140 may be configured to store copies or duplicates of at least a portion of the information in the multiple storage media in order to provide redundancy and to back up the information or associated data. Furthermore, the storage 140 may also store computer-readable instructions or computer-executable instructions that, when executed by one or more processors (e.g., processor 120), cause one or more processors to perform one or more operations / behaviors described herein.

[0045] The processor 120 may include at least one processor that is programmable or configurable to perform functions or operations described herein. For example, the processor 120 may be configured to execute computer executable instructions stored in at least one storage medium or memory storage (e.g., storage 140) to perform one or more operations or actions described herein.

[0046] According to exemplary embodiments, the processor 120 may be configured to receive one or more signals and / or one or more user inputs (e.g., via a communication interface 110, an input / output unit 130, etc.) that define one or more instructions that perform one or more operations. Furthermore, the processor 120 may be implemented in hardware, firmware, or a combination of hardware and software. For example, the processor 120 may include at least one of a central processing unit (CPU), an image processing unit (GPU), an accelerator processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), and / or another type of processing or computing component.

[0047] According to an exemplary embodiment, the processor 120 may be configured to collect, extract, and / or receive one or more pieces of information (such as signals or data), process the received pieces of information, and thereby test the vehicle software.

[0048] The number and arrangement of components shown in Figure 1 are provided as an example. In fact, VST system 100 may include additional components, fewer components, different components, or components arranged differently compared to those shown in Figure 1. Furthermore or alternatively, a set of components of VST system 100 (e.g., one or more components) may perform one or more functions that are described as being performed by another set of components of VST system 100.

[0049] A description of some exemplary operations that the processor 120 may perform is provided below with reference to Figures 4 to 7.

[0050] Examples of vehicle software testing in this disclosure According to an exemplary embodiment, one or more test cases may be designed and operated within a test environment to test vehicle software.

[0051] A test case can refer to a specific set of conditions and / or steps designed to verify the functionality or behavior of software (or the system under test). A test case may define one or more operations and expected outcomes for a particular test scenario. Conditions and / or steps designed to verify the functionality or behavior of software may be referred to as test steps included in a test case. According to exemplary embodiments, one or more test steps may be included in a test case, which may include one or more of the following: a test case ID (e.g., a set of unique identifiers or numbers assigned to a test case for tracking and reference purposes), a test objective (e.g., a description or specification of the goal or objective of the test case), at least one test condition (e.g., prerequisites or initial states required to run the test case, specific settings or configurations required to trigger the test run), at least one test operation (e.g., an operation or action performed to run the test case, required inputs or interactions with the software), at least one expected outcome (e.g., the expected result or behavior expected from the software when the test case runs successfully), and one or more of any other suitable information.

[0052] Figure 2 shows an exemplary test case according to one or more embodiments. Test case 200 may include five test steps, namely test step 1 210, test step 2 220, test step 3 230, test step 4 240, and test step 5 250. It should be understood that the configuration shown in Figure 2 is simplified for illustrative purposes and is not intended in any way to limit the scope of this disclosure. For example, in fact, the number of test steps in a test case may be any number, and the specific content and format of the test steps may be arbitrary.

[0053] As shown in Figure 2, test step 1 210 may include the step “Given SCENARIO_1 Start” which defines the operation to start a test environment with the default conditions and / or elements on which the test case operates (see below for details). Test step 2 220 may include the step “And Turn On ADAS_ECU” which defines the operation to start an advanced driver assistance system (ADAS) electronic control unit (ECU) within the test environment. Similarly, test step 3 230 may include the step “And Turn On IVI_ECU” which defines the operation to start an in-vehicle infotainment (IVI) electronic control unit (ECU) within the test environment.

[0054] Test step 4 240 may include a step defining a conditional scenario in which the obstacle is 2.0 meters away from the front right side of the vehicle: "When OBSTACLE is 2.0 meters away from the FRONT_RIGHT side of the vehicle." Test step 5 250 may include a step defining the expected outcome of the conditional scenario defined in test step 4 240: "Then the IVI indicates that the OBSTACLE is at MAX_DISTANCE from the FRONT_RIGHT side of the vehicle," where the in-vehicle infotainment indicates that the obstacle is at the maximum distance from the front right side of the vehicle.

[0055] A test environment may refer to a specific set of conditions and / or elements, and a test case may operate in relation to those conditions and / or elements. In particular, a test environment may include one or more vehicle elements involved in a test case. One or more vehicle elements may refer to one or more elements associated with one or more components of a vehicle, for example, an electronic control unit (ECU). For example, in a test case designed to test software related to detecting and displaying obstacles near the rear of a vehicle, the test environment may include the vehicle's (one or more vehicle elements) IVI ECU and ADAS ECU. According to an exemplary embodiment, a test environment may further include one or more environmental elements involved in a test case. One or more environmental elements may refer to one or more elements associated with the vehicle's environment. For example, in a test case designed to test software related to detecting and displaying obstacles near the rear of a vehicle, the test environment may further include obstacles near the rear of the vehicle and rain (one or more environmental elements), and as a result, when the vehicle's IVI ECU and ADAS ECU detect and display obstacles near the rear of the vehicle during rain, the test case may be able to evaluate the software associated with the vehicle's IVI ECU and ADAS ECU.

[0056] Figure 3 shows exemplary components of a graphical user interface (GUI) for testing vehicle software according to one or more embodiments.

[0057] As shown in Figure 3, the graphical user interface (GUI) 300 may include at least a first portion 320 and a second portion 340.

[0058] According to an exemplary embodiment, the first part 320 may include a section for entering code. The code may be associated with a test step. For example, a user may manually enter code associated with a test step into the first part in a manner similar to that described above for Figure 2, and as a result, the test step is added to a test case.

[0059] According to an exemplary embodiment, the code associated with the test steps in the first part may be in a natural language format such as the Gherkin language format (i.e., natural language code). According to an exemplary embodiment, the natural language code may be entered into the first part 320 by a first user. According to an exemplary embodiment, the first user may include a test engineer, a test case manager, and the like, who is responsible for designing and reviewing the test cases (and associated test steps) but does not have extensive knowledge of low-level programming.

[0060] Therefore, the above makes it possible for the first user to easily input and review test steps without requiring them to have extensive knowledge of low-level programming, and to design test cases in a behavior-driven manner.

[0061] According to an exemplary embodiment, the second portion 340 may include at least a search area 342 and a test step area 344. Here, it can be understood that the memory storage of the VST system may store a plurality of default test steps. The plurality of default test steps may be test steps predetermined and stored by the software developer, which can be searched by a first user of the GUI 300.

[0062] In particular, the first user may provide a search input to the search area 342, where the VST system may then retrieve (search) one or more test steps associated with the search input from a plurality of stored default test steps, and display the retrieved one or more test steps in the test step area 344.

[0063] According to an exemplary embodiment, the search input may include any type of keyword associated with a test step, such as the name of the test step, the type of the test step, and similar keywords. Furthermore, multiple default test steps may be grouped by type of test step and / or assigned tags to be searched based on the type of test step. For example, test steps associated with CAN communication may be assigned the CAN communication tag and / or grouped into a CAN communication group, so that when a first user enters "CAN" or "CAN communication" in the search area 342 (i.e., the search input includes "CAN" or "CAN communication"), all test steps assigned the CAN communication tag and / or all test steps within the CAN communication group (i.e., all test steps associated with CAN communication) may be displayed in the test step area 344. In another example, test steps describing conditional behavior such as "given," "when," "then," and similar keywords may be assigned corresponding tags for "given," "when," "then," and similar keywords.

[0064] According to an exemplary embodiment, each of a plurality of default test steps may be grouped into a plurality of test step types and / or assigned a plurality of tags to be searched based on the type of test step. For example, a test step describing conditional operation related to CAN communication may be assigned both a corresponding condition tag and a CAN communication tag. Thus, the test step may be displayed when the corresponding condition tag and / or CAN communication tag are entered as search input. Furthermore, if the search input includes a keyword that is assigned both a corresponding condition tag and a CAN communication tag, only test steps that are assigned both a corresponding condition tag and a CAN communication tag may be displayed. Thus, the user can easily find the desired test step.

[0065] Once one or more acquired test steps are displayed in the test step area 344, the first user may then provide a selection input to select one of the one or more test steps displayed in the test step area 344. According to an exemplary embodiment, the first user may provide a selection input by dragging and dropping a test step from the test step area 344 onto the first portion 320. According to an exemplary embodiment, the first user may provide a selection input by clicking on a test step in the test step area 344.

[0066] Furthermore, once a selection input is received from the first user, the VST system may automatically enter the code associated with one of the selected test steps into the first section 320. It can be understood that the code entered into the first section 320 can be edited and deleted as desired by the first user. Thus, the first user does not need to manually type the test steps, and the test steps can be quickly searched and added to the first section 320 (and test cases).

[0067] According to an exemplary embodiment, the first part 320 may also include a window 322. The window 322 may be a pop-up window that appears (pops up) when a first user is typing code into the first part 320.

[0068] In particular, the first user may provide partial natural language code input (i.e., partial code input that is in natural language format) to the first part 320 of the GUI 300. When the first user provides partial natural language code input, the VST system may retrieve one or more test steps associated with the partial natural language code input and display them in window 322. For example, the VST system may display one or more test steps that are most similar to the partial natural language code input in window 322. It can be understood that the test steps displayed in window 322 may be retrieved from a set of stored default test steps in a similar manner to the test steps displayed in the test step area 344.

[0069] The first user may then provide a selection input to select one of the one or more test steps displayed in window 322, where the VST system may then automatically replace (fill in) the partial natural language code input in the first part 320 with the complete code associated with the selected one or more test steps.

[0070] For example, the first user may have typed a first part of a natural language code associated with a particular test step (i.e., a partial natural language code input) into the first part 320, and the VST system may then determine and display in window 322 one test step associated with the code most similar to the first part, or the VST system may determine and display in window 322 several test steps (candidates) associated with the code most similar to the first part (including the particular test step the first user is about to type). Here, the first user may select the particular test step the first user is about to type in window 322, and the VST system may then automatically fill in the rest of the particular test step (replacing the first part of the code already entered in the first part 320 with the code associated with the selected particular test step (the complete code)). Alternatively, the first user may select a different test step in window 322 (for example, initially the first user enters incorrect but similar code), where the VST system may then automatically replace the first part of the code already entered in the first part 320 with the code (complete code) associated with the selected different test step.

[0071] Therefore, the first user does not need to manually type all of the test steps, and the test steps can be quickly searched and added to the first part 320 (and test cases).

[0072] According to an exemplary embodiment, the GUI300 may further include the ability to add new test steps via the GUI to a set of default test steps stored in memory storage when designing test cases.

[0073] In particular, it can be understood that each of the multiple default test steps stored in memory storage may be associated with programming language code and natural language code. Natural language code may refer to code written in a natural language format (e.g., Gherkin), which allows the user to easily type and examine the code without requiring a broad knowledge of low-level programming, but cannot be compiled and executed in simulation. On the other hand, programming language code may refer to code written in a programming language format (e.g., non-natural programming language code such as Python®), which can be compiled and executed in simulation, but requires the user to have a broad knowledge of low-level programming in order to type and examine it.

[0074] Here, "code" can refer to a single line of code / command, and it can be understood that one natural language code may correspond to one or more programming language codes. Therefore, in order to execute a test step, the natural language code associated with the test step in the first part 320 may be executed by executing the corresponding one or more programming language codes. Thus, default test steps may be stored and added by storing the natural language code associated with the default test step, along with the corresponding one or more programming language codes.

[0075] According to an exemplary embodiment, in order to add a new test step, a first user may first input one or more natural language codes into a first part. These one or more natural language codes may correspond to one or more test steps that the first user wants to execute.

[0076] When one or more natural language codes are received from a first user, the system may determine whether the received one or more natural language codes include any new natural language codes that do not correspond to any of the one or more programming language codes stored in memory storage.

[0077] In particular, for example, a first user may input three natural language codes corresponding to three test steps that the first user wants to execute. In this respect, the first and second natural language codes are the same as the natural language codes stored in memory storage and may have corresponding programming language codes stored in memory storage (i.e., the first and second test steps (the first and second natural language codes and their corresponding programming language codes) correspond to default test steps already stored in memory storage). Therefore, the first and second natural language codes can be executed by executing their corresponding programming language codes. On the other hand, a third natural language code may be new and not correspond to any of the natural language codes or programming language codes stored in memory storage (i.e., the third test step (the third natural language code) is new and needs to be added). Therefore, the third natural language code is not executable because there is no corresponding executable programming language code. Therefore, the programming language code corresponding to the third natural language code may need to be added in order to add the third test step.

[0078] According to an exemplary embodiment, the system may determine whether the received natural language code includes any new natural language code by simply executing the received natural language code, and may also determine whether any of the received natural language code cannot be executed due to the lack of a corresponding programming language code.

[0079] In this regard, depending on whether the system has determined that one or more received natural language codes include one or more new natural language codes, it may send one or more new natural language codes to a second user. The second user may be a software developer or similar with broad knowledge of low-level programming. Thus, the second user may develop and prepare a programming language corresponding to one or more new natural language codes and then provide that programming language to the system for execution.

[0080] According to exemplary embodiments, the GUI 300 may further include additional options (not shown) for adding test steps. In particular, according to exemplary embodiments, a first user may simply input one or more new natural language codes (i.e., natural language codes that do not have corresponding programming language codes) into the first section. The first user may then select an additional option, where the system may then send one or more new natural language codes to a second user and receive the corresponding new programming language codes in a manner similar to that described above. According to exemplary embodiments, the GUI 300 may further include a new code section (not shown). The new code section may include a section for inputting new code. In particular, according to exemplary embodiments, a first user may simply input one or more new natural language codes (i.e., natural language codes that do not have corresponding programming language codes) into the new code section. The system may then automatically send one or more new natural language codes in the new code section to a second user and receive the corresponding new programming language codes in a manner similar to that described above.

[0081] When new programming language code (i.e., programming code corresponding to a new natural language code) is received from a second user, the system may store the new programming language code and the new natural language code in memory storage as a new default test step. In this regard, the new default test step may be retrieved and displayed in the same manner as described above (in the test step area 344 and / or in window 322).

[0082] Furthermore, the system can also check and verify whether one or more received natural language codes (including new natural language codes) can now be successfully executed by the newly received programming language code.

[0083] Furthermore, the system may also send a notification to the first user indicating that new programming language code has been received.

[0084] Therefore, the first user may provide instructions to execute one or more received natural language codes (i.e., natural language codes provided by the first user in the first part, including new natural language codes). Upon receiving instructions, the system may then execute one or more received natural language codes by executing the corresponding programming language codes (which should now succeed, since all of the received natural language codes should have corresponding programming language codes). It can be understood that the system may utilize any appropriate functions and elements, such as APIs (e.g., the Python® API), context objects, and the like, to facilitate the execution of natural language codes and programming language codes.

[0085] Therefore, the above process enables users without extensive knowledge of low-level programming to easily design and review test cases, while simultaneously providing such users with the flexibility to request the creation of new test steps.

[0086] According to an exemplary embodiment, the GUI300 may further include several options (not shown) that constitute the execution of a test case.

[0087] In particular, in order to run (execute) test cases in a test environment, test cases may run online by utilizing a continuous integration and continuous delivery (CICD) framework, and as a result, test cases may be uploaded to a server and accessed by a group of users. Accordingly, according to an exemplary embodiment, GUI300 may further include a first operation option and a second operation option. In response to receiving a selection input from a first user to select the first operation option, the VST system may upload and run (execute) a test case containing one or more test steps in the first part 320 online by utilizing the CICD framework. On the other hand, in response to receiving a selection input from a first user to select the second operation option, the VST system may run a test case containing one or more test steps in the first part 320 offline without utilizing the CICD framework.

[0088] Therefore, a VST system can enable users to quickly switch between online execution, where test cases can be uploaded and accessed by groups of other users, and offline execution, where test cases can be run quickly while avoiding the risk of uploading undesirable test cases to the server.

[0089] Furthermore, in order to run a test case in a test environment, a simulated test environment may be generated for each action (each execution) of the test case. Once a simulated test environment is generated for the actions of a test case, the simulated test environment may be terminated after the test case is completed. Accordingly, according to an exemplary embodiment, the GUI 300 may further include a third action option and a fourth action option. In response to receiving a selection input from a first user to select the third action option, the VST system may maintain the simulated test environment after executing a test case comprising one or more test steps in the first part 320. On the other hand, in response to receiving a selection input from a first user to select the fourth action option, the VST system may terminate the simulated test environment after executing a test case comprising one or more test steps in the first part 320.

[0090] Therefore, the VST system may allow users to quickly switch between runs that maintain the simulated test environment, enabling subsequent test cases to be executed quickly within the same simulated test environment without having to restart the simulated test environment for each execution of the same or different test cases, and runs that terminate the simulated test environment, saving resources.

[0091] It should be understood that the configuration shown in Figure 3 is simplified for illustrative purposes and is not intended in any way to limit the scope of this disclosure. For example, the relative position, relative size, and / or shape of each element within GUI300 may actually be arbitrary.

[0092] Exemplary behavior for testing vehicle software in this disclosure Some exemplary operations that can be performed by the VST system of this disclosure are described below with reference to Figures 4 to 7.

[0093] Figure 4 shows a flowchart of an exemplary method 400 for testing vehicle software according to one or more embodiments. One or more operations in method 400 may be performed by at least one processor (e.g., processor 120) of the VST system.

[0094] As shown in Figure 4, in operation S410, at least one processor may be configured to receive one or more natural language codes. One or more natural language codes may be received from a first user and may refer to one or more codes written in natural language format. For example, one or more natural language codes may be in Gherkin programming language format. The method then proceeds to operation S420.

[0095] In operation S420, at least one processor may be configured to determine whether one or more received natural language codes include one or more new natural language codes that do not correspond to any of the one or more programming language codes stored in memory storage. One or more programming language codes may refer to one or more codes written in a programming language format (e.g., a non-natural language format). For example, one or more programming language codes may be in the Python® programming language format.

[0096] According to an exemplary embodiment, at least one processor may be configured to determine whether a received set of natural language codes includes one or more new natural language codes by executing one or more received natural language codes, determining that one or more received natural language codes cannot be executed, and identifying one or more codes from the one or more received natural language codes that cannot be executed.

[0097] According to an exemplary embodiment, one or more received natural language codes can be executed by executing one or more corresponding programming language codes. Therefore, according to an exemplary embodiment, one or more received natural language codes may be unexecutable because one or more codes from the (unexecutable) received natural language codes do not have one or more corresponding programming language codes to execute. In this regard, one or more codes from the (unexecutable) received natural language codes may correspond to one or more new natural language codes. The method then proceeds to operation S430.

[0098] In operation S430, depending on the determination that one or more received natural language codes include one or more new natural language codes, at least one processor may be configured to transmit one or more new natural language codes. One or more new natural language codes may be transmitted to a second user. The method then proceeds to operation S440.

[0099] In operation S440, at least one processor may be configured to receive one or more new programming language codes. One or more new programming language codes may be received from a second user and may correspond to one or more new natural language codes.

[0100] According to an exemplary embodiment, at least one processor may also be configured to store one or more new programming language codes and corresponding one or more new natural language codes in memory storage as a default test step.

[0101] According to an exemplary embodiment, at least one processor may also be configured to determine whether one or more received natural language codes are successfully executed, in response to having received one or more new programming language codes.

[0102] According to exemplary embodiments, at least one processor may be configured to generate a graphical user interface. According to exemplary embodiments, the graphical user interface may include at least a first part having a section for inputting code. According to exemplary embodiments, one or more natural language codes may be received in the first part.

[0103] According to an exemplary embodiment, at least one processor may also be configured to send a notification to a first user indicating that one or more new programming language codes have been received, in response to receiving one or more new programming language codes.

[0104] When operation S440 is performed, method 400 may terminate or end. Alternatively, method 400 may return to operation S410, as a result, at least one processor may be configured to repeatedly, for at least a predetermined amount of time, receive one or more natural language codes (in operation S410), determine (in operation S420) whether the received one or more natural language codes include one or more new natural language codes, transmit one or more new natural language codes (in operation S430), and receive one or more new programming language codes (in operation S440).

[0105] For example, a first user may provide more natural language codes (which may include more new natural language codes) sequentially (or periodically). Thus, at least one processor may receive one or more natural language codes sequentially (or periodically), and then resume receiving one or more natural language codes (in operation S410), determining (in operation S420) whether the received one or more natural language codes include one or more new natural language codes, transmitting one or more new natural language codes (in operation S430), and receiving one or more new programming language codes (in operation S440).

[0106] Figure 5 shows a flowchart of an exemplary method 500 for testing vehicle software according to one or more embodiments. One or more operations in method 500 may be performed by at least one processor (e.g., processor 120) of the VST system.

[0107] As shown in Figure 5, in operation S510, at least one processor may be configured to receive one or more new natural language codes. In a manner similar to that described above for method 400, one or more new natural language codes may refer to one or more codes that can be received from a first user, are written in a natural language format (e.g., Gherkin), and do not correspond to any of the one or more programming language codes stored in memory storage. The method then proceeds to operation S520.

[0108] In operation S520, at least one processor may be configured to determine (check) whether one or more new natural language codes received correspond to one or more programming language codes stored in memory storage. According to an exemplary embodiment, at least one processor may be configured to determine whether one or more new natural language codes received correspond to one or more programming language codes stored in memory storage by searching and analyzing one or more programming language codes stored in memory storage.

[0109] Therefore, depending on the determination that one or more received new natural language codes correspond to one or more programming language codes stored in memory storage, at least one processor may determine that it does not need to obtain a new programming language code corresponding to the one or more received new natural language codes, and the method proceeds to termination. On the other hand, depending on the determination that one or more received new natural language codes do not correspond to any of the one or more programming language codes stored in memory storage, at least one processor may determine that it does need to obtain a new programming language code corresponding to the one or more received new natural language codes, and the method proceeds to operation S530.

[0110] In operation S530, at least one processor may be configured to send one or more new natural language codes. One or more new natural language codes may be sent to a second user. The method then proceeds to operation S540.

[0111] In operation S540, at least one processor may be configured to receive one or more new programming language codes. One or more new programming language codes may be received from a second user and may correspond to one or more new natural language codes.

[0112] According to an exemplary embodiment, at least one processor may also be configured to store one or more new programming language codes and corresponding one or more new natural language codes in memory storage as a default test step.

[0113] According to an exemplary embodiment, at least one processor may also be configured to determine whether one or more received natural language codes are successfully executed, in response to having received one or more new programming language codes.

[0114] According to an exemplary embodiment, at least one processor may be configured to generate a graphical user interface. According to an exemplary embodiment, the graphical user interface may include at least a first part having a section for entering code, and an additional option for adding a new test step. According to an exemplary embodiment, one or more new natural language codes may be received in the first part. According to an exemplary embodiment, one or more new natural language codes may be transmitted in response to receiving a selection input from a first user for selecting an additional option. According to an exemplary embodiment, the selection input may include a click input.

[0115] According to an exemplary embodiment, the graphical user interface may include at least a new code section having a portion for inputting new code. According to an exemplary embodiment, one or more new natural language codes may be received in the new code section. According to an exemplary embodiment, one or more new natural language codes may be transmitted in response to the receipt of one or more new natural language codes from a first user in the new code section.

[0116] According to an exemplary embodiment, at least one processor may also be configured to send a notification to a first user indicating that one or more new programming language codes have been received, in response to receiving one or more new programming language codes.

[0117] When operation S540 is performed, method 500 may terminate or end. Alternatively, method 500 may return to operation S510, as a result, at least one processor may be configured to repeatedly (in operation S510) receive one or more new natural language codes, (in operation S520) determine whether the received one or more new natural language codes correspond to one or more programming language codes stored in memory storage, (in operation S530) transmit one or more new natural language codes, and (in operation S540) receive one or more new programming language codes for at least a predetermined amount of time.

[0118] For example, the first user may provide more new natural language codes sequentially (or periodically). Thus, at least one processor may receive one or more new natural language codes sequentially (or periodically), and then resume receiving one or more new natural language codes (in operation S510), determining (in operation S520) whether the received one or more new natural language codes correspond to one or more programming language codes stored in memory storage, transmitting one or more new natural language codes (in operation S530), and receiving one or more new programming language codes (in operation S540).

[0119] Figure 6 shows a flowchart of an exemplary method 600 for testing vehicle software according to one or more embodiments. One or more operations in method 600 may be performed by at least one processor (e.g., processor 120) of the VST system.

[0120] As shown in Figure 6, in operation S610, at least one processor may be configured to generate a graphical user interface. The graphical user interface may include at least a first and a second part, where the first part may include a code input area, and the second part may include a search area and a test step area. The method then proceeds to operation S620.

[0121] In operation S620, at least one processor may be configured to receive a search input. The search input may be received from a first user in the search area. The method then proceeds to operation S630.

[0122] In operation S630, at least one processor may be configured to display one or more test steps associated with a search input. According to an exemplary embodiment, one or more test steps may include one or more of the following: a test case ID (e.g., a set of unique identifiers or numbers assigned to a test case for tracking and reference purposes), a test objective (e.g., a description or specification of the goal or purpose of the test case), at least one test condition (e.g., prerequisites or initial states required to run the test case, specific settings or configurations required to trigger the test run), at least one test behavior (e.g., an operation or action performed to run the test case, required inputs or interactions with software), at least one expected result (e.g., an expected result or behavior from the software when the test case runs successfully), and any other suitable information. Furthermore, the test steps may be associated with natural language code and programming language code and may be displayed in the test step area.

[0123] According to an exemplary embodiment, at least one processor may be configured to display one or more test steps associated with a search input by retrieving (searching for) one or more test steps based on a search input, and then displaying the retrieved one or more test steps.

[0124] According to an exemplary embodiment, one or more test steps may be retrieved based on a search input by determining one or more keywords from the search input. For example, one or more keywords in the search input may include the name of a test step, the type of a test step, and similar ones. According to an exemplary embodiment, one or more test steps may further be retrieved based on a search input by retrieving one or more test steps associated with the determined one or more keywords. For example, at least one processor may be configured to retrieve one or more test steps having names similar to the names of the test steps included in the search input. In another example, at least one processor may be configured to retrieve one or more test steps that are in a group, or one or more test steps having tags similar to the types of test steps included in the search input.

[0125] According to an exemplary embodiment, the memory storage of the VST system may include a plurality of default test steps, and one or more test steps (i.e., one or more test steps associated with a search input) may be obtained from the plurality of default test steps. The method then proceeds to operation S640.

[0126] In operation S640, at least one processor may be configured to receive a selection input. The selection input may be received from a first user and may relate to selecting one of one or more test steps displayed in the test step area. According to an exemplary embodiment, the selection input may include a drag-and-drop input. According to an exemplary embodiment, the selection input may include a click input. The method then proceeds to operation S650.

[0127] In operation S650, at least one processor may be configured to input code associated with one of a selected test step into the first part. According to an exemplary embodiment, the code in the first part (i.e., code manually entered by a first user, and code automatically entered by the VST system in the first part) may be in natural language format (i.e., natural language code).

[0128] According to an exemplary embodiment, once code associated with one or more selected test steps is entered into the first part, the user can edit and modify the entered code as desired. For example, the user can edit a placeholder part of the code (e.g., "[duration]") to a specific number (e.g., "

[10] ") in order to add a specific value for a parameter.

[0129] When operation S650 is performed, method 600 may terminate or end. Alternatively, method 600 may return to operation S620, as a result, at least one processor may be configured to repeatedly receive a search input (in operation S620), display one or more test steps (in operation S630), receive a selection input (in operation S640), and enter a code (in operation S650) for at least a predetermined amount of time. For example, a first user may search for more test steps sequentially (or periodically). Thus, at least one processor may receive a search input sequentially (or periodically) and then resume receiving a search input (in operation S620), displaying one or more test steps (in operation S630), receiving a selection input (in operation S640), and entering a code (in operation S650).

[0130] Figure 7 shows a flowchart of an exemplary method 700 for testing vehicle software according to one or more embodiments. One or more operations in method 700 may be performed by at least one processor (e.g., processor 120) of the VST system.

[0131] As shown in Figure 7, in operation S710, at least one processor may be configured to generate a graphical user interface. The graphical user interface may include at least a first part, where the first part may include a code input section and a pop-up window. The method then proceeds to operation S720.

[0132] In operation S720, at least one processor may be configured to receive partial code input. Partial code input may be received from a first user in a first part. Furthermore, it may be understood that partial code input may be part of the code (complete code) associated with the test step. The method then proceeds to operation S730.

[0133] In operation S730, at least one processor may be configured to display one or more test steps associated with a partial code input. One or more test steps may be displayed in a pop-up window.

[0134] According to an exemplary embodiment, at least one processor may be configured to display one or more test steps associated with a partial code input by retrieving (searching for) one or more test steps based on a partial code input, and then displaying the retrieved one or more test steps.

[0135] According to an exemplary embodiment, one or more test steps may be obtained based on a partial code input by determining one or more test steps that are most similar to the partial code input. For example, a first user may have typed a first part of code associated with a particular test step (i.e., a partial code input) into a first part, and the VST system may then determine one or more test steps associated with the code that are most similar to the first part. The method then proceeds to operation S740.

[0136] In operation S740, at least one processor may be configured to receive a selection input. The selection input may be received from a first user and may relate to selecting one of one or more test steps displayed in a pop-up window. The method then proceeds to operation S750.

[0137] In operation S750, at least one processor may be configured to replace a partial code input in the first part with a code (i.e., a complete code) associated with one of a selected test step of one or more. According to an exemplary embodiment, the code in the first part (i.e., a partial code manually entered by a first user, and a complete code automatically entered by the VST system in the first part) may be in natural language format (i.e., a partial natural language code and a complete natural language code).

[0138] According to an exemplary embodiment, if code associated with one of one or more selected test steps is replaced in the first part with a partial code input, the user can edit and modify the replaced code as desired. For example, the user can edit a placeholder portion of the code (e.g., "[duration]") to a specific number (e.g., "

[10] ") in order to add a specific value for a parameter.

[0139] When operation S750 is performed, method 700 may terminate or end. Alternatively, method 700 may return to operation S720, as a result, at least one processor may be configured to repeatedly receive a partial code input (in operation S720), display one or more test steps (in operation S730), receive a selection input (in operation S740), and replace a partial code input (in operation S750) for at least a predetermined amount of time.

[0140] For example, the first user may type more code into the first part sequentially (or periodically). Thus, at least one processor may receive partial code input sequentially (or periodically) and then resume receiving partial code input (in operation S720), displaying one or more test steps (in operation S730), receiving selection input (in operation S740), and replacing partial code input (in operation S750).

[0141] According to exemplary embodiments, one or more operations in methods 400, 500, 600, and 700 can be combined with one another.

[0142] For example, the system may generate a graphical user interface, and the first user may first search for and select a first test step via a search area and a test step area (where operations S610 to S650 are performed), and then type a partial code and select code for a second test step in a pop-up window (where operations S720 to S750 are performed).

[0143] In another example, the system may generate a graphical user interface, and a first user may first search for and select a first test step via a search area and a test step area (where operations S610 to S650 are performed). The system may then receive a new natural language code from the first user associated with a second test step in the first part / new code section of the graphical user interface, send the new natural language code to a second user, and receive one or more new programming language codes corresponding to the new natural language code (where operations S510 to S530 are performed).

[0144] Various embodiments The foregoing disclosures are illustrative and illustrative, but are not intended to be exhaustive or to limit implementations to the exact forms disclosed. Modifications and variations may be possible in view of the foregoing disclosures or may be derived from implementations.

[0145] Some embodiments may relate to systems, methods, and / or computer-readable media at any possible level of technical detail of integration. Furthermore, one or more of the above-described components may be implemented as instructions stored in a computer-readable medium and executable by at least one processor (and / or may include at least one processor). The computer-readable medium may include a computer-readable non-temporary storage medium (or multiple mediums) having computer-readable program instructions for causing a processor to perform an operation.

[0146] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction-executing device. A computer-readable storage medium may be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any preferred combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes, namely, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or grooved raised structures on which instructions are recorded, and any preferred combination thereof. The computer-readable storage media used herein should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmitting media (e.g., light pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0147] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device, or they may be downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0148] The computer-readable program code / instructions that perform the operation may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, or similar, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may be fully executed on the user's computer, partially executed on the user's computer, executed as a standalone software package, partially executed on the user's computer and partially executed on a remote computer, or fully executed on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), or a connection to an external computer may be made (for example, via the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer-readable program instructions by personalizing the electronic circuit using state information of computer-readable program instructions in order to perform a particular action or operation.

[0149] The computer-readable program instructions may be provided to a processor of a general-purpose computer, a dedicated computer, or other programmable data processing device to generate a machine, and as a result, the instructions executed via the processor of the computer or other programmable data processing device create means for implementing functions / actions specified in blocks or blocks of a flowchart and / or block diagram. The computer-readable program instructions may also be stored in a computer-readable storage medium that can instruct computers, programmable data processing devices, and / or other devices to function in a particular way, and as a result, the computer-readable storage medium in which the instructions are stored comprises a manufactured article containing instructions for implementing modes of functions / actions specified in blocks or blocks of a flowchart and / or block diagram.

[0150] Computer-readable program instructions can also be loaded onto a computer, other programmable data processing device, or other device to perform a series of operational steps on the computer, other programmable device, or other device, thereby generating a computer implementation process, the instructions executed on the computer, other programmable device, or other device, which implement the functions / actions specified in the blocks or blocks(s) of a flowchart and / or block diagram.

[0151] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable media according to various embodiments. In this regard, each block in a flowchart or block diagram may represent a microservice module, segment, or part of an instruction set comprising one or more executable instructions that implement a specified logical function. Methods, computer systems, and computer-readable media may include additional blocks, fewer blocks, different blocks, or blocks in different arrangements compared to those depicted in the figures. In some alternative implementations, the functions described in the blocks may occur regardless of the order in which they are shown in the figures. For example, two blocks shown consecutively may actually be executed simultaneously or substantially simultaneously, or blocks may be executed in reverse order depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs a specified function or action or executes a dedicated combination of hardware and computer instructions.

[0152] It will be apparent that the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or combinations of hardware and software. The actual dedicated control hardware or software code used to implement such systems and / or methods is not a limitation of the implementation. Therefore, the operation and behavior of the systems and / or methods are described herein without reference to specific software code, and it is understood that software and hardware can be designed to implement the systems and / or methods based on the descriptions herein.

[0153] It can be understood that many modifications and variations of this disclosure are possible in light of the teachings above. Within the scope of the attached sections, it will be apparent that this disclosure may be implemented in ways other than those specifically described herein.

Claims

1. A system for testing vehicle software, wherein the system is Memory storage that stores executable computer instructions, At least one processor that is communicatively connected to the memory storage, The processor comprises the above, and the at least one processor executes the instructions, One or more natural language codes are received from the first user, Determine whether the received natural language code includes one or more new natural language codes that do not correspond to any of the one or more programming language codes stored in the memory storage. In response to determining that the received one or more natural language codes include the one or more new natural language codes, the one or more new natural language codes are transmitted to a second user. One or more new programming language codes are received from the second user. It is configured in such a way, The system wherein the one or more new programming language codes received correspond to the one or more new natural language codes.

2. The system according to claim 1, wherein the at least one processor is further configured to execute the instructions and store the one or more new programming language codes and the one or more corresponding new natural language codes in the memory storage as a default test step.

3. The at least one processor executes the instruction, The process involves executing one or more received natural language codes, which are executed by executing one or more corresponding programming language codes. The determination that one or more of the received natural language codes cannot be executed, Identifying one or more codes from the one or more received natural language codes that cannot be executed, The system according to claim 1 or 2, configured to determine whether the received one or more natural language codes include the one or more new natural language codes.

4. The system according to claim 1 or 2, wherein the at least one processor is further configured to execute the instruction and, in response to receiving the one or more new programming language codes, determine whether the one or more received natural language codes are successfully executed.

5. The at least one processor further executes the instruction: To generate a graphical user interface comprising at least a first part and a second part, wherein the first part comprises a code input area, and the second part comprises a search area and a test step area, Receiving search input from the first user in the aforementioned search area, Display one or more test steps associated with the search input in the test step area, The first user provides a selection input to select one of the one or more test steps displayed in the test step area, Inputting a natural language code associated with one of the selected test steps into the first part, The system according to claim 1 or 2, configured to perform the following:

6. The system according to claim 5, wherein the one or more natural language codes are received in the first part.

7. The memory storage includes a number of default test steps, Each of the aforementioned set of default test steps is associated with a natural language code and one or more corresponding programming language codes. The system according to claim 5, wherein the one or more test steps associated with the search input are obtained from the plurality of default test steps.

8. The first part further comprises a pop-up window, The at least one processor further executes the instruction: In the first part, a partial natural language code input is received from the first user. One or more test steps associated with the aforementioned partial natural language code input are displayed in the pop-up window. The system receives a selection input from the first user, allowing the user to select one of the one or more test steps displayed in the pop-up window. The system according to claim 5, wherein the first part is configured to replace the partial natural language code input with a natural language code associated with one of the selected test steps.

9. The system according to claim 1 or 2, wherein the one or more natural language codes are in the Gherkin programming language format, and the one or more programming language codes are in the Python® programming language format.

10. A method performed by a system for testing vehicle software, Receiving one or more natural language codes from the first user, Determining whether the received natural language code includes one or more new natural language codes that do not correspond to any of the one or more programming language codes stored in the system's memory storage, In response to determining that the received one or more natural language codes include the one or more new natural language codes, the one or more new natural language codes are transmitted to a second user. Receiving one or more new programming language codes from the second user, Includes, A method wherein the received one or more new programming language codes correspond to the one or more new natural language codes.

11. The method according to claim 10, further comprising storing the one or more new programming language codes and the one or more corresponding new natural language codes in the memory storage as a default test step.

12. The determination of whether the received one or more natural language codes include the one or more new natural language codes is, The process involves executing one or more received natural language codes, which are executed by executing one or more corresponding programming language codes. The determination that one or more of the received natural language codes cannot be executed, Identifying one or more codes from the one or more received natural language codes that cannot be executed, The method according to claim 10 or 11, including the method described in claim 10 or 11.

13. The method according to claim 10 or 11, further comprising determining whether the received one or more natural language codes are successfully executed in response to the receipt of the one or more new programming language codes.

14. To generate a graphical user interface comprising at least a first part and a second part, wherein the first part comprises a code input area, and the second part comprises a search area and a test step area, Receiving search input from the first user in the aforementioned search area, Display one or more test steps associated with the search input in the test step area, The first user provides a selection input to select one of the one or more test steps displayed in the test step area, Inputting a natural language code associated with one of the selected test steps into the first part, The method according to claim 10 or 11, further comprising:

15. The method according to claim 14, wherein the one or more natural language codes are received in the first part.

16. The memory storage includes a number of default test steps, Each of the aforementioned set of default test steps is associated with a natural language code and one or more corresponding programming language codes. The method according to claim 14, wherein the one or more test steps associated with the search input are obtained from the plurality of default test steps.

17. The first part further comprises a pop-up window, The aforementioned method, In the first part, partial natural language code input is received from the first user, Displaying one or more test steps associated with the aforementioned partial natural language code input in the pop-up window, The first user provides a selection input to select one of the one or more test steps displayed in the pop-up window, In the first part, the partial natural language code input is replaced with a natural language code associated with one of the selected test steps, The method according to claim 14, further comprising:

18. The method according to claim 10 or 11, wherein the one or more natural language codes are in the Gherkin programming language format, and the one or more programming language codes are in the Python® programming language format.

19. A computer program for causing at least one processor to perform a method, wherein the method is Receiving one or more natural language codes from the first user, Determining whether the received natural language code includes one or more new natural language codes that do not correspond to any of the one or more programming language codes stored in the memory storage of the system performing the method, In response to determining that the received one or more natural language codes include the one or more new natural language codes, the one or more new natural language codes are transmitted to a second user. Receiving one or more new programming language codes from the second user, Includes, The one or more new programming language codes received are computer programs that correspond to the one or more new natural language codes.

20. The determination of whether the received one or more natural language codes include the one or more new natural language codes is, The process involves executing one or more received natural language codes, which are executed by executing one or more corresponding programming language codes. The determination that one or more of the received natural language codes cannot be executed, Identifying one or more codes from the one or more received natural language codes that cannot be executed, The computer program according to claim 19, including the computer program described in claim 19.

Images