System, method and computer program product for testing software of vehicle

Abstract

The present disclosure relates to the provision of a system. Systems, methods, and computer program products for testing vehicle software are provided. According to an exemplary embodiment, a system may include: a storage store storing computer executable instructions; and at least one processor communicatively coupled to the storage, the at least one processor may be configured to execute instructions to: receive one or more natural language codes from a first user; determining whether the received one or more natural language codes include one or more new natural language codes that do not correspond to any one of the one or more programming language codes stored in the storage; in response to determining that the received more than one natural language code contains more than one new natural language code, sending the more than one new natural language code to a second user; and receiving one or more new programming language codes from the second user.

Description

Technical Field

[0001] Systems, methods, and computer program products that match the exemplary embodiments of this disclosure relate to vehicle software testing, and more specifically, to writing test steps in vehicle software testing. Background Technology

[0002] In software development, software needs to be tested to ensure that it functions as expected, meets specified requirements, and reliably performs its functions in various scenarios. Software testing is an important part of the Software Development Life Cycle (SDLC). Before deploying software in a real system, testing is conducted to identify defects, errors, or bugs in the software.

[0003] Testing vehicle-related software can be performed by designing test cases that include multiple test steps and operate within a test environment. The results of these test cases are then generated and can be viewed by users to evaluate the software.

[0004] In related technologies, test cases are designed by inputting and typing test steps using programming languages ​​such as Python. However, this process requires users to have extensive knowledge of the relevant programming language, and users without such knowledge may find it difficult to design test cases, thus increasing time and the risk of errors. Therefore, there is a need for a system that allows users to design test cases quickly and efficiently without requiring knowledge of programming languages. Summary of the Invention

[0005] The exemplary embodiments of this disclosure test vehicle software. Therefore, the exemplary embodiments of this disclosure enable users without extensive knowledge of low-level programming to easily design and research test cases, while also providing flexibility for such users to create new test steps upon request. Furthermore, the exemplary embodiments of this disclosure allow test steps to be added to test cases quickly and efficiently, without requiring users to manually type such test steps.

[0006] According to an exemplary embodiment, a system is provided. The system may relate to testing software of a vehicle. The system may include: a storage device storing computer-executable instructions; and at least one processor communicatively connected to the storage device, the at least one processor being configured to execute the instructions to perform the following actions: receiving one or more natural language codes from a first user; determining whether the received one or more natural language codes contain one or more new natural language codes that do not correspond to any of the one or more programming language codes stored in the storage device; in response to determining that the received one or more natural language codes contain one or more new natural language codes, sending one or more new natural language codes to a second user; and receiving one or more new programming language codes from the second user, the received one or more new programming language codes corresponding to one or more new natural language codes.

[0007] According to an exemplary implementation, at least one processor may also be configured to execute instructions to store one or more new programming language codes and corresponding one or more new natural language codes as predetermined test steps in a memory.

[0008] According to an exemplary implementation, at least one processor may be configured to determine whether one or more received natural language codes contain one or more new natural language codes by executing instructions to: execute one or more received natural language codes, wherein the one or more received natural language codes are executed by executing one or more corresponding programming language codes; determine that one or more received natural language codes cannot be executed; and identify one or more codes from the one or more received natural language codes that cannot be executed.

[0009] According to an exemplary implementation, at least one processor may also be configured to execute instructions to perform the following actions: in response to receiving more than one new programming language code, determining whether the received more than one natural language code is being executed correctly.

[0010] According to an exemplary embodiment, at least one processor may also be configured to execute instructions to perform the following actions: generating a graphical user interface having at least a first part and a second part, wherein the first part may include a part for inputting code, and the second part may include a search area and a test step area; receiving search input from a first user in the search area; displaying one or more test steps associated with the search input in the test step area; receiving selection input from the first user for selecting one of the test steps displayed in the test step area; and inputting natural language code associated with the selected test step from the one or more test steps into the first part.

[0011] According to an exemplary implementation, more than one natural language code may be received in the first part.

[0012] According to an exemplary implementation, the storage device may include a plurality of predetermined test steps, each of which may be associated with natural language code and one or more corresponding programming language codes, and the one or more test steps associated with the retrieval input may be obtained from the plurality of predetermined test steps.

[0013] According to an exemplary embodiment, the first part may further include a pop-up window, and at least one processor may be configured to execute instructions to perform the following actions: receiving partial natural language code input from a first user in the first part; displaying one or more test steps associated with the partial natural language code input in the pop-up window; receiving selection input from the first user to select one of the test steps displayed in the pop-up window; and replacing the partial natural language code input with natural language code associated with the selected test step from the one or more test steps in the first part.

[0014] According to an exemplary implementation, one or more natural language codes may be in the Gherkin programming language format, and one or more programming language codes may be in the Python programming language format.

[0015] According to an exemplary implementation, a method is provided. The method, which can be performed by a system for testing software of a vehicle, may include: receiving one or more natural language codes from a first user; determining whether the received one or more natural language codes contain one or more new natural language codes that do not correspond to any of the one or more programming language codes stored in the system's storage; in response to determining that the received one or more natural language codes contain one or more new natural language codes, sending one or more new natural language codes to a second user; and receiving one or more new programming language codes from the second user, wherein the received one or more new programming language codes may correspond to one or more new natural language codes.

[0016] According to an exemplary implementation, the method may further include: storing one or more new programming language codes and corresponding one or more new natural language codes as predetermined test steps in a storage device.

[0017] According to an exemplary implementation, determining whether one or more received natural language codes contain one or more new natural language codes may include: executing one or more received natural language codes, wherein the one or more received natural language codes are executed by executing one or more corresponding programming language codes; determining that one or more received natural language codes cannot be executed; and identifying one or more codes from the one or more received natural language codes that cannot be executed.

[0018] According to an exemplary implementation, the method may further include: in response to receiving one or more new programming language codes, determining whether the received one or more natural language codes are executed normally.

[0019] According to an exemplary embodiment, the method may further include: generating a graphical user interface having at least a first part and a second part, wherein the first part may include a part for inputting code, and the second part may include a search area and a test step area; receiving search input from a first user in the search area; displaying one or more test steps associated with the search input in the test step area; receiving selection input from the first user for selecting one of the test steps displayed in the test step area; and inputting natural language code associated with the selected test step from the one or more test steps into the first part.

[0020] According to an exemplary implementation, more than one natural language code may be received in the first part.

[0021] According to an exemplary implementation, the storage device may include a plurality of predetermined test steps, each of which may be associated with natural language code and one or more corresponding programming language codes, and the one or more test steps associated with the retrieval input may be obtained from the plurality of predetermined test steps.

[0022] According to an exemplary embodiment, the first part may further include a pop-up window, and the method may further include: receiving partial natural language code input from a first user in the first part; displaying one or more test steps associated with the partial natural language code input in the pop-up window; receiving selection input from the first user to select one of the test steps displayed in the pop-up window; and in the first part, replacing the partial natural language code input with natural language code associated with the selected test step from the one or more test steps.

[0023] According to an exemplary implementation, one or more natural language codes may be in the Gherkin programming language format, and one or more programming language codes may be in the Python programming language format.

[0024] According to an exemplary embodiment, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium may record instructions executable by at least one processor for causing at least one processor to perform a method. The method includes: receiving one or more natural language codes from a first user; determining whether the received one or more natural language codes contain one or more new natural language codes that do not correspond to any of the one or more programming language codes stored in the storage memory of the system executing the method; in response to determining that the received one or more natural language codes contain one or more new natural language codes, sending one or more new natural language codes to a second user; and receiving one or more new programming language codes from the second user, wherein the received one or more new programming language codes may correspond to one or more new natural language codes.

[0025] According to an exemplary implementation, determining whether one or more received natural language codes contain one or more new natural language codes may include: executing one or more received natural language codes, wherein the one or more received natural language codes are executed by executing one or more corresponding programming language codes; determining that one or more received natural language codes cannot be executed; and identifying one or more codes from the one or more received natural language codes that cannot be executed.

[0026] Additional solutions will be described in part in the following description, and may become apparent from the description or may be implemented by practicing the embodiments presented in this disclosure. Attached Figure Description

[0027] Hereinafter, the features, advantages and importance of preferred embodiments of the present disclosure will be described with reference to the accompanying drawings, in which the same reference numerals denote the same elements.

[0028] Figure 1 A block diagram illustrating exemplary components of a vehicle software testing (VST) system with one or more implementations.

[0029] Figure 2 Exemplary test cases representing more than one implementation.

[0030] Figure 3 Exemplary components of a graphical user interface (GUI) for testing vehicle software, representing one or more implementations.

[0031] Figure 4 A flowchart illustrating an exemplary method for testing vehicle software according to one or more implementations.

[0032] Figure 5 A flowchart illustrating an exemplary method for testing vehicle software according to one or more implementations.

[0033] Figure 6 A flowchart illustrating an exemplary method for testing vehicle software according to one or more implementations.

[0034] Figure 7 A flowchart illustrating an exemplary method for testing vehicle software according to one or more implementations. Detailed Implementation

[0035] The following detailed description of exemplary embodiments is with reference to the accompanying drawings. The same reference numerals in different drawings may identify the same or identical elements.

[0036] The above disclosure provides examples and descriptions, but is not intended to be exhaustive or to limit the implementation to the exact form disclosed. Modifications and variations can be made based on the above disclosure, or can be obtained through implementation. Furthermore, one or more features or constituent elements of one embodiment can be incorporated into another embodiment (or one or more features of another embodiment) or combined with another embodiment (or one or more features of another embodiment). Moreover, it is understood that in the flowcharts and descriptions of the actions provided below, one or more actions may be omitted, one or more actions may be added, one or more actions may be performed simultaneously (at least partially), and the order of one or more actions may be changed.

[0037] It is evident that the systems and / or methods described in this specification can be implemented in various forms, including hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement the system and / or method is not a limitation of the implementation. Therefore, the operation and behavior of the system and / or method are described in this specification without reference to specific software code. It is understood that software and hardware can be designed to implement the system and / or method based on the description in this specification.

[0038] Even if a specific combination of features is listed in the claims and / or disclosed in this specification, such combination is not intended to limit the disclosure of possible implementations. In fact, many features can be combined in ways not specifically listed in the claims and / or not specifically disclosed in this specification. Each dependent claim listed below may be directly dependent on only one claim, but the disclosure of possible implementations includes each dependent claim combined with all other claims in the set of claims.

[0039] Unless otherwise explicitly stated, elements, actions, or instructions used in this specification should not be construed as essential or necessary. Furthermore, when used in this specification, the articles “a” and “an” refer to more than one item and can be used interchangeably with “more than one.” When referring to only one item, the term “one” or similar terms is used. Additionally, when used in this specification, the terms “has,” “have,” “having,” “include,” “including,” or similar terms imply open-ended usage. Moreover, unless otherwise explicitly stated, the phrase “based on” means “at least partially based on.” Furthermore, expressions such as “at least one of [A] and [B],” “[A] and / or [B],” or “at least one of [A] or [B]” should be understood as referring to only A, only B, or including both A and B.

[0040] It is assumed that the features, advantages, and importance of the exemplary embodiments described above in this specification are only part of this disclosure and are not intended to be exhaustive or to limit the scope of this disclosure.

[0041] The following provides further description of the features, constituent elements, configuration, operation, and implementation scheme of the vehicle software testing system of this disclosure with respect to one or more embodiments.

[0042] Exemplary system architecture

[0043] Figure 1 This is a block diagram illustrating exemplary components of a vehicle software testing (VST) system 100 with more than one implementation. The VST system 100 may include means, systems, platforms, modules, or the like that can be configured to perform more than one action or operation to test vehicle software.

[0044] like Figure 1 As shown, the VST system 100 may include at least one communication interface 110, at least one processor 120, at least one input / output unit 130, and at least one memory 140. However, it will be understood that the VST system 100 may include more than [other components] without departing from the scope of this disclosure. Figure 1 The constituent elements shown may include more or fewer constituent elements and / or may be combined with Figure 1 The methods shown are configured differently.

[0045] The communication interface 110 may include at least one transceiver-type component (e.g., transceiver, separate receiver and transmitter, bus, etc.) that enables the components of the VST system 100 to communicate with each other and / or with one or more components located outside the VST system 100 via wired connection, wireless connection, or a combination of wired and wireless connection. For example, the communication interface 110 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, or a similar interface.

[0046] For example, communication interface 110 can connect processor 120 to memory 140, thereby enabling them to communicate and interact with each other when performing more than one action.

[0047] According to one or more embodiments, the communication interface 110 may include one or more application programming interfaces (APIs) that enable the VST system 100 (or one or more components included in the VST system 100) to communicate with one or more software applications.

[0048] The input / output unit 130 may include at least one component enabling the VST system 100 to receive information and / or provide output information. In some embodiments, it is understood that the input / output unit 130 may include at least one input component (e.g., a touchscreen display, keyboard, keypad, mouse, button, switch, microphone, etc.) and at least one output component (e.g., a display, speaker, one or more light-emitting diodes (LEDs), etc.), and these components may be separable from each other. Further or alternatively, at least one input component may include a sensor for sensing information (e.g., a Global Positioning System (GPS) component, an accelerometer, a gyroscope, and / or an actuator).

[0049] Storage 140 may include one or more storage media suitable for storing data, information, and / or computer-executable instructions internally. According to an exemplary embodiment, storage 140 may include at least one storage device such as random access memory (RAM), read-only memory (ROM), and / or other types of dynamic or static storage devices (e.g., flash memory, magnetic storage, and / or optical storage) that stores information and / or instructions for use by processor 120. Further or alternatively, storage 140 may include a hard disk (e.g., magnetic disk, optical disk, magneto-optical disk, and / or solid-state drive), a compact optical disk (CD), a digital versatile optical disk (DVD), a floppy disk, a cartridge, magnetic tape, and / or other types of non-transitory computer-readable media, and include a corresponding drive. In this specification, computer-readable media are defined as non-transitory storage devices. Storage devices include storage space within a single physical storage device or storage space spread across multiple physical storage devices.

[0050] According to an exemplary embodiment, storage 140 can be configured to store information such as raw data, metadata, or similar data. Further or alternatively, storage 140 can be configured to store more than one piece of information associated with more than one action performed by processor 120. For example, storage 140 can store information specifying past actions performed by processor 120 to test vehicle software, the results of more than one action performed by processor 120, or similar information. Moreover, storage 140 can store data or information required when testing vehicle software. For example, storage 140 (hereinafter referred to as...) Figure 4 (Recording) can store at least several predetermined test steps.

[0051] In any implementation, storage 140 may include multiple storage media, and storage 140 may be configured to store copies or replicas of at least a portion of information on the multiple storage media to provide redundancy and backup of information or associated data. Furthermore, storage 140 may also store computer-readable or computer-executable instructions that, when executed by more than one processor (e.g., processor 120), cause more than one processor to perform more than one operation / action described in this specification.

[0052] Processor 120 may include at least one processor that can be programmed or configured to perform the functions or actions described herein. For example, processor 120 may be configured to execute computer-executable instructions stored in at least one storage medium or storage device (e.g., storage device 140, etc.) thereby performing one or more operations or actions described herein.

[0053] According to an exemplary embodiment, processor 120 may be configured (e.g., via communication interface 110, input / output unit 130, etc.) to receive one or more signals and / or one or more user inputs that define one or more instructions to perform one or more actions. Furthermore, processor 120 may be implemented in hardware, firmware, or a combination of hardware and software. For example, processor 120 may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), and / or other types of processing or computing components.

[0054] According to an exemplary embodiment, the processor 120 may be configured to collect, extract, and / or receive more than one piece of information (in the form of signals or data) and process the received information, thereby testing the vehicle software.

[0055] Figure 1 The number and configuration of the constituent elements shown are provided as an example. In fact, with... Figure 1 Compared to the constituent elements shown, VST system 100 may include additional constituent elements, fewer constituent elements, different constituent elements, or constituent elements configured differently. Further or alternatively, the set of constituent elements of VST system 100 (e.g., more than one constituent element) may perform more than one function as described by other sets of constituent elements of VST system 100.

[0056] The following is for reference Figures 4-7 This provides an explanation of some exemplary actions that can be performed by the processor 120.

[0057] Examples of vehicle software testing disclosed herein

[0058] According to an exemplary implementation, one or more test cases can be designed and executed within a test environment to test vehicle software.

[0059] A test case can refer to a specific set of conditions and / or steps designed to verify the functionality or behavior of software (or a system under test). A test case can specify more than one operation and expected result for a specific test scenario. The conditions and / or steps designed to verify the functionality or behavior of the software can be referred to as the test steps included in the test case. According to an exemplary implementation, more than one test step may be included in a test case, which may include: a test case ID (e.g., a set of inherent identifiers or numbers assigned to test cases for tracking and reference purposes), a test purpose (e.g., a description or specification of the goal or purpose of the test case), at least one test condition (e.g., preconditions or initial states required to execute the test case, specific settings or configurations required to trigger test execution, etc.), at least one test action (e.g., an operation or action performed to execute the test case, required input or interaction with the software, etc.), at least one expected result (e.g., the expected result or behavior of the software when the test case is executed normally, etc.), and any other appropriate information, one or more of these.

[0060] Figure 2 Exemplary test cases illustrating more than one implementation are provided. Test case 200 may include five test steps: test step 1 210, test step 2 220, test step 3 230, test step 4 240, and test step 5 250. It is understood that... Figure 2 The configuration shown is simplified for illustrative purposes and is in no way intended to limit the scope of this disclosure. For example, in practice, the number of test steps within a test case can be arbitrary, and the specific content and format of the test steps can be arbitrary.

[0061] like Figure 2As shown, test step 1 210 may include the prescribed steps of “Given (prerequisite) Start SCENARIO_1 (Scenario_1)” to begin operation of the test environment with the established conditions and / or elements for the test cases to function (see below for details). Test step 2 220 may include the prescribed steps of “And (and) power on the ADAS_ECU” to begin operation of the Advanced Driver Assistance System (ADAS) Electronic Control Unit (ECU) within the test environment. Similarly, test step 3 230 may include the prescribed steps of “And (and) power on the IVI_ECU” to begin operation of the In-Vehicle Infotainment System (IVI) Electronic Control Unit (ECU) within the test environment.

[0062] Test step 4 240 may include the step of specifying the condition scenario where the obstacle is 2.0 meters away from the front right of the vehicle: "When (condition) OBSTACLE (obstacle) is 2.0 meters away from the front right of the vehicle." Test step 5 250 may include the step of specifying the expected result of the condition scenario specified in test step 4 240: "Then (result) IVI displays OBSTACLE (obstacle) at the MAX DISTANCE (maximum distance) from the front right of the vehicle." Here, the in-vehicle infotainment system displays the obstacle at the maximum distance from the front right of the vehicle.

[0063] A test environment can refer to a specific set of conditions and / or elements under which test cases operate (for which conditions and / or elements). In particular, a test environment may include more than one vehicle element associated with a test case. More than one vehicle element can refer to more than one element associated with more than one component of the vehicle, such as an electronic control unit (ECU). For example, with respect to test cases designed to test software associated with the detection and display of obstacles near the rear of the vehicle, the test environment may include the vehicle's IVI ECU and ADAS ECU (more than one vehicle element). According to an exemplary implementation, the test environment may also include more than one environmental element associated with a test case. More than one environmental element can refer to more than one element associated with the vehicle's environment. For example, test cases designed to test software associated with detecting and displaying obstacles near the rear of the vehicle may include obstacles near the rear of the vehicle and rain (more than one environmental element). As a result, test cases can evaluate the software associated with the vehicle's IVI ECU and ADAS ECU while the vehicle's IVI ECU and ADAS ECU are detecting and displaying obstacles near the rear of the vehicle during rain.

[0064] Figure 3 Exemplary components of a graphical user interface (GUI) for testing vehicle software, representing one or more implementations.

[0065] like Figure 3 As shown, the graphical user interface (GUI) 300 may include at least a first part 320 and a second part 340.

[0066] According to an exemplary implementation, the first portion 320 may include a section for inputting code. The code can be associated with a test step. For example, a user can input code via the method described above. Figure 2 The same method described above involves manually entering the code associated with the test step into the first part, resulting in the test step being appended to the test case.

[0067] According to an exemplary implementation, the code associated with the test steps in the first part can be a natural language format (i.e., natural language code) such as the Gherkin language format. According to an exemplary implementation, the natural language code can be input into the first part 320 by a first user. According to an exemplary implementation, the first user can include users responsible for the design and research of test cases (and the associated test steps) but who do not have extensive knowledge related to low-level programming, such as test engineers, test case managers, and similar users.

[0068] Therefore, through the above, the first user can easily input and study the test steps without having extensive knowledge related to low-level programming, and can design test cases through a behavior-driven approach.

[0069] According to an exemplary implementation, the second part 340 may include at least a retrieval area 342 and a test step area 344. Here, it is understood that the VST system's storage can store multiple predetermined test steps. These predetermined test steps may be test steps pre-defined and stored by the software developer that can be retrieved by the first user of the GUI 300.

[0070] In particular, the first user can provide search input to the search area 342, whereby the VST system can then retrieve one or more test steps associated with the search input from a plurality of stored predetermined test steps, and display the retrieved test steps in the test step area 344.

[0071] According to an exemplary implementation, the search input can include any kind of keywords associated with the test steps, such as the name of the test step, the type of the test step, and similar terms. Furthermore, multiple predetermined test steps can be grouped by test step type and / or assigned tags for retrieval based on that test step type. For example, test steps associated with CAN communication can be assigned a CAN communication tag and / or grouped into a CAN communication group. As a result, if the first user enters "CAN" or "CAN communication" in the search area 342 (i.e., the search input contains "CAN" or "CAN communication"), all test steps assigned the CAN communication tag and / or all test steps within the CAN communication group (i.e., all test steps associated with CAN communication) can be displayed in the test step area 344. In another example, test steps describing conditional actions such as "given," "when," "then," and similar terms can be assigned corresponding tags for "given," "when," "then," and similar terms.

[0072] According to an exemplary implementation, each of a plurality of predetermined test steps can be grouped into multiple types of test steps and / or assigned multiple tags for retrieval based on that test step type. For example, a test step describing a conditional action associated with CAN communication can be assigned both a corresponding condition tag and a CAN communication tag. Therefore, the test step can be displayed when the corresponding condition tag and / or CAN communication tag is entered as retrieval input. Moreover, if the retrieval input contains keywords for both the assigned corresponding condition tag and CAN communication tag, only the test steps assigned the corresponding condition tag and CAN communication tag can be displayed. Thus, the user can easily retrieve the desired test step.

[0073] When one or more test steps are displayed in the test step area 344, the first user can then provide selection input to select one of the test steps displayed in the test step area 344. According to an exemplary embodiment, the first user can provide selection input by dragging and dropping a test step from the test step area 344 onto the first portion 320. According to an exemplary embodiment, the first user can provide selection input by clicking on a test step in the test step area 344.

[0074] Furthermore, upon receiving selection input from the first user, the VST system can automatically input the code associated with one of the selected test steps from more than one test steps into the first section 320. Understandably, the code input into the first section 320 can be edited and removed as desired by the first user. Therefore, the first user does not need to manually type in the test steps; the test steps can be quickly retrieved and appended to the first section 320 (and test cases).

[0075] According to an exemplary implementation, the first part 320 may also include a window 322. The window 322 may be a pop-up window that appears when the first user types a code into the first part 320.

[0076] Specifically, the first user can provide partial natural language code input (i.e., code input as a part of the natural language format) to the first part 320 of the GUI 300. When the first user provides partial natural language code input, the VST system can retrieve one or more test steps associated with the partial natural language code input and display them in window 322. For example, the VST system can display one or more test steps most similar to the partial natural language code input in window 322. It is understood that the test steps displayed in window 322 can be retrieved from a plurality of stored predetermined test steps in the same way as the test steps displayed in test step area 344.

[0077] The first user can then provide selection input for choosing one of the more than one test steps displayed in window 322. Here, the VST system can then automatically replace (record) the portion of the natural language code input in the first part 320 with the complete code associated with the selected test step among the more than one test steps.

[0078] For example, a first user might type the first part (i.e., a partial natural language code input) of a natural language code associated with a specific test step into first section 320. The VST system can then determine a test step most similar to that first part and display it in window 322. Alternatively, the VST system can determine (including the specific test step the first user wants to type) multiple test steps (candidates) most similar to that first part and display them in window 322. Here, the first user can select the specific test step they want to type in window 322. The VST system can then automatically enter the remaining part of the specific test step (replacing the first part of the code already entered into first section 320 with the code associated with the selected specific test step (the complete code)). Alternatively, the first user can select another test step in window 322 (e.g., the first user initially entered the same but incorrect code). Here, the VST system can then automatically replace the first part of the code already entered into first section 320 with the code associated with the selected other test step (the complete code).

[0079] Therefore, the first user does not need to manually type in all the test steps; the test steps can be quickly retrieved and appended to the first part 320 (and the test cases).

[0080] According to an exemplary implementation, GUI300 may also include the ability to add new test steps to a plurality of predetermined test steps stored in a storage memory via the GUI when designing test cases.

[0081] In particular, it is understood that each of the multiple predetermined test steps stored in the memory can be associated with programming language code and natural language code. Natural language code can refer to code written in a natural language format (e.g., Gherkin), which allows users to easily type and study the code without requiring extensive knowledge of low-level programming, but cannot be compiled and executed in a simulation. On the other hand, programming language code can refer to code written in a programming language format (e.g., non-natural programming language code such as Python), which can be compiled and executed in a simulation, but requires users to have extensive knowledge of low-level programming to type and study it.

[0082] Here, "code" can refer to a single line of code / command, and it is understood that one natural language code can correspond to more than one programming language code. Therefore, in order to execute a test step, the natural language code associated with the test step in Part 1 320 can be executed by executing one or more corresponding programming language codes. Thus, a given test step can be stored and appended by storing the natural language code associated with the given test step along with one or more corresponding programming language codes.

[0083] According to an exemplary implementation, in order to add new test steps, a first user can initially input one or more natural language codes into the first part. These one or more natural language codes can correspond to one or more test steps that the first user wants to execute.

[0084] When more than one natural language code is received from the first user, the system can determine whether the received more than one natural language code contains any new natural language code that does not correspond to any of the more than one programming language codes stored in the storage.

[0085] Specifically, for example, a first user can input three natural language codes corresponding to three test steps that the first user wants to perform. At this point, the first and second natural language codes are identical to the natural language codes stored in memory, and may also have corresponding programming language codes stored in memory (i.e., the first and second test steps (the first and second natural language codes and their corresponding programming language codes) correspond to predetermined test steps already stored in memory). Therefore, the first and second natural language codes can be executed by executing their corresponding programming language codes. On the other hand, the third natural language code is sometimes new and does not correspond to either the natural language codes or programming language codes stored in memory (i.e., the third test step (the third natural language code) is new and needs to be appended). Therefore, since there is no corresponding executable programming language code, the third natural language code is not executable. Therefore, it may be necessary to append programming language code corresponding to the third natural language code to append the third test step.

[0086] According to an exemplary implementation, the system can determine whether the received natural language code contains any new natural language code simply by executing the received natural language code, and confirm whether any of the received natural language code cannot be executed due to the lack of a corresponding programming language code.

[0087] In this respect, in response to determining that the received natural language code contains more than one new natural language code, the system can send more than one new natural language code to a second user. The second user can be a software developer or similar user with extensive knowledge related to low-level programming. Therefore, the second user can develop and prepare a programming language corresponding to the more than one new natural language code, and then provide that programming language to the system for execution.

[0088] According to an exemplary embodiment, GUI 300 may further include additional options (not shown) for adding test steps. Specifically, according to an exemplary embodiment, a first user can input only one or more new natural language codes (i.e., natural language codes without corresponding programming language codes) into the first part. Then, the first user can select an additional option, whereby the system can then send one or more new natural language codes to a second user, who receives the corresponding new programming language codes using the same method described above. According to an exemplary embodiment, GUI 300 may further include a new code section (not shown). The new code section may include a part for inputting new codes. Specifically, according to an exemplary embodiment, a first user can input only one or more new natural language codes (i.e., natural language codes without corresponding programming language codes) into the new code section. Then, the system can then automatically send one or more new natural language codes from the new code section to a second user, who receives the corresponding new programming language codes using the same method described above.

[0089] When a new programming language code (i.e., programming code corresponding to the new natural language code) is received from a second user, the system can store the new programming language code and the new natural language code as a new predetermined test step in the storage memory. At this point, the new predetermined test step can be retrieved and displayed in the same way as described above (in test step area 344 and / or in window 322).

[0090] Furthermore, the system can also examine and verify whether more than one received natural language code (including new natural language code) can be executed normally by the received new programming language code.

[0091] Furthermore, the system can also send a notification indicating that new programming language code has been received to the first user.

[0092] Therefore, the first user can provide instructions to execute one or more received natural language codes (i.e., natural language codes provided by the first user in the first part that include new natural language codes). In response to receiving instructions, the system can then execute the one or more received natural language codes by executing the corresponding programming language codes (all of the received natural language codes should have corresponding programming language codes, and therefore should succeed at this point). It is understood that the system can utilize any appropriate functions and elements for facilitating the execution of natural language codes and programming language codes, such as APIs (e.g., Python APIs), context objects, and similar things.

[0093] Therefore, the above process enables users who do not have extensive knowledge related to low-level programming to easily design and research test cases, while also providing such users with the flexibility to create new test steps upon request.

[0094] According to an exemplary implementation, GUI300 may also include multiple options (not shown) for the execution of test cases.

[0095] In particular, to make test cases work (execute) in a test environment, test cases can be made online using a Continuous Integration and Continuous Delivery (CICD) framework. As a result, test cases can be uploaded to a server and accessed by user groups. Therefore, according to an exemplary implementation, GUI 300 may also include a first working option and a second working option. In response to receiving selection input from a first user to select the first working option, the VST system uploads and makes work (executes) a test case comprising one or more test steps from the first part 320 online using the CICD framework. On the other hand, in response to receiving selection input from the first user to select the second working option, the VST system can make test cases comprising one or more test steps from the first part 320 work offline without using the CICD framework.

[0096] Therefore, the VST system enables users to quickly switch between online and offline execution. Online execution allows test cases to be uploaded and accessed by other user groups, while offline execution avoids the risk of uploading unwanted test cases to the server and allows for rapid execution of test cases.

[0097] Furthermore, in order to make the test cases work in the test environment, the simulated test environment can be generated for each task (execution) of the test cases. When the simulated test environment generates tasks for the test cases, it can terminate after the test cases are completed. Therefore, according to the exemplary implementation, GUI300 may also include a third task option and a fourth task option. In response to receiving a selection input from the first user to select the third task option, the VST system can maintain the simulated test environment after executing a test case having one or more test steps of the first part 320. On the other hand, in response to receiving a selection input from the first user to select the fourth task option, the VST system can terminate the simulated test environment after executing a test case having one or more test steps of the first part 320.

[0098] Therefore, the VST system enables users to quickly switch between maintaining the execution of the simulated test environment and terminating the execution of the simulated test environment. Maintaining the execution of the simulated test environment means that the simulated test environment does not need to be restarted every time the same or different test cases are executed. It allows the subsequent test cases to be executed quickly in the same simulated test environment. Terminating the execution of the simulated test environment is a resource-saving execution.

[0099] Understandable, Figure 3 The configurations shown are simplified for illustrative purposes and are by no means intended to limit the scope of this disclosure. For example, in practice, the relative positions, relative dimensions, and / or shapes associated with each element within the GUI300 can be arbitrary.

[0100] Exemplary actions for testing vehicle software as disclosed herein

[0101] The following is for reference Figures 4-7 This document describes some exemplary actions that can be performed by the VST system disclosed herein.

[0102] Figure 4 The flowchart illustrates an exemplary method 400 for testing vehicle software according to one or more embodiments. One or more actions in method 400 may be performed by at least one processor (e.g., processor 120) of the VST system.

[0103] like Figure 4 As shown, in action S410, at least one processor can be configured to receive more than one natural language code. The more than one natural language code can be received from the first user and can refer to more than one code written in a natural language format. For example, the more than one natural language code could be in Gherkin programming language format. Next, the method proceeds to action S420.

[0104] In action S420, at least one processor may be configured to determine whether the received one or more natural language codes contain one or more new natural language codes that do not correspond to any of the one or more programming language codes stored in the memory. One or more programming language codes may refer to one or more codes written in a programming language format (e.g., a non-natural language format). For example, one or more programming language codes may be in Python programming language format.

[0105] According to an exemplary implementation, at least one processor may be configured to determine whether a received natural language code contains a new natural language code by performing the following actions: executing the received natural language code; determining that the received natural language code cannot be executed; and identifying a new code from the received natural language code that cannot be executed.

[0106] According to an exemplary implementation, one or more received natural language codes can be executed by executing one or more corresponding programming language codes. Therefore, according to an exemplary implementation, one or more received natural language codes may sometimes be unexecutable because one or more codes from (unexecutable) received natural language codes do not have one or more corresponding programming language codes for execution. In this case, one or more codes from (unexecutable) received natural language codes can correspond to one or more new natural language codes. Next, the method proceeds to action S430.

[0107] In action S430, in response to determining that the received natural language code contains more than one new natural language code, at least one processor can be configured to send more than one new natural language code. The more than one new natural language code can then be sent to the second user. The method then proceeds to action S440.

[0108] In action S440, at least one processor can be configured to receive more than one new programming language code. The more than one new programming language code can be received from a second user and can correspond to more than one new natural language code.

[0109] According to an exemplary implementation, at least one processor may also be configured to store one or more new programming language codes and corresponding one or more new natural language codes as predetermined test steps in a storage memory.

[0110] According to an exemplary implementation, at least one processor may also be configured to determine whether the received natural language code is executed correctly in response to receiving more than one new programming language code.

[0111] According to an exemplary embodiment, at least one processor can be configured to generate a graphical user interface. According to an exemplary embodiment, the graphical user interface may include at least a first portion having input codes. According to an exemplary embodiment, more than one natural language code may be received in the first portion.

[0112] According to an exemplary implementation, at least one processor may also be configured to send a notification indicating that more than one new programming language code has been received to a first user in response to receiving more than one new programming language code.

[0113] When action S440 is performed, method 400 may end or terminate. Alternatively, method 400 may return to action S410, resulting in at least one processor being configured to repeatedly perform the following actions for at least a predetermined amount of time: (in action S410) receiving more than one natural language code, (in action S420) determining whether the received more than one natural language code contains more than one new natural language code, (in action S430) sending more than one new natural language code, and (in action S440) receiving more than one new programming language code.

[0114] For example, a first user can continuously (or periodically) provide more natural language codes (which may contain more new natural language codes). Therefore, at least one processor can continuously (or periodically) receive more than one natural language code, and then restart the following actions: (in action S410) receiving more than one natural language code, (in action S420) determining whether the received more than one natural language code contains more than one new natural language code, (in action S430) sending more than one new natural language code, and (in action S440) receiving more than one new programming language code.

[0115] Figure 5 The flowchart illustrates an exemplary method 500 for testing vehicle software according to one or more embodiments. One or more actions in method 500 may be performed by at least one processor (e.g., processor 120) of the VST system.

[0116] like Figure 5 As shown, in action S510, at least one processor can be configured to receive more than one new natural language code. More than one new natural language code can be received from the first user using the same method as described above for method 400. This more than one new natural language code can refer to more than one code written in a natural language format (e.g., Gherkin) and not corresponding to any of the more than one programming language codes stored in memory. The method then proceeds to action S520.

[0117] In action S520, at least one processor may be configured to determine (verify) whether one or more new natural language codes received correspond to any one of one or more programming language codes stored in the storage memory. According to an exemplary embodiment, at least one processor may be configured to determine whether one or more new natural language codes received correspond to any one of one or more programming language codes stored in the storage memory by retrieving and analyzing the programming language codes stored in the storage memory.

[0118] Therefore, in response to determining that one or more new natural language codes received correspond to any one of one or more programming language codes stored in the memory, at least one processor can determine that it is not necessary to obtain new programming language codes corresponding to the one or more new natural language codes received, and the method proceeds to the end. On the other hand, in response that one or more new natural language codes received do not correspond to any of the one or more programming language codes stored in the memory, at least one processor can determine that it is necessary to obtain new programming language codes corresponding to the one or more new natural language codes received, and the method proceeds to action S530.

[0119] In action S530, at least one processor can be configured to send more than one new natural language code. This more than one new natural language code can then be sent to a second user. The method then proceeds to action S540.

[0120] In action S540, at least one processor can be configured to receive more than one new programming language code. The more than one new programming language code can be received from a second user and can correspond to more than one new natural language code.

[0121] According to an exemplary implementation, at least one processor may also be configured to store one or more new programming language codes and corresponding one or more new natural language codes as predetermined test steps in a storage memory.

[0122] According to an exemplary implementation, at least one processor may also be configured to determine whether the received natural language code is executed correctly in response to receiving more than one new programming language code.

[0123] According to an exemplary embodiment, at least one processor can be configured to generate a graphical user interface. According to an exemplary embodiment, the graphical user interface may include at least a first portion having an input code section and additional options for adding new test steps. According to an exemplary embodiment, one or more new natural language codes may be received in the first portion. According to an exemplary embodiment, one or more new natural language codes may be sent in response to receiving selection input from a first user to select additional options. According to an exemplary embodiment, the selection input may include click input.

[0124] According to an exemplary embodiment, the graphical user interface may include at least a new code section for inputting new codes. According to an exemplary embodiment, one or more new natural language codes may be received in the new code section. According to an exemplary embodiment, one or more new natural language codes may be sent in response to receiving one or more new natural language codes from the new code section from a first user.

[0125] According to an exemplary implementation, at least one processor may also be configured to send a notification indicating that more than one new programming language code has been received to a first user in response to receiving more than one new programming language code.

[0126] When action S540 is performed, method 500 may end or terminate. Alternatively, method 500 may return to action S510, resulting in at least one processor being configured to repeatedly perform the following actions for at least a predetermined amount of time: (in action S510) receiving more than one new natural language code, (in action S520) determining whether the received more than one new natural language code corresponds to any one of more than one programming language code stored in memory, (in action S530) sending more than one new natural language code, and (in action S540) receiving more than one new programming language code.

[0127] For example, the first user can continuously (or periodically) provide more new natural language codes. Therefore, at least one processor can continuously (or periodically) receive more than one new natural language code, and then restart the following actions: (in action S510) receiving more than one new natural language code, (in action S520) determining whether the received more than one new natural language code corresponds to any of more than one programming language code stored in the memory, (in action S530) sending more than one new natural language code, and (in action S540) receiving more than one new programming language code.

[0128] Figure 6The flowchart illustrates an exemplary method 600 for testing vehicle software according to one or more embodiments. One or more actions in method 600 may be performed by at least one processor (e.g., processor 120) of the VST system.

[0129] like Figure 6 As shown, in action S610, at least one processor can be configured to generate a graphical user interface. The graphical user interface may include at least a first part and a second part; the first part may include a section for inputting code, and the second part may include a retrieval area and a test step area. The method then proceeds to action S620.

[0130] In action S620, at least one processor can be configured to receive retrieval input. The retrieval input can be received from a first user in the retrieval area. The method then proceeds to action S630.

[0131] In action S630, at least one processor can be configured to display one or more test steps associated with the retrieved input. According to an exemplary implementation, one or more test steps may include a test case ID (e.g., a set of inherent identifiers or numbers assigned to test cases for tracking and reference purposes), a test purpose (e.g., a description or specification of the goal or purpose of the test case), at least one test condition (e.g., preconditions or initial states required to execute the test case, specific settings or configurations required to trigger test execution, etc.), at least one test action (e.g., an operation or action performed to execute the test case, required input or interaction with the software, etc.), at least one expected result (e.g., the anticipated result or behavior expected by the software when the test case is executed normally, etc.), and any other appropriate information. Furthermore, test steps can be associated with natural language code and programming language code and can be displayed in the test step area.

[0132] According to an exemplary implementation, at least one processor can be configured to acquire (retrieve) one or more test steps based on a retrieval input, and then display the acquired one or more test steps, thereby displaying one or more test steps associated with the retrieval input.

[0133] According to an exemplary embodiment, one or more test steps can be obtained based on a search input by determining one or more keywords. For example, one or more keywords in the search input may include the name of the test step, the type of the test step, and similar terms. According to an exemplary embodiment, one or more test steps can also be obtained based on the search input by acquiring one or more test steps associated with the determined keywords. For example, at least one processor can be configured to acquire one or more test steps with the same name as the test steps included in the search input. In another example, at least one processor can be configured to acquire one or more test steps within a group or with the same tag as the test steps included in the search input.

[0134] According to an exemplary implementation, the storage memory of the VST system may include multiple predetermined test steps, and more than one test step (i.e., more than one test step associated with the retrieval input) can be obtained from the multiple predetermined test steps. The method then proceeds to action S640.

[0135] In action S640, at least one processor may be configured to receive selection input. The selection input may be received from a first user and may involve selecting one of more than one test steps displayed in the test step area. According to an exemplary embodiment, the selection input may include drag-and-drop input. According to an exemplary embodiment, the selection input may include click input. The method then proceeds to action S650.

[0136] In action S650, at least one processor may be configured to input code associated with one of the selected test steps from more than one test step into the first part. According to an exemplary implementation, the code in the first part (i.e., code manually entered by the first user and code automatically entered by the VST system in the first part) may be in natural language format (i.e., natural language code).

[0137] According to an exemplary implementation, when code associated with a selected test step from more than one test steps is entered into the first part, the user can edit and modify the entered code as desired. For example, in order to add a specific value to a parameter, the user can edit the placeholder part of the code (e.g., "[duration]") to a specific number (e.g., "

[10] ").

[0138] When action S650 is performed, method 600 can end or terminate. Alternatively, method 600 can return to action S620, resulting in at least one processor being configured to repeatedly perform the following actions at least for a predetermined amount of time: (in action S620) receiving retrieval input, (in action S630) displaying more than one test step, (in action S640) receiving selection input, and (in action S650) entering a code. For example, a first user can retrieve more test steps continuously (or periodically). Therefore, at least one processor can continuously (or periodically) receive retrieval input and then restart the following actions: (in action S620) receiving retrieval input, (in action S630) displaying more than one test step, (in action S640) receiving selection input, and (in action S650) entering a code.

[0139] Figure 7 The flowchart illustrates an exemplary method 700 for testing vehicle software according to one or more embodiments. One or more actions in method 700 may be performed by at least one processor (e.g., processor 120) of the VST system.

[0140] like Figure 7 As shown, in action S710, at least one processor can be configured to generate a graphical user interface. The graphical user interface may include at least a first part, which may include a code input section and a pop-up window. The method then proceeds to action S720.

[0141] In action S720, at least one processor can be configured to receive partial code input. The partial code input can be received from a first user in the first part. Furthermore, it is understood that the partial code input can be a part of the code (complete code) associated with the test step. The method then proceeds to action S730.

[0142] In action S730, at least one processor can be configured to display more than one test step associated with a portion of the code input. More than one test step can be displayed in a pop-up window.

[0143] According to an exemplary implementation, at least one processor can be configured to acquire (retrieve) more than one test step based on partial code input, and then display the acquired more than one test step, thereby displaying more than one test step associated with the partial code input.

[0144] According to an exemplary implementation, one or more test steps can be obtained based on the partial code input by determining one or more test steps most similar to the partial code input. For example, a first user sometimes types the first part (i.e., the partial code input) of the code associated with a specific test step in the first part, and the VST system can then determine one or more test steps that are most similar to the code of that first part. The method then proceeds to action S740.

[0145] In action S740, at least one processor can be configured to receive selection input. The selection input can be received from a first user and may involve selecting one of more than one test steps displayed in a pop-up window. The method then proceeds to action S750.

[0146] In action S750, at least one processor may be configured to replace partial code input in the first part with code (i.e., complete code) associated with a selected test step among more than one test step. According to an exemplary implementation, the code in the first part (i.e., partial code manually entered by a first user and complete code automatically entered by the VST system in the first part) may be in natural language format (i.e., partial natural language code and natural language code).

[0147] According to an exemplary implementation, when the code associated with a selected test step out of more than one test step is replaced with a portion of the code input in the first part, the user can edit and modify the replaced code as desired. For example, in order to add a specific value to a parameter, the user can edit the placeholder part of the code (e.g., "[duration]") to a specific number (e.g., "

[10] ").

[0148] When action S750 is performed, method 700 may end or terminate. Alternatively, method 700 may return to action S720, resulting in at least one processor being configured to repeatedly perform the following actions for at least a predetermined amount of time: (in action S720) receiving code input for the portion, (in action S730) displaying more than one test step, (in action S740) receiving selection input, and (in action S750) receiving code input for the replacement portion.

[0149] For example, a first user can continuously (or periodically) type more codes into the first part. Therefore, at least one processor can continuously (or periodically) receive code input for the part, and then restart the following actions: (in action S720) receiving code input for the part, (in action S730) displaying more than one test step, (in action S740) receiving selection input, and (in action S750) replacing code input for the part.

[0150] According to the exemplary implementation, one or more of the actions of methods 400, 500, 600 and 700 can be combined with each other.

[0151] For example, the system can generate a graphical user interface where the first user can first retrieve and select the first test step via the retrieval area and the test step area (here, actions S610 to S650 are performed). Then, the user can type in a portion of the code and select the code related to the second test step in the pop-up window (here, actions S720 to S750 are performed).

[0152] In another example, the system can generate a graphical user interface (GUI), where a first user can first retrieve and select a first test step via a retrieval area and a test step area (here, actions S610 to S650 are performed). Next, the system can receive new natural language code from the first user that is associated with a second test step in the first part / new code section of the GUI, send the new natural language code to a second user, and receive one or more new programming language codes corresponding to the new natural language code (here, actions S510 to S540 are performed).

[0153] Various implementation schemes

[0154] The above disclosure provides examples and illustrations, but is not intended to be exhaustive or to limit the implementation to the exact form disclosed. Modifications and variations can be made based on the above disclosure, or can be obtained through implementation.

[0155] A portion of the implementation may relate to systems, methods, and / or computer-readable media at a detailed level of any possible integrated technology. Furthermore, one or more of the aforementioned constituent elements may be implemented as instructions stored in a computer-readable medium and executable by at least one processor (and / or may include at least one processor). The computer-readable medium may include a computer-readable non-transitory storage medium (or media (multiple)) having computer-readable program instructions for causing the processor to perform actions.

[0156] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof, but is not limited thereto. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile optical disc (DVD), memory sticks, floppy disks, punch cards, or devices with mechanically encoded structures in slots that record instructions therein, and any suitable combinations thereof. The computer-readable storage medium used in this specification should not be construed as a temporary signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0157] The computer-readable program instructions described in this specification can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded to external computers or external storage devices via networks such as the Internet, local area networks, wide area networks, and / or wireless networks. The network may include copper cables, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. Network adapter cards or network interfaces within each computing / processing device receive and transmit the computer-readable program instructions from the network for storage on the computer-readable storage media within the respective computing / processing device.

[0158] Computer-readable program code / instructions that perform actions can be any of the following: assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages. These one or more programming languages ​​include object-oriented programming languages ​​such as Smalltalk, C++, or similar languages, and procedural programming languages ​​such as the "C" programming language or similar languages. Computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer can connect to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or (for example, via the internet using an internet service provider) connect to an external computer. In some implementations, for example, electronic circuits including programmable logic circuits, field-programmable gate arrays (FPGAs) or programmable logic arrays (PLAs) can be personalized to execute computer-readable program instructions by utilizing state information of computer-readable program instructions to perform schemes or actions.

[0159] The computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to generate a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create parts that implement the functions / behaviors specified in the flowcharts and / or block diagrams or in the function blocks(s). The computer-readable program instructions can also be stored in a computer-readable storage medium that can instruct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer-readable storage medium containing the instructions comprises an article of manufacture including instructions for implementing the functions / behaviors specified in the function blocks of the flowcharts and / or block diagrams or in the function blocks(s).

[0160] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device, and perform a series of action steps on the computer, other programmable apparatus or other device to generate a computer-implemented process, the result of which the instructions executed on the computer, other programmable apparatus or other device implement the functions / behaviors specified in the function blocks of the flowchart and / or block diagram or in the function blocks(multiple).

[0161] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and action of various implementations of systems, methods, and computer-readable media. In this regard, each functional block in a flowchart or block diagram may represent a microservice module, segment, or part of an instruction having one or more executable instructions that implement the specified logical function. Methods, computer systems, and computer-readable media may include additional functional blocks, fewer functional blocks, different functional blocks, or functional blocks configured differently compared to those depicted in the figures. In some alternative implementations, the functions described in the functional blocks may occur independently of the order shown in the figures. For example, two consecutively shown functional blocks may be executed practically or substantially simultaneously, or functional blocks may sometimes be executed in reverse order depending on the functions involved. It should also be noted that the functional blocks of the block diagrams and / or flowcharts, and combinations of functional blocks in the block diagrams and / or flowcharts, can be implemented by a system based on dedicated hardware that performs the specified functions or behaviors or executes a combination of dedicated hardware and computer instructions.

[0162] It is evident that the systems and / or methods described in this specification can be implemented in various forms, including hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement the system and / or method is not a limitation on the implementation. Therefore, it is understood that the operation and behavior of the system and / or method are not described in this specification with reference to specific software code, and software and hardware can be designed to implement the system and / or method based on the description in this specification.

[0163] It will be understood that many modifications and variations of this disclosure can be made in light of the foregoing teachings. It will be apparent that, within the scope of the appended sections, this disclosure can be practiced in ways different from those specifically described in this specification.

[0164] Alternatively, the computer program product including the computer program of the above embodiments may be stored in a storage medium or distributed through a communication line.

Claims

1. A system for testing the software of a vehicle, wherein, The system has the following features: Storage memory, which stores computer-executable instructions; and At least one processor is connected to the storage device in a communicative manner. The at least one processor is configured to execute the instructions to perform the following actions: Receive more than one natural language code from the first user; Determine whether the received natural language code contains more than one new natural language code that does not correspond to any of the more than one programming language codes stored in the storage device; In response to determining that the received natural language code contains the one or more new natural language codes, the one or more new natural language codes are sent to the second user; and Receive more than one new programming language code from the second user. The received one or more new programming language codes correspond to the one or more new natural language codes.

2. The system according to claim 1, wherein, The at least one processor is further configured to execute the instructions to store the one or more new programming language codes and the corresponding one or more new natural language codes as predetermined test steps in the storage memory.

3. The system according to claim 1 or 2, wherein, The at least one processor is configured to determine whether the received one or more natural language codes contain the one or more new natural language codes by executing the instructions: Execute one or more received natural language codes, wherein the one or more received natural language codes are executed by executing one or more corresponding programming language codes; It was determined that more than one of the received natural language codes could not be executed; and Identify one or more codes from one or more natural language codes that cannot be executed.

4. The system according to any one of claims 1 to 3, wherein, The at least one processor is also configured to execute the instructions to perform the following actions: In response to receiving the one or more new programming language codes, determine whether the received one or more natural language codes are executed correctly.

5. The system according to any one of claims 1 to 4, wherein, The at least one processor is also configured to execute the instructions to perform the following actions: Generate a graphical user interface having at least a first part and a second part, wherein the first part has a code input area and the second part has a search area and a test step area; Receive search input from the first user in the search area; One or more test steps associated with the search input will be displayed in the test step area; The user receives selection input from the first user to select one of the more than one test steps displayed in the test step area; and The natural language code associated with one of the selected test steps from the more than one test steps is input into the first part.

6. The system according to claim 5, wherein, The one or more natural language codes are received in the first part.

7. The system according to claim 5 or 6, wherein, The storage device has several predetermined testing procedures. Each of the multiple predetermined test steps is associated with natural language code and corresponding code in one or more programming languages. The one or more test steps associated with the retrieval input are obtained from the plurality of predetermined test steps.

8. The system according to any one of claims 5 to 7, wherein, The first part also includes a pop-up window. The at least one processor is also configured to execute the instructions to perform the following actions: Natural language code input is received from the first user in the first part; One or more test steps that are associated with the natural language code input of the aforementioned section will be displayed in the pop-up window; Receive selection input from the first user to select one of the more than one test steps displayed in the pop-up window; and In the first part, the natural language code input of the part is replaced with natural language code that is associated with one of the selected test steps in the more than one test steps.

9. The system according to any one of claims 1 to 8, wherein, The more than one natural language code is in the Gherkin programming language format, and the more than one programming language code is in the Python programming language format.

10. A method performed by a system for testing the software of a vehicle, comprising: Receive more than one natural language code from the first user; Determine whether the received natural language code contains more than one new natural language code that does not correspond to any of the more than one programming language codes stored in the system's storage. In response to determining that the received natural language code contains the one or more new natural language codes, the one or more new natural language codes are sent to the second user; and Receive more than one new programming language code from the second user. The received one or more new programming language codes correspond to the one or more new natural language codes.

11. The method of claim 10, further comprising: The one or more new programming language codes and the corresponding one or more new natural language codes are stored in the storage device as predetermined test steps.

12. The method according to claim 10 or 11, wherein, The determination of whether the received one or more natural language codes contain the one or more new natural language codes includes: Execute one or more received natural language codes, wherein the one or more received natural language codes are executed by executing one or more corresponding programming language codes; It was determined that more than one of the received natural language codes could not be executed; and Identify one or more codes from one or more natural language codes that cannot be executed.

13. The method according to any one of claims 10 to 12, further comprising: In response to receiving the one or more new programming language codes, determine whether the received one or more natural language codes are executed correctly.

14. The method according to any one of claims 10 to 13, further comprising: Generate a graphical user interface having at least a first part and a second part, wherein the first part has a code input area and the second part has a search area and a test step area; Receive search input from the first user in the search area; One or more test steps associated with the search input will be displayed in the test step area; The user receives selection input from the first user to select one of the more than one test steps displayed in the test step area; and The natural language code associated with one of the selected test steps from the more than one test steps is input into the first part.

15. The method according to claim 14, wherein, The one or more natural language codes are received in the first part.

16. The method according to claim 14 or 15, wherein, The storage device has several predetermined testing procedures. Each of the multiple predetermined test steps is associated with natural language code and corresponding code in one or more programming languages. The one or more test steps associated with the retrieval input are obtained from the plurality of predetermined test steps.

17. The method according to any one of claims 14 to 16, wherein, The first part also includes a pop-up window. The method further includes: Natural language code input is received from the first user in the first part; One or more test steps that are associated with the natural language code input of the aforementioned section will be displayed in the pop-up window; Receive selection input from the first user to select one of the more than one test steps displayed in the pop-up window; and In the first part, the natural language code input of the part is replaced with natural language code that is associated with one of the selected test steps in the more than one test steps.

18. The method according to any one of claims 10 to 17, wherein, The more than one natural language code is in the Gherkin programming language format, and the more than one programming language code is in the Python programming language format.

19. A computer program product comprising a computer program for causing at least one processor to perform a method, wherein, The method includes: Receive more than one natural language code from the first user; Determine whether the received one or more natural language codes contain one or more new natural language codes that do not correspond to any of the one or more programming language codes stored in the storage memory of the system executing the method; In response to determining that the received natural language code contains the one or more new natural language codes, the one or more new natural language codes are sent to the second user; and Receive more than one new programming language code from the second user. The received one or more new programming language codes correspond to the one or more new natural language codes.

20. The computer program product according to claim 19, wherein, The determination of whether the received one or more natural language codes contain the one or more new natural language codes includes: Execute one or more received natural language codes, wherein the one or more received natural language codes are executed by executing one or more corresponding programming language codes; It was determined that more than one of the received natural language codes could not be executed; and Identify one or more codes from one or more natural language codes that cannot be executed.

Images