A keyword-driven automated test parallel execution system

Abstract

The application discloses a kind of based on keyword-driven automated testing parallel execution system, comprising: parallel execution management module, for management to test subthread;Keyword step parallel execution module, for creating the executable environment of test subthread;Information interaction module, for realizing the information transmission and interaction between threads;Exception handling module, for judging whether test case continues to execute when keyword step executes error;Execution result management module, for the execution result of keyword step is counted and managed;Log information module, for forming complete test execution process record;Parallel execution keyword library, for the parallel execution mechanism is integrated into the automated testing framework based on keyword-driven;Parallel test driving module, for realizing to embedded software full digital automated testing.The application solves the problem that the efficiency of test is not high due to single thread execution keyword step under the existing automated testing framework based on keyword-driven.

Description

Technical Field

[0001] This invention belongs to the field of automated test execution technology, and in particular relates to a keyword-driven automated test parallel execution system. Background Technology

[0002] Automated testing transforms human-driven testing actions into machine-executed processes, effectively improving testing efficiency and serving as an essential means to ensure software reliability. Based on this, automated testing technology has developed significantly, with keyword-driven automated testing being the most mainstream approach. However, as software becomes increasingly large-scale and complex, the current sequential execution mode of keyword steps struggles to support complex, cross-cutting test logic designs. Therefore, the need to design various complex cross-cutting test logics using simple and intuitive keyword configuration methods within a keyword-driven testing framework to automate testing is extremely urgent.

[0003] Currently, in keyword-driven automated testing frameworks, keyword steps can only be executed sequentially: test instructions are executed in a single-threaded manner, first performing the test operation and then waiting for the result to be interpreted. While this sequential execution flow can meet most testing needs, the operation steps cannot be executed in parallel, making it difficult to further improve testing efficiency. Furthermore, when designing serialized test logic for cross-execution testing requirements, the keyword step configuration is overly complex and lacks sufficient or even full support for typical testing scenarios: if the test result is not generated immediately after instruction execution but requires a long delay for interpretation, or even if the arrival time of the result cannot be accurately predicted and requires continuous monitoring, this leads to a large amount of ineffective waiting time and low testing efficiency. Especially when waiting for multiple execution results simultaneously without a certain output order, single-threaded sequential execution often only allows for a definite order of judgment, which may result in the output results being overwritten or lost, thus missing the interpretation opportunity and causing test case execution failure. Therefore, it is often necessary to forcibly change the system under test to specify the output order, or to manually review the results afterward, greatly increasing testing costs and reducing automation and testing efficiency. Summary of the Invention

[0004] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide a keyword-driven parallel execution system for automated testing, which aims to solve the problem of low testing efficiency caused by single-threaded execution of keyword steps in the existing keyword-driven automated testing framework.

[0005] To address the aforementioned technical problems, this invention discloses a keyword-driven automated test parallel execution system, comprising:

[0006] The parallel execution management module is used to manage test sub-threads, including creating test sub-threads, saving test sub-thread data, and terminating test sub-threads.

[0007] The keyword step parallel execution module is used to create an executable environment for test sub-threads, in which keyword steps are executed in a serial order.

[0008] The information interaction module is used to realize the information transmission and interaction between the main test thread and each test sub-thread, as well as among the test sub-threads;

[0009] The exception handling module is used to determine whether to continue executing the test case based on the test case execution criteria when an error occurs during the execution of a keyword step.

[0010] The execution result management module is used to statistically analyze and manage the execution results of keyword steps.

[0011] The log information module is used to merge the log information of keyword execution from each test sub-thread to form a complete record of the test execution process;

[0012] Parallel execution keyword library, used to integrate parallel execution mechanisms into keyword-driven automated testing architecture, encapsulated in a keyword manner;

[0013] The parallel test driver module is used to design a parallel test instruction driver framework based on the VTest embedded simulation system under the newly added keyword-based parallel execution framework. It designs the main driver logic and sub-driver logic, which correspond to the test main thread and test sub-thread of automated testing, respectively, to realize fully digital automated testing of embedded software.

[0014] In the aforementioned keyword-driven automated test parallel execution system, the parallel execution management module is used for:

[0015] After receiving the message to start the test sub-thread, the test sub-thread is created using the thread keyword and the test sub-thread ID is returned; when the test case requires the design of parallel execution keyword test steps, the test main thread sends the message to start the test sub-thread to the parallel execution management module.

[0016] Define a data structure for storing test sub-thread data; each data structure object corresponds to storing one test sub-thread data, including: handle information, test sub-thread ID, and test sub-thread name;

[0017] After the keyword step in the test sub-thread is completed, the corresponding test sub-thread will automatically terminate; or, if the keyword step in the test sub-thread is not completed, the corresponding test sub-thread will be forcibly terminated.

[0018] In the aforementioned keyword-driven automated test parallel execution system, the keyword step parallel execution module is used for:

[0019] Based on the context of the main test thread, create the execution context of the current test child thread; global variables use pointers to maintain consistency, while local variables are recreated to maintain independence.

[0020] Within the created execution context, keyword steps are executed sequentially. For basic keywords, the executable function is called directly by the keyword execution engine based on the basic keyword index. For composite keywords, a recursive index is used until the basic keyword is reached, and then the executable function is called by the keyword execution engine.

[0021] In the aforementioned keyword-driven parallel execution system for automated testing, the contexts that support keyword execution include: variable lists, keyword function lists, namespaces, and test case sets.

[0022] In the aforementioned keyword-driven automated test parallel execution system, the information interaction module provides two information interaction methods: asynchronous information interaction through global variables and synchronization of each test sub-thread through semaphores.

[0023] In the keyword-driven parallel execution system for automated testing described above, the exception handling module determines whether to continue executing a test case: if it determines that the test case should continue, it saves the exception information in the current test sub-thread; if it determines that the test case should stop, each test sub-thread exits, the test case execution ends, and the test case fails.

[0024] In the aforementioned keyword-driven automated test parallel execution system, the execution result management module, when statistically analyzing and managing the execution results of keyword steps, has the following:

[0025] If the test sub-thread finishes before the test main thread completes, and all keyword steps are executed correctly, the test case passes; otherwise, the test case fails.

[0026] If the test sub-thread has not finished after the test main thread has finished, then the execution of the keyword step in the test sub-thread will be forcibly terminated, and the keyword execution result of the test sub-thread will be set to failure.

[0027] In the aforementioned keyword-driven automated parallel execution system, log information is organized using test case sets, test cases, combined keywords, and basic steps, and displayed in a tree structure. When a keyword is executed in a single thread, log information is recorded sequentially according to the order of keyword execution. When multiple keywords are executed in parallel, log information is processed in parallel to prevent cross-referencing and errors. An information management module is configured for each keyword execution context to process log information sequentially during keyword execution. After a test case finishes, the corresponding log information for the test sub-thread is added to the log information section where the main test thread starts the test sub-thread, thus forming complete test case execution information.

[0028] In the aforementioned keyword-driven automated test parallel execution system, the parallel execution keyword library encapsulates the following basic keywords: parallel initial setup keyword, parallel execution start keyword, parallel termination operation keyword, and forced termination of parallel execution keyword.

[0029] In the aforementioned keyword-driven parallel execution system for automated testing, the parallel test-driven module, when implementing fully digital automated testing of embedded software, has the following characteristics:

[0030] During test initialization, the main test thread establishes a connection with the driver test main processing module and performs test operations serially.

[0031] When the test sub-thread is started at a certain simulation moment, the test main thread sends a command to the driver test main processing module to create the driver test sub-processing module through the parallel initial setting keyword; after receiving the creation command, the driver test main processing module creates the driver test sub-processing module and passes the port information.

[0032] The main test thread creates test child threads by executing the startup keyword in parallel;

[0033] The driver sub-processing module initiates a communication connection with the test sub-thread based on the port information to receive the driver test instructions sent by the test sub-thread; and executes the driver tests indicated by the driver test instructions in sequence until all driver tests at the current simulation moment are completed, and then feeds back the execution results.

[0034] The main driver test processing module sends the execution results fed back by the driver sub-processing module to the main test thread;

[0035] After receiving the execution result, the main test thread continues to execute subsequent keyword test steps; after all keyword test steps are completed, the operation keywords are terminated in parallel, and a termination command is sent to the driver test subprocessing module.

[0036] After receiving the termination instruction, the driver test sub-processing module stops receiving driver test instructions, returns termination information to the test sub-thread, closes the communication connection, and exits.

[0037] After receiving the termination message, the test sub-thread closes the connection, saves the keyword execution process information, and then the test sub-thread terminates.

[0038] The present invention has the following advantages:

[0039] This invention discloses a keyword-driven automated test parallel execution system. Building upon a keyword-driven automated testing architecture, it adds a keyword-based parallel execution mechanism, encapsulated in keyword form. Testers can still design test cases using keyword configuration. Based on testing requirements, they can intuitively design parallel test logic by referencing the required parallel execution keywords, eliminating the need to break down parallel logic and sequentially arrange and combine it into complex single-threaded test cases. Furthermore, for test requirements involving multiple operations and results with uncertain execution order, single-threaded execution is insufficient, while the parallel execution mechanism provides a simple and effective solution. The parallel execution mechanism simplifies test case design complexity, expands the application scope and applicable test scenarios of the keyword-driven automated testing framework, and improves testing efficiency. Attached Figure Description

[0040] Figure 1 This is a structural block diagram of a keyword-driven automated test parallel execution system according to an embodiment of the present invention;

[0041] Figure 2 This is a technical architecture diagram of a test driver based on a fully digital simulation system in an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments disclosed in the present invention will be described in further detail below with reference to the accompanying drawings.

[0043] One of the core ideas of this invention is that, based on a keyword-driven automated test execution framework, a parallel execution mechanism for keyword test steps is added, and this parallel execution mechanism is encapsulated as a keyword; by designing test cases through parallel execution of keywords, the parallel execution of keyword test steps is achieved.

[0044] In this embodiment, test sub-threads can be started using the parallel execution keyword. The test sub-threads execute test steps in parallel with the main test thread, and the parallel execution management module manages the test sub-threads. During the execution of the test steps, the test steps inject test data into the system under test through the parallel test driver module, and then obtain the output data of the software under test for interpretation. If the execution is correct, it passes; otherwise, the exception handling module throws an exception. After the test is completed, the log information module records the test step execution information in detail, and the execution result management module compiles the execution results. The information between the main test thread and the test sub-threads, as well as the information between the test sub-threads, is stored and forwarded by the information interaction module.

[0045] This keyword-driven automated test parallel execution system may specifically include:

[0046] The parallel execution management module is used to manage the test sub-threads.

[0047] In this embodiment, the management of test sub-threads mainly includes: creating test sub-threads, saving test sub-thread data, and terminating test sub-threads.

[0048] Creating a test sub-thread: Upon receiving the message to start a test sub-thread, a test sub-thread is created using the thread keyword, and the test sub-thread ID is returned. Specifically, when test cases require parallel execution keyword test steps, the main test thread sends a message to the parallel execution management module to start the test sub-thread.

[0049] Storing test sub-thread data: Defines a data structure for storing test sub-thread data. Each data structure object corresponds to one test sub-thread data, including: handle information, test sub-thread ID, and test sub-thread name, etc.

[0050] End test sub-thread: Automatically end the corresponding test sub-thread after the keyword step in the test sub-thread has been executed; or, forcibly end the corresponding test sub-thread before the keyword step in the test sub-thread has been executed.

[0051] The keyword step parallel execution module is used to create an executable environment for test sub-threads, in which keyword steps are executed in a serial order.

[0052] In this embodiment, the keyword step parallel execution module is mainly responsible for creating the executable environment and executing keyword steps. The context supporting keyword execution includes environment elements such as variable lists, keyword function lists, namespaces, and test case sets. Therefore, when executing test sub-threads, it is necessary to create a context.

[0053] Preferably, the keyword step parallel execution module is specifically used for:

[0054] Based on the context of the main test thread, create the execution context of the current test child thread; global variables are handled by pointers to maintain consistency, while local variables are recreated to maintain independence.

[0055] Within the created execution context, keyword steps are executed sequentially. For basic keywords, the executable function is called directly by the keyword execution engine based on the basic keyword index; for composite keywords, a recursive index is used until the basic keyword is reached, and then the executable function is called by the keyword execution engine.

[0056] The information interaction module is used to realize the information transmission and interaction between the main test thread and each test sub-thread, as well as among the test sub-threads.

[0057] In this embodiment, the information interaction module provides two information interaction methods: asynchronous information interaction via global variables and synchronization of each test sub-thread via semaphores. In the keyword-driven automated test execution framework, a context mechanism for keyword execution is maintained. Information storage is achieved through the keyword "set global variable," and information access is achieved through the keyword "get global variable." Thread blocking is achieved through the keyword "wait for semaphore," and thread resumption is achieved through the keyword "set semaphore."

[0058] The exception handling module is used to determine whether to continue executing the test case based on the test case execution criteria when an error occurs during the execution of a keyword step.

[0059] In this embodiment, when the exception handling module determines whether the test case should continue to execute, it does the following: if it determines that the test case should continue to execute, it saves the exception information in the current test sub-thread; if it determines that the test case should stop executing, each test sub-thread exits, the test case execution ends, and the test case fails.

[0060] The execution result management module is used to statistically analyze and manage the execution results of keyword steps.

[0061] In this embodiment, as shown in Table 1, when the execution result management module performs statistics and management on the execution results of keyword steps, it has the following:

[0062] If the test sub-thread finishes before the test main thread completes, and all keyword steps are executed correctly, the test case passes; otherwise, the test case fails.

[0063] If the test sub-thread has not finished after the test main thread has finished, then the execution of the keyword step in the test sub-thread will be forcibly terminated, and the keyword execution result of the test sub-thread will be set to failure.

[0064]

[0065]

[0066] Table 1. Criteria for Judging Test Case Execution Results

[0067] The log information module is used to merge the log information of keyword execution from each test sub-thread to form a complete record of the test execution process.

[0068] In this embodiment, log information is organized using test case sets, test cases, combined keywords, and basic steps, and displayed in a tree structure. When a keyword is executed in a single thread, log information is recorded sequentially according to the order of keyword execution. When multiple keywords are executed in parallel, log information is processed in parallel to prevent cross-contamination. An information management module is configured for each keyword execution context to process log information sequentially during keyword execution. After the test case ends, the corresponding log information of the test sub-thread is added to the log information of the test sub-thread started by the main test thread, thus forming complete test case execution information.

[0069] Parallel execution keyword library, used to integrate parallel execution mechanisms into keyword-driven automated testing architectures, encapsulating them in a keyword manner.

[0070] In this embodiment, the creation of an asynchronous keyword mainly consists of three parts: a parallel initialization keyword, a parallel execution start keyword, and a parallel termination keyword. The parallel initialization keyword mainly configures the initialization operations before the parallel execution of the keyword, preparing for the start of the asynchronous thread. This operation is performed in the main execution thread. The parallel execution start keyword mainly configures the keyword steps that need to be executed in parallel in the asynchronous thread. The parallel termination keyword mainly configures the subsequent operations that need to be performed after the parallel keyword steps are completed.

[0071] The parallel keyword design pattern is shown below:

[0072] Parallel initialization keywords:

[0073] Initialization operation keyword steps;

[0074] Parallel execution startup keyword:

[0075] Keyword Step 1;

[0076] Keyword step 2;

[0077] ...

[0078] Keyword step n;

[0079] Keyword for ending parallel operations:

[0080] End of operation keyword steps.

[0081] Preferably, the parallel execution keyword library encapsulates the following basic keywords: parallel initial setup keywords, parallel execution start keywords, parallel termination operation keywords, and forced termination of parallel execution keywords.

[0082]

[0083] Table 2, Examples of Basic Keywords

[0084] The parallel test driver module is used to design a parallel test instruction driver framework based on the VTest embedded simulation system under the newly added keyword-based parallel execution framework. It designs the main driver logic and sub-driver logic, which correspond to the test main thread and test sub-thread of automated testing, respectively, to realize fully digital automated testing of embedded software.

[0085] In this embodiment, under the newly added keyword-based parallel execution framework, a parallel test instruction-driven framework based on the VTest embedded simulation system is designed, such as... Figure 2 As shown, the main driver logic and sub-driver logic are designed, corresponding to the main execution thread and sub-execution thread of the automated test, respectively. When implementing fully digital automated testing of embedded software, the parallel test driver module has the following features:

[0086] During test initialization, the main test thread establishes a connection with the driver test main processing module and performs test operations serially.

[0087] When the test sub-thread is started at a certain simulation moment, the test main thread sends a command to the driver test main processing module to create the driver test sub-processing module through the parallel initial setting keyword; after receiving the creation command, the driver test main processing module creates the driver test sub-processing module and passes the port information.

[0088] The main test thread creates test child threads by executing the startup keyword in parallel;

[0089] The driver subprocessing module initiates a communication connection with the test sub-thread based on the port information to receive the driver test instructions sent by the test sub-thread; and executes the driver tests indicated by the driver test instructions in sequence until all driver tests at the current simulation moment are completed, and then feeds back the execution results.

[0090] The main driver test processing module sends the execution results fed back by the driver sub-processing module to the main test thread;

[0091] After receiving the execution result, the main test thread continues to execute subsequent keyword test steps; after all keyword test steps are completed, the operation keywords are terminated in parallel, and a termination command is sent to the driver test subprocessing module.

[0092] After receiving the termination instruction, the driver test sub-processing module stops receiving driver test instructions, returns termination information to the test sub-thread, closes the communication connection, and exits.

[0093] After receiving the termination message, the test sub-thread closes the connection, saves the keyword execution process information, and then the test sub-thread terminates.

[0094] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

[0095] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A keyword-driven automated test parallel execution system, characterized in that, include: The parallel execution management module is used to manage test sub-threads, including creating test sub-threads, saving test sub-thread data, and terminating test sub-threads. The keyword step parallel execution module is used to create an executable environment for test sub-threads, in which keyword steps are executed in a serial order. The information interaction module is used to realize the information transmission and interaction between the main test thread and each test sub-thread, as well as among the test sub-threads; The exception handling module is used to determine whether to continue executing the test case based on the test case execution criteria when an error occurs during the execution of a keyword step. The execution result management module is used to statistically analyze and manage the execution results of keyword steps. The log information module is used to merge the log information of keyword execution from each test sub-thread to form a complete record of the test execution process; Parallel execution keyword library, used to integrate parallel execution mechanisms into keyword-driven automated testing architecture, encapsulated in the form of keywords; The parallel test driver module is used to design a parallel test instruction driver framework based on the VTest embedded simulation system under the newly added keyword-based parallel execution framework. It designs the main driver logic and sub-driver logic, which correspond to the test main thread and test sub-thread of automated testing, respectively, to realize fully digital automated testing of embedded software. The keyword step parallel execution module is specifically used to: create the current test sub-thread's execution context based on the main test thread's context; global variables are handled using pointers to maintain consistency, while local variables are recreated to maintain independence; within the created execution context, keyword steps are executed sequentially; for basic keywords, the executable function is called directly by the keyword execution engine based on the basic keyword index; for composite keywords, a recursive index is used until the basic keyword is reached, and then the executable function is called by the keyword execution engine.

2. The keyword-driven automated test parallel execution system according to claim 1, characterized in that, The parallel execution management module is used for: After receiving the message to start the test sub-thread, the test sub-thread is created using the thread keyword and the test sub-thread ID is returned; when the test case requires the design of parallel execution keyword test steps, the test main thread sends the message to start the test sub-thread to the parallel execution management module. Define a data structure for storing test sub-thread data; each data structure object corresponds to storing one test sub-thread data, including: handle information, test sub-thread ID, and test sub-thread name; After the keyword step in the test sub-thread is completed, the corresponding test sub-thread will automatically terminate; or, if the keyword step in the test sub-thread is not completed, the corresponding test sub-thread will be forcibly terminated.

3. The keyword-driven automated test parallel execution system according to claim 1, characterized in that, Contexts that support keyword-executable functions include: variable lists, keyword function lists, namespaces, and test case sets.

4. The keyword-driven automated test parallel execution system according to claim 1, characterized in that, The information interaction module provides two information interaction methods: asynchronous information interaction through global variables and synchronization of each test sub-thread through semaphores.

5. The keyword-driven automated test parallel execution system according to claim 1, characterized in that, When determining whether a test case should continue execution, the exception handling module does the following: if it determines that the test case should continue execution, it saves the exception information in the current test sub-thread; if it determines that the test case should stop execution, each test sub-thread exits, the test case execution ends, and the test case fails.

6. The keyword-driven automated test parallel execution system according to claim 1, characterized in that, When the execution result management module performs statistics and management on the execution results of keyword steps, it has the following functions: If the test sub-thread finishes before the test main thread completes, and all keyword steps are executed correctly, the test case passes; otherwise, the test case fails. If the test sub-thread has not finished after the test main thread has completed, then the execution of the keyword step in the test sub-thread will be forcibly terminated, and the keyword execution result of the test sub-thread will be set to failure.

7. The keyword-driven automated test parallel execution system according to claim 1, characterized in that, Log information is organized using test case sets, test cases, combined keywords, and basic steps, and displayed in a tree structure. When a keyword is executed in a single thread, log information is recorded sequentially according to the order of keyword execution. When multiple keywords are executed in parallel, log information is processed in parallel to prevent log information from overlapping and becoming disordered. An information management module is configured for each keyword execution context to process log information during keyword execution sequentially. After the test case ends, the corresponding log information of the test sub-thread is added to the log information of the test sub-thread started by the test main thread, thus forming complete test case execution information.

8. The keyword-driven automated test parallel execution system according to claim 1, characterized in that, The parallel execution keyword library encapsulates the following basic keywords: parallel initialization keyword, parallel execution start keyword, parallel termination operation keyword, and forced termination of parallel execution keyword.

9. The keyword-driven automated test parallel execution system according to claim 1, characterized in that, When implementing fully digital automated testing of embedded software, the parallel test-driven module has the following features: During test initialization, the main test thread establishes a connection with the driver test main processing module and performs test operations serially. When the test sub-thread is started at a certain simulation moment, the test main thread sends a command to create the driver test sub-processing module to the driver test main processing module through the parallel initial setting keyword. After receiving the creation command, the main driver test processing module creates the driver test sub-processing module and passes the port information. The main test thread creates test child threads by executing the startup keyword in parallel; The driver sub-processing module initiates a communication connection with the test sub-thread based on the port information in order to receive driver test commands sent by the test sub-thread. The driver tests indicated by the driver test instructions are executed sequentially until all driver tests at the current simulation moment are completed, and the execution results are fed back. The main driver test processing module sends the execution results fed back by the driver sub-processing module to the main test thread; After receiving the execution result, the main test thread continues to execute subsequent keyword test steps; after all keyword test steps are completed, the operation keywords are terminated in parallel, and a termination command is sent to the driver test subprocessing module. After receiving the termination instruction, the driver test sub-processing module stops receiving driver test instructions, returns termination information to the test sub-thread, closes the communication connection, and exits. After receiving the termination message, the test sub-thread closes the connection, saves the keyword execution process information, and then the test sub-thread terminates.

Images