A cross-platform multi-scene automated testing system and a control method thereof

Abstract

The application provides a cross-platform multi-scene automatic test system and a control method thereof, which comprises a front-end module, a middleware module and a back-end module.The front-end module provides a graphical interactive interface for test environment configuration and test result display, and realizes data communication with the middleware module; the middleware module sends configuration instructions of the front-end module to the back-end module, subscribes to test data of the back-end module, and publishes the test data to the front-end module; and the back-end module automatically deploys a test environment according to the received configuration instructions, controls a test board card to automatically execute a test item, and publishes a test report to the front-end module in real time through the middleware module; the test environment is automatically deployed according to the received configuration instructions, the test board card is controlled to automatically execute the test item, and the test report is published to the front-end module in real time, so that the manual test mode in the prior art is replaced by an automatic test mode, the labor cost and the manual time consumption are saved, and the test efficiency is improved.

Description

Technical Field

[0001] This invention relates to the technical field of testing systems and methods, specifically to a cross-platform, multi-scenario automated testing system and its control method. Background Technology

[0002] To ensure the normal operation of computer software, it is necessary to test it in order to promptly identify errors and vulnerabilities in system analysis and design.

[0003] The existing testing systems and methods still have the following defects and shortcomings:

[0004] 1) Manual testing is inefficient: Traditional testing techniques often rely on manual execution, with a high proportion of repetitive operations, a high proportion of single-round testing, long testing time, and labor costs accounting for more than 40% of the total project cost.

[0005] 2) Insufficient test standardization: Different testing teams often use different testing processes and have inconsistent script formats, resulting in poor consistency of test results and a human error rate as high as 25%.

[0006] 3) Insufficient coverage of complex scenarios: Multi-device collaborative testing (such as communication between the board and the test machine) requires manual configuration of the network and dependent environment. It is difficult to simulate abnormal scenarios (such as timeout, module loading failure, etc.), and the test coverage is generally less than 70%.

[0007] 4) Difficulty in data traceability and analysis: Test logs are stored in a scattered manner, lacking real-time monitoring and visualization tools, and the average time to locate problems exceeds 4 hours.

[0008] 5) Existing automated testing technologies are mostly designed for single platforms (such as Windows or Linux), making it difficult to support embedded systems (such as SylixOS) and multi-device linkage scenarios. Furthermore, their scalability is limited, and they cannot flexibly adapt to the personalized testing processes of enterprises.

[0009] Therefore, there is an urgent need to provide a new solution to address the defects and shortcomings of the existing technologies. Summary of the Invention

[0010] To address the shortcomings and deficiencies in the existing technologies, this invention provides a cross-platform, multi-scenario automated testing system and its control method.

[0011] The specific solution provided by this invention is as follows:

[0012] A cross-platform, multi-scenario automated testing system, the system comprising:

[0013] The front-end module is connected to the middleware module. It provides a graphical interactive interface for configuring the test environment and displaying test results, while also communicating data with the middleware module.

[0014] The middleware module is connected to both the front-end module and the back-end module. It sends the configuration instructions from the front-end module to the back-end module, subscribes to the test data from the back-end module, and publishes it to the front-end module.

[0015] The backend module automatically deploys the test environment according to the received configuration instructions, controls the test board to automatically execute test projects, and publishes the test report to the frontend module in real time through the middleware module;

[0016] Its features are:

[0017] It also includes a test template module, which comprises a test script unit, a test requirement unit, and a test record unit; wherein,

[0018] The test requirement unit and the test script unit are connected to the front-end module. The test requirement unit obtains test requirement parameters, and the test script unit defines test script parameters based on the obtained test requirement parameters.

[0019] The test recording unit is connected to the backend module to record the test process and test results in real time and generate structured test records in real time.

[0020] As a further preferred embodiment of the present invention, when configuring the test environment in the front-end module, the configured test environment parameters include at least the IP addresses of the test board and the test companion machine, the test type, and the template file.

[0021] As a further preferred embodiment of the present invention, the middleware module realizes bidirectional communication between the front-end module and the back-end module through the VSOA API interface.

[0022] As a further preferred embodiment of the present invention, the backend module executes the test engine according to the received test instructions to perform at least the following types of tests: BSP test, Kernel test and custom test;

[0023] When the backend module executes test projects, it automatically configures the test server environment and the IP addresses of the test board and the test server, and supports multi-platform collaboration. It automatically trims test cases according to the test requirements document and test scripts, executes test projects in the order of preset fields, captures test logs in real time and marks the abnormal test results. It automatically generates test reports in preset formats, and the test reports support one-click export.

[0024] As a further preferred embodiment of the present invention, the test requirement parameters include at least software package information, hardware configuration information, software configuration information, test equipment peripheral information, supporting test equipment information, and requirement list information.

[0025] As a further preferred embodiment of the present invention, the test script parameters include at least test case parameters, dependency relationships, and keyword substitution relationships.

[0026] As a further preferred embodiment of the present invention, the system further includes a test companion device, which is connected to the front-end module. The test companion device simulates network protocol verification and interacts with the test board.

[0027] As a further preferred embodiment of the present invention, the dependency relationship between the test server and the test board is determined by a preset field or a preset program, and the priority of the test server and the test board is determined according to the dependency relationship:

[0028] When the dependency relationship between the test bench and the test board is that the test board is more dependent on the test board, the test board is considered to have a higher priority.

[0029] When the dependency relationship between the test bench and the test board is that the test bench is more dependent on the test bench, the test bench is considered to have a higher priority.

[0030] As a further preferred embodiment of the present invention, the test companion machine is also provided with an anomaly simulation unit, and the anomaly simulation items in the anomaly simulation unit include at least timeout duration anomaly simulation, environmental limitation anomaly simulation, board restart anomaly simulation, and module loading failure anomaly simulation.

[0031] Furthermore, the present invention also provides a control method for a cross-platform, multi-scenario automated testing system, characterized by comprising the following steps:

[0032] S1: Configure the test environment through the front-end module;

[0033] S2: The middleware module sends configuration instructions to the backend module, and the backend module automatically deploys the test environment according to the received configuration instructions;

[0034] S3: The test board executes test items sequentially according to the test script parameters and sends test logs to the front-end module in real time;

[0035] S4: Determine the dependency relationship between the test bench and the test board;

[0036] S5: The backend generates a test report and pushes it to the frontend module through the middleware module;

[0037] S6: Automatically archive test logs to the installation directory.

[0038] Compared with existing technologies, the technical effects that this invention can achieve include:

[0039] 1) This invention provides a cross-platform, multi-scenario automated testing system and its control method. The system automatically deploys the test environment according to the received configuration instructions, controls the test board to automatically execute test items, and publishes the test report to the front-end module in real time. This replaces the manual testing method in the prior art with an automated testing method, saving labor costs and time, and improving testing efficiency.

[0040] 2) This invention provides a cross-platform, multi-scenario automated testing system and its control method. Through the communication connection between the front-end module, middleware module and back-end module, it realizes full-process automation from environment setup to test execution and report generation. It adopts a unified test process and script format, and replaces the lack of standardization in the manual testing process with the standardization of the automatic process.

[0041] 3) This invention provides a cross-platform, multi-scenario automated testing system and its control method. By adopting a synchronous testing method between the test board and the test companion machine, and by automatically configuring the test environment and dependencies, it can realize abnormal environment simulation, including simulation of timeout duration anomalies, simulation of environmental limitation anomalies, simulation of board restart anomalies, and simulation of module loading failure anomalies.

[0042] 4) This invention provides a cross-platform, multi-scenario automated testing system and its control method. By sending test logs and test reports to the front-end module in real time and automatically and centrally archiving the test logs, the front-end module provides a graphical interactive interface to provide visualization tools to facilitate real-time monitoring, thereby improving data traceability and analysis capabilities and facilitating subsequent data statistics and organization.

[0043] 5) This invention provides a cross-platform, multi-scenario automated testing system and its control method. Through a three-layer architecture of front-end module, middleware module and back-end module, combined with log subscription and publishing mechanism and template-driven technology, it realizes full-process automation from environment deployment to report generation. It can support BSP, Kernel and custom tests, and solves the problems of high manpower cost, insufficient standardization and difficulty in covering complex scenarios in traditional testing. It is suitable for testing work in high reliability fields such as embedded development, operating system testing and aerospace. Attached Figure Description

[0044] Figure 1 The diagram shown is a logical structure diagram of the automated testing system provided by the present invention.

[0045] Figure 2 The diagram shows the steps of the automated testing system control method provided by the present invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," "outer," "front end," "rear end," "both ends," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0048] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0049] [First Embodiment]

[0050] like Figure 1 As shown, the first embodiment of the present invention provides a cross-platform, multi-scenario automated testing system, the system comprising:

[0051] The front-end module connects to the middleware module. It provides a graphical user interface (GUI) for configuring the test environment and displaying test results, while also communicating with the middleware module. This communication facilitates sending configuration commands to the back-end module via the middleware module, while simultaneously receiving test data from the back-end module and displaying it through the GUI. Visualization tools are provided to enable real-time monitoring of exported logs and reports. The front-end module is developed based on the Vue.js framework and supports real-time WebSocket data retrieval from the middleware.

[0052] In the front-end module of this embodiment, when configuring the test environment, the configured test environment parameters include at least the IP addresses of the test board and the test companion machine, the test type, and the template file.

[0053] In this embodiment, the test types include at least BSP tests, Kernel tests, and custom tests.

[0054] The backend module executes tests performed by the test engine based on received test instructions. These tests include at least BSP tests, kernel tests, and custom tests. BSP tests verify the functionality and performance of the Board Support Package (BSP), typically including hardware peripheral verification tests. Kernel tests, or Linux kernel tests, typically include source code debugging and SylixOS API interface testing. Custom tests define corresponding test logic through test scripts to implement the corresponding test items.

[0055] The middleware module connects to both the front-end and back-end modules. It sends configuration commands from the front-end module to the back-end module while simultaneously subscribing to test data from the back-end module and publishing it to the front-end module. In this embodiment, the middleware module uses the VSOA API interface and a publish-subscribe mechanism to achieve bidirectional communication between the front-end and back-end modules. For example... Figure 1 As shown, it can subscribe to test data and test logs produced by the backend module through the VSOA server (which are packaged by the backend module and stored in the log server), and then produce the required files for the frontend through the VSOA client and publish them to the frontend module. At the same time, it forwards the configuration instructions of the frontend module to the backend module and supports remote calls of programmable controller RPC and high-bandwidth data flow.

[0056] The backend module automatically deploys the test environment based on the received configuration instructions, controls the test board's Python + JS-based test execution engine, parses test scripts (such as the JSON scripts described later), automatically executes test projects, and publishes test reports to the frontend module in real time through the middleware module. Corresponding to the frontend module, the backend module should execute the test engine based on the received test instructions to perform at least the following types of tests: BSP tests, Kernel tests, and custom tests.

[0057] When the backend module executes a test project, it can achieve full automation in the following ways:

[0058] Environment Deployment: One-click installation of the test server environment, automatic configuration of the IP addresses of the test board and the test server, and support for collaborative use on multiple platforms such as SylixOS, Linux, and Windows;

[0059] Test execution: Test cases are automatically tailored according to the test requirements document (e.g., the freq file mentioned below) and the test script (e.g., the JSON script mentioned below), and test items are executed in the order of the preset fields (CGNumber). Test logs are captured in real time and the abnormal test results are marked.

[0060] Report generation: Automatically generates test reports in preset formats (such as DOCX or EXCEL format), including test parameters such as pass rate, performance indicators, and exception logs, and supports one-click export function.

[0061] The testing system provided in this embodiment also includes a test template module, which includes a test script unit, a test requirement unit, and a test record unit. In this embodiment, the test script in the test script unit can be implemented using a JSON script, the test requirement document in the test requirement unit can be implemented using a freq file, and the test record file in the test record unit can be implemented using an ftemp file.

[0062] The test requirement unit and test script unit are connected to the front-end module. The test requirement unit obtains test requirement parameters, and the test script unit then defines test script parameters based on the obtained test requirement parameters; among them,

[0063] The test requirement parameters mentioned in this embodiment include at least the following: software package information (e.g., operating system version, BSP version, dependent library version, etc.), hardware configuration information (e.g., CPU information, memory capacity, storage capacity, network card information, etc.), software configuration information (e.g., Tick clock frequency, GPIO interrupt, etc.), test device peripheral information (serial port number of serial port device, network card device, CAN device, CANFD, RTC, SD bus, SPI bus, I2C bus, etc.), supporting test device information (device serial port, network card information, etc.), and requirement list information (functional test items, performance test items, stability test items, etc.).

[0064] The test script parameters should at least include test case parameters, dependencies (e.g., the FpatsDepend_M1 / M2 field can be used to specify dependencies between the local or test machine programs), and keyword substitution relationships (e.g., ...). <fpatsip>(Dynamically replaced with the test board IP).

[0065] The test recording unit is connected to the backend module to record the test process and test results in real time and generate structured test records in real time. These test records can be downloaded using preset software (such as Lark) and modified locally.

[0066] The automated testing system provided in this embodiment also includes a test companion unit (PCU). The PCU is connected to the front-end module. By supporting multiple systems, the PCU enables cross-device TCP / UDP communication testing. The PCU simulates network protocol verification and interacts with the test board. By configuring the PCU, it can facilitate data interaction with the device under test (DUT). When the DUT requires tests for serial communication, network communication, CAN communication, or other forms of communication, the PCU acts as the sender or receiver of data. During testing—whether functional, performance, or other tests—if the PCU malfunctions and cannot interact with the DUT, the front-end module will feed back the output of the back-end module and save it in the log file.

[0067] As an auxiliary testing device, the test bench operates throughout the entire testing process. Its core function is to cooperate with the test board to complete cross-device collaborative testing and abnormal scenario simulation. Its operation specifically includes the following steps:

[0068] S010: Test Configuration Phase

[0069] S011: The front-end module receives the IP address of the test machine, system type (e.g., SylixOS, Linux, Windows, etc.) and test type (e.g., BSP / Kernel / Custom, etc.) from the user through a graphical interactive interface.

[0070] S012: The middleware module forwards configuration commands to the backend module through the VSOA communication framework. The backend module parses the test requirement document (.freq) and JSON template, and automatically deploys the test server environment (such as one-click installation of dependent programs, configuration of network parameters, etc.) to ensure that the test server and the test board can communicate with each other via IP.

[0071] S020: Test Execution Phase

[0072] S021: Based on the FpatsDepend_M2 field in the JSON template (indicating a program or operation that is more dependent on the test server), the backend module triggers the test server to execute a pre-processing task. For example, in TCP communication testing, the test server starts the enet_test_auto program as a client according to the FpatsDepend_M2 configuration and establishes a connection with the server of the test board;

[0073] S022: The test companion machine executes test commands in real time (such as sending network data packets, loading specified modules, etc.) and pushes the execution logs to the middleware module through the VSOA publish-subscribe mechanism, and then synchronizes them to the front-end module for real-time monitoring;

[0074] S023: Supports simulation of abnormal scenarios. It uses exception simulation projects in the JSON template to simulate Timeout, Forbid_env, Reboot, and module loading failure, etc., to verify the fault tolerance capability of the test board.

[0075] S030: Results Processing Stage

[0076] S031: The test results of the test companion (such as execution status, abnormal information, etc.) are aggregated to the backend module through the middleware module and used together with the results of the test board to generate a structured report (which can be in DOCX / EXCEL format, etc.).

[0077] S032: All logs (including the interaction records between the test machine and the test board) are automatically archived to the local directory, supporting subsequent backtracking and problem reproduction.

[0078] When discrepancies arise between the test results of the auxiliary testing machine and the system test results, the handling method relies on manual analysis combined with log tracing. The specific process is as follows:

[0079] S041: Full Log Recording: Captures the complete interaction logs between the test machine and the test board (including their respective execution steps, return values, exception markers, etc.), and displays them in real time through a graphical interactive interface via the front-end module. The logs also support keyword filtering (such as "fail", "error", etc.).

[0080] S042: Anomaly Correlation Analysis: When a disagreement occurs, the cause of the disagreement can be located by checking the dependency configuration in the JSON template (such as whether FpatsDepend_M2 is correctly defined), the execution order of both parties (the priority determined by the CGNumber field), and the anomaly trigger point (such as timeout time, reason for module loading failure).

[0081] S043: Visualized Report Presentation: The test report will list the test results of the test bench and the test board separately and mark the points of divergence (e.g., one side shows "pass" and the other side shows "fail"), and associate them with the corresponding log segments to help manual identification of the problem.

[0082] In this embodiment, the dependency relationship between the test companion and the test board is determined by preset fields or preset procedures, for example:

[0083] The field FpatsDepend_M1 can be used to indicate that the current test case is more dependent on the test board (specifically, it may depend on its application, kernel module (.ko) or environment configuration, etc.).

[0084] The field FpatsDepend_M2 can be used to indicate that the current test case is more dependent on the test companion machine (specifically, it may depend on its application, network service, or collaborative operation).

[0085] For example, when the test device needs to transmit a stream to the device under test to verify its network bandwidth, the specific preset fields are as follows:

[0086] " / apps / swissAKS / net / iperf":{

[0087] "parameter": "-s"

[0088] "Description": "UDP receive iperf server"

[0089] "Builtin": true.

[0090] "CGNumber": 3,

[0091] "Depend": true.

[0092] "FpatsDepend_M1" :{

[0093] " / apps / swissAKS / net / iperf" :{

[0094] "Parameter": "-c <fpatsip>-t 100 –u –b 0m”

[0095] "Builtin": true.

[0096] "CGNumber": 4,

[0097] "Description": "UDP receiving iperf client"

[0098] }

[0099] }

[0100] In this example program, the "FpatsDepend_M1" field indicates whether a test companion device is required; it can be seen that the test cases are more dependent on the test board at this time.

[0101] Based on this, the priority of the test bench and the test board is determined according to their dependencies:

[0102] When the dependency relationship between the test bench and the test board is that the test board is more dependent on the test board, the test board is considered to have a higher priority.

[0103] When the dependency relationship between the test bench and the test board is that the test bench is more dependent on the test bench, the test bench is considered to have a higher priority.

[0104] For example, in the example program above, it is shown that the dependency relationship between the test server and the test board is that the test board is more dependent on the test board. Based on knowing the dependency relationship, the priority relationship between the test server and the test board is determined by the preset field "CGNumber". At this time, the preset field corresponding to the test board is "CGNumber": 3, and the preset field corresponding to the test server is "CGNumber": 4. It is determined that the execution priority of the test board is higher, that is, the test board is executed first, while the test server is executed later, so as to ensure that the main test case is started only after the dependency is ready.

[0105] The aforementioned preset fields used to determine dependencies and priorities are configured via a JSON template before testing. After being parsed by the backend module, the dependencies are automatically maintained to ensure the orderliness of the testing process.

[0106] The core purpose of determining the dependency relationship between test boards and test benches is to ensure the correctness and automated execution of the testing process, which can be specifically manifested in:

[0107] 1) Priority management of execution order:

[0108] Dependencies can be linked through the CGNumber field (test case execution sequence number) to ensure that dependent items (such as programs or modules defined in FpatsDepend_M1 / FpatsDepend_M2) are executed before the main test case. For example, if test case A depends on program B of the test machine, program B (with a smaller CGNumber) will be executed first, and then test case A (with a larger CGNumber) will be executed after it is completed.

[0109] 2) Resource and environmental dependency management:

[0110] Clearly defining dependencies can automatically trigger pre-processing operations: for example, when dependent on FpatsDepend_M1, the specified module of the test board will be loaded automatically (using the insmod command); when dependent on FpatsDepend_M2, the network parameters of the test machine will be configured automatically or the corresponding service (such as iperf3.exe) will be started to ensure that the test environment meets the execution conditions.

[0111] 3) Correlation in anomaly tracing:

[0112] When a test fails, dependencies can help pinpoint the root cause: if the main test case fails, the execution results of its dependencies (FpatsDepend_M1 / FpatsDepend_M2) will be checked first to determine if the failure is due to the dependencies not running properly (such as module loading failure or the test server service not starting).

[0113] The test unit in this embodiment also includes an anomaly simulation unit. The anomaly simulation items in this unit include at least the following: timeout duration anomaly simulation (Timeout), environment restriction anomaly simulation (Forbid_env), board restart anomaly simulation (Reboot), and module loading failure anomaly simulation.

[0114] A) Simulate Timeout Exception

[0115] The configuration method is as follows: Define the timeout duration (in seconds) in the JSON test script using a preset field, such as the "Timeout" field. This field can be set at the chapter level or the test case level.

[0116] The simulation steps include:

[0117] SA01: During test execution, the backend module parses the "Timeout" value in the JSON script. For example, "Timeout": 30 indicates that a 30-second timeout threshold is set.

[0118] SA02: Monitors the execution time of the current test case in real time. If the execution time exceeds the timeout threshold defined by "Timeout" and no completion signal is received, the current test case is determined to be a "timeout exception" test case and an exception is triggered.

[0119] SA03: After an exception is triggered, the front-end module receives the timeout log pushed by the middleware module in real time via WebSocket (timeout exceptions are usually marked as [Timeout Error]), and records the time of the exception and the test case ID in the test report;

[0120] B) Simulation of environmental constraints and anomalies: Forbid_env

[0121] The configuration method is as follows: use preset fields in the JSON script, such as the "Forbid_env" field, to specify the environment identifier (such as specific hardware model and system version) that is not suitable for executing the test case. For example, the field "Forbid_env":"ECS_C" can be used to indicate that execution is prohibited in the "ECS_C" environment.

[0122] The simulation steps include:

[0123] SB01: During the test configuration phase, the backend module obtains the current test requirement parameters (such as hardware version, system type, etc.) through the test requirement unit.

[0124] SB02: Before executing a test case, compare the actual test environment with the "Forbid_env" field: if they match, trigger an "environment restriction exception", skip the test case and mark it as "not executed (environment restricted)";

[0125] SB03: Anomaly information is synchronized to the front end through the VSOA framework of the middleware module, displayed in the log [ForbidEnvironment:ECS_C], and included in the "Anomaly Statistics" section of the final report.

[0126] C) Simulate Reboot for abnormal board restart

[0127] The configuration method is as follows: set the field "Reboot":true in the dependency of the JSON script (FpatsDepend_M1 or FpatsDepend_M2) to specify whether to trigger the board restart after the test case is executed.

[0128] The simulation steps include:

[0129] SC01: When the test case configuration field "Reboot":true, the backend module will trigger the restart command of the test companion machine (or the board under test) via RPC remote call after the test case is executed;

[0130] SC02: During the restart process, monitor the device status in real time (e.g., whether the IP connection is lost, whether the VSOA heartbeat is interrupted). If the connection cannot be restored after restarting, it is judged as "restart abnormal".

[0131] SC03: Abnormal logs are automatically archived to the logs folder in the installation directory, including data such as restart trigger time and device offline time, supporting subsequent backtracking analysis;

[0132] D) Simulation of module loading failure exception

[0133] The configuration method is as follows: specify an incorrect module path or invalid module file through the FpatsDepend_M1 (depends on the test board) or FpatsDepend_M2 (depends on the test companion machine) field in the JSON script.

[0134] The simulation steps include:

[0135] SD01: Intentionally setting an incorrect kernel module path in the dependency configuration, such as "FpatsParameter":" / lib / modules / invalid_module.ko" (the file does not actually exist);

[0136] SD02: When executing the test case: an attempt was made to load the specified module via insmod (load kernel module) or a similar command, but the loading failed due to an incorrect path;

[0137] SD03: The backend module captures a loading failure signal (such as returning -1 or "insmodfailed" appearing in the log), determines it as a "module loading failure exception", and records the specific error information in the log (such as [ModuleLoadError:invalid_module.konotfound]).

[0138] SD04: Abnormal results are pushed to the front-end module through the middleware module, the test status is updated to "failed" synchronously, and the dependency configuration details are associated in the report.

[0139] The above-mentioned anomaly simulations are all based on template-driven automated execution processes. Anomaly triggering conditions (such as configuration of corresponding fields) are defined through JSON scripts. After the backend module parses the scripts, it automatically detects and triggers anomalies during the test execution phase. Then, the VSOA subscription and publishing mechanism is used to realize the real-time flow and visualization of anomaly information, which is finally incorporated into a structured report to ensure the traceability and reproducibility of anomaly scenarios.

[0140] Furthermore, the automated testing system provided in this embodiment also offers a pluggable architecture and cross-platform compatibility. The pluggable architecture includes support for custom callback functions (such as fields like "Callback") and the ability to extend test result analysis logic via Python scripts. Cross-platform compatibility is reflected in its support for operating platforms such as Windows 10 / 11, Ubuntu 20.04+, and SylixOS, requiring only 4GB of RAM and 200MB of storage.

[0141] [Second Embodiment]

[0142] like Figure 2 As shown, the second embodiment of the present invention also provides a control method for a cross-platform, multi-scenario automated testing system, comprising the following steps:

[0143] S100: Configure the test environment via the front-end module;

[0144] S200: The middleware module sends configuration instructions to the backend module, and the backend module automatically deploys the test environment according to the received configuration instructions;

[0145] S300: The test board executes test items sequentially according to the test script parameters and sends test logs to the front-end module in real time;

[0146] S400: Determine the dependency relationship between the test bench and the test board;

[0147] S500: The backend generates a test report and pushes it to the frontend module through the middleware module;

[0148] S600: Automatically archives test logs to the installation directory.

[0149] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.< / fpatsip> < / fpatsip>

Claims

1. A cross-platform, multi-scenario automated testing system, the system comprising: The front-end module is connected to the middleware module. It provides a graphical interactive interface for configuring the test environment and displaying test results, while also communicating data with the middleware module. The middleware module is connected to both the front-end module and the back-end module. It sends the configuration instructions from the front-end module to the back-end module, subscribes to the test data from the back-end module, and publishes it to the front-end module. The backend module automatically deploys the test environment according to the received configuration instructions, controls the test board to automatically execute test projects, and publishes the test report to the frontend module in real time through the middleware module; Its features are: It also includes a test template module, which comprises a test script unit, a test requirement unit, and a test record unit; wherein, The test requirement unit and the test script unit are connected to the front-end module. The test requirement unit obtains test requirement parameters, and the test script unit defines test script parameters based on the obtained test requirement parameters. The test recording unit is connected to the backend module to record the test process and test results in real time and generate structured test records in real time. The system also includes a test companion device, which is connected to the front-end module. The test companion device simulates network protocol verification and interacts with the test board. The dependency relationship between the test server and the test board is determined by preset fields or preset procedures, and the priority of the test server and the test board is determined based on the dependency relationship: When the dependency relationship between the test bench and the test board is that the test board is more dependent on the test board, the test board is considered to have a higher priority. When the dependency relationship between the test bench and the test board is that the test bench is more dependent on the test bench, the test bench is considered to have a higher priority. The accompanying tester is also equipped with an anomaly simulation unit. The anomaly simulation items in the anomaly simulation unit include at least timeout duration anomaly simulation, environmental limitation anomaly simulation, board restart anomaly simulation, and module loading failure anomaly simulation. The testing machine operates through the following steps: S010: Test configuration phase; S011: The front-end module receives the IP address of the test companion machine, system type, and test type input by the user through a graphical interactive interface; S012: The middleware module forwards configuration commands to the backend module through the VSOA communication framework. The backend module parses the test requirement document and JSON template, and automatically deploys the test server environment to ensure that the test server and the test board can communicate with each other via IP. S020: Test execution phase; S021: Based on the FpatsDepend_M2 field in the JSON template, the backend module triggers the test companion machine to execute the pre-processing task; S022: The test companion machine executes test commands in real time and pushes the execution log to the middleware module through the VSOA subscription and publish mechanism, and then synchronizes it to the front-end module for real-time monitoring; S023: Supports simulation of abnormal scenarios. Through simulations of timeout duration, environmental limitation, board restart, and module loading failure in the JSON template, the fault tolerance capability of the test board can be verified. S030: Result Processing Stage; S031: The test results of the test companion are aggregated to the backend module through the middleware module and used together with the results of the test board to generate a structured report; S032: All logs are automatically archived to the local directory, supporting subsequent backtracking and problem reproduction; When the test results of the auxiliary tester differ from the system test results, the handling procedure is as follows: S041: Full Log Recording: Captures the complete interaction logs between the test device and the test board, and displays them in real time through a graphical interface via the front-end module. The logs also support keyword filtering. S042: Anomaly Correlation Analysis: When a disagreement occurs, the cause of the disagreement can be located by examining the dependency configuration in the JSON template, the execution order of both parties, and the point of exception triggering. S043: Visualized Report Presentation: The test report will list the test results of the test bench and the test board separately, mark the points of divergence, and associate them with the corresponding log fragments to assist manual identification of the problem.

2. The cross-platform, multi-scenario automated testing system according to claim 1, characterized in that: In the front-end module, when configuring the test environment, the configured test environment parameters include at least the IP addresses of the test board and the test companion machine, the test type, and the template file.

3. The cross-platform, multi-scenario automated testing system according to claim 1, characterized in that: The middleware module enables bidirectional communication between the front-end and back-end modules through the VSOA API interface.

4. The cross-platform, multi-scenario automated testing system according to claim 1, characterized in that: The backend module executes the test engine according to the received test instructions to perform tests of at least the following types: BSP test, Kernel test, and custom test. When the backend module executes test projects, it automatically configures the test server environment and the IP addresses of the test board and the test server, and supports multi-platform collaboration. It automatically trims test cases according to the test requirements document and test scripts, executes test projects in the order of preset fields, captures test logs in real time and marks the abnormal test results. It automatically generates test reports in preset formats, and the test reports support one-click export.

5. The cross-platform, multi-scenario automated testing system according to claim 1, characterized in that: The test requirement parameters include at least the software package information, hardware configuration information, software configuration information, test equipment peripheral information, supporting test equipment information, and requirement list information.

6. The cross-platform, multi-scenario automated testing system according to claim 1, characterized in that: The test script parameters include at least test case parameters, dependencies, and keyword substitution relationships.

7. A control method for a cross-platform, multi-scenario automated testing system according to any one of claims 1-6, characterized in that: Includes the following steps: S1: Configure the test environment through the front-end module; S2: The middleware module sends configuration instructions to the backend module, and the backend module automatically deploys the test environment according to the received configuration instructions; S3: The test board executes test items sequentially according to the test script parameters and sends test logs to the front-end module in real time; S4: Determine the dependency relationship between the test bench and the test board; S5: The backend generates a test report and pushes it to the frontend module through the middleware module; S6: Automatically archive test logs to the installation directory.

Images