Automatic integration test method and system for optical module firmware verification

Abstract

The invention relates to the technical field of optical module firmware verification, in particular to an automatic integration test method and system for optical module firmware verification, and the method comprises the following steps: building a test environment of a to-be-tested optical module; the batch processing script is run, a batch processing entry file of the test case is retrieved from a test case library directory, the retrieved batch processing entry file is loaded to an execution queue, and the test case is stored in a test case library in the form of a standardized batch processing entry file; test cases are loaded according to the batch processing entry files in the execution queue, and test configuration files corresponding to the test cases are loaded into global configuration in sequence; and executing the test case, and generating a test log and a test report. According to the method, a framework in which the test case is separated from entry retrieval is adopted, and meanwhile, the test task is automatically executed based on the test configuration file, so that the reusability and compatibility of the test case are improved, and the automation degree and the test efficiency of the test are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of optical module firmware verification, and in particular to an automated integrated testing method and system for optical module firmware verification. Background Art

[0002] With the growing demand for high-speed data transmission, the performance, reliability and stability of optical modules are particularly important. During the design and development process, firmware, as a software layer embedded in the optical module hardware, is responsible for controlling and coordinating the various functions of the optical module. In order to ensure that the optical module can operate stably in a complex working environment, firmware verification testing has become an indispensable step.

[0003] During the optical module firmware verification process, it is usually necessary to write specific test programs or use dedicated test instruments and their supporting software to fully verify the functions and performance of the firmware. These tests cover the stability, compatibility, response time and other aspects of the firmware under different working conditions.

[0004] Traditional testing methods often face problems such as low testing efficiency, insufficient test item coverage and low degree of automation. For example, the Chinese invention patent with publication number CN115733543A provides an optical module firmware timing test platform and test method. The prior art automatically captures and analyzes timing waveforms from the oscilloscope through the collaboration of a host computer, a control board and an oscilloscope, thereby reducing the need for manual intervention and improving the efficiency and consistency of the single test item of timing testing. For another example, the Chinese invention patent with publication number CN116506007A provides an optical module firmware testing system and method. The prior art introduces additional equipment such as optical splitters for multiple tests, distributes optical signals and electrical signals to different test equipment, realizes synchronous signal acquisition during the test process, and combines scripted automated execution on the test board to further improve the accuracy and coverage of the test.

[0005] Although the above-mentioned existing technologies have achieved partial automation, they still have certain limitations, such as the following: (1) Poor flexibility and compatibility. The existing technologies often require customized configuration according to different types of optical modules, and the modification or expansion of the test environment usually requires manual intervention. This method leads to poor flexibility and compatibility of the test environment, and it cannot be efficiently reused between different types of optical modules. Moreover, after each optical module firmware update, the test environment may need to be readjusted, which reduces the scalability and adaptability of the system. (2) Low degree of automation. The existing technologies use partial automation means (such as host computer control of test hardware, script execution test, etc.), but the test process still relies on a lot of manual intervention. For example, testers need to manually modify test scripts to adapt to different optical module types or adjust the test environment according to test requirements; this not only increases the complexity of operations, but also easily introduces human errors, reducing the accuracy and consistency of test results; (3) Strong dependence on hardware equipment and lack of flexibility. The test solutions in the existing technology are highly dependent on hardware equipment (such as oscilloscopes, bit error meters, optical splitters, etc.); when hardware equipment fails or needs to be upgraded, the configuration and operation of the entire test system may be affected, resulting in a large workload and high cost for maintenance and adjustment; in addition, some equipment (such as optical splitters) has certain limitations in compatibility and cannot widely adapt to the testing requirements of different optical modules.

[0006] In general, the existing technologies have problems such as poor flexibility, poor compatibility, low automation and strong hardware dependence in optical module firmware verification testing. These limitations make it difficult for existing testing methods to meet the requirements of efficiency, flexibility and scalability. Summary of the invention

[0007] The purpose of the present invention is to provide an automated integrated testing method and system for optical module firmware verification, which combines test cases with batch scripts, proposes an architecture in which test cases are separated from entry retrieval, and automatically executes test tasks based on test configuration files, thereby improving the reusability and compatibility of test cases, and significantly improving the automation level and test efficiency of the test, so as to solve the technical problems of poor flexibility, poor compatibility, low automation level and strong hardware dependence in the prior art.

[0008] The present invention is implemented by the following technical scheme: an automated integrated test method for optical module firmware verification, applied to an automated integrated test system, the automated integrated test system comprising a hardware platform and a test case library, the method comprising the following steps:

[0009] Build the test environment for the optical module to be tested;

[0010] Run the batch processing script, retrieve the batch processing entry file of the test case from the test case library directory, and load the retrieved batch processing entry file into the execution queue, wherein the test case is stored in the test case library in the form of a standardized batch processing entry file;

[0011] Load test cases according to the batch entry file in the execution queue, and load the test configuration files of the corresponding test cases into the global configuration in sequence;

[0012] Execute test cases and generate test logs and test reports.

[0013] According to a preferred implementation, the specific process of building a test environment for the optical module to be tested includes: establishing a connection between N optical modules to be tested and an optical module communication device, where N is a positive integer greater than or equal to 1, establishing a communication link between the optical module communication device and a hardware platform, and building a test hardware environment.

[0014] According to a preferred implementation, the specific process of building a test environment for the optical module to be tested also includes: loading a test framework program package onto a hardware platform and building a test software environment, wherein the test framework program package includes a test framework, a resource directory, and a test case directory, wherein the resource directory is used to store test resources, and the test resources include the MCU firmware, DSP firmware, code files, and test configuration files of the optical module to be tested, and the test case directory is used to store the directory of each test item, and the directory of each test item stores a corresponding test program and batch execution entry.

[0015] According to a preferred embodiment, each test case in the test case library has a unified batch execution entry.

[0016] According to a preferred implementation, the specific process of building the test software environment further includes: acquiring the test content of the optical module to be tested, and configuring a test configuration file based on the test content.

[0017] According to a preferred implementation, the specific process of building the test software environment also includes: obtaining updated content of the test framework and test cases from a Git server, and updating the test framework and test case library based on the updated content.

[0018] According to a preferred implementation, the specific process of executing the test cases includes: parsing the test configuration files in the global configuration, determining the enabled test cases, loading the enabled test cases into the test case queue, and executing the test cases in sequence according to the order in which the test cases in the test case queue are added.

[0019] According to a preferred implementation, the specific process of generating the test log and the test report includes:

[0020] Capture the data flow of all optical modules to be tested and test cases in real time during operation, and generate test logs;

[0021] Obtain the single test report of the test case generated during the test process, summarize all the single test reports, and generate a complete test report.

[0022] The present invention also provides an automated integrated test system for optical module firmware verification, which uses the automated integrated test method described above, and the system includes a test case library, a hardware platform, and a test framework running on the test platform;

[0023] The test framework includes:

[0024] A test case retrieval module is configured to run a batch processing script, retrieve a batch processing entry file of a test case from a test case library directory, and load the retrieved batch processing entry file into an execution queue;

[0025] A test case management module is configured to load test cases according to the batch entry file in the execution queue, and sequentially load the test configuration files of the corresponding test cases into the global configuration;

[0026] A test case execution module, configured to execute the test case;

[0027] A report integration analysis module is configured to generate a test report;

[0028] A log management module, configured to generate a test log;

[0029] An automated execution engine that is configured to manage the execution of test cases and test tasks.

[0030] According to a preferred embodiment, the test framework further includes:

[0031] The test environment initialization module is configured to interact with the Git server and update the test environment;

[0032] The test parameter configuration module is configured to obtain the test content of the optical module to be tested and configure a test configuration file based on the test content.

[0033] The technical solution of an automated integrated test method and system for optical module firmware verification provided by the present invention has at least the following advantages and beneficial effects:

[0034] (1) The present invention closely combines test cases with batch processing scripts and proposes an architecture in which test cases are separated from entry retrieval. Test cases only need to write standardized batch processing entry files without paying attention to specific retrieval methods, thereby supporting unified retrieval and execution of test cases in different programming languages ​​and test frameworks. The architecture can support flexible addition, deletion and modification of test cases, significantly improving the reusability and compatibility of test cases. In addition, the integration, flexibility and scalability of the test system are also improved, and it can better adapt to different types of optical modules and their diverse testing requirements;

[0035] (2) Each test case in the test case library of the present invention has a unified batch execution entry, which supports the independent execution of a test case to generate an independent test report, and can also be executed as part of the overall test process to generate a complete test report, thereby further enhancing the reusability, flexibility and maintainability of the test system;

[0036] (3) By combining the Git server with batch scripts, it can support automatically pulling the latest version of the test framework and test cases from the Git server, automatically executing test tasks based on the test configuration files, and generating standard test reports. This mechanism reduces manual intervention and operational errors, significantly improves the automation level and test efficiency of the test system, and improves the accuracy and reliability of the test process.

[0037] (4) By redirecting all output streams generated during the running of the test system and test cases to the log management module, the running process information can be fully recorded and displayed. Even when the test case exits abnormally, the exit code and error information can be accurately recorded and will not affect the execution of subsequent test cases. This log processing method effectively improves the traceability of the test process and the stability of the test system. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic flow chart of an automated integrated testing method for optical module firmware verification provided in Embodiment 1 of the present invention;

[0039] Figure 2 A schematic diagram of the structure of an automated integrated testing system provided in Example 1 of the present invention;

[0040] Figure 3 A schematic diagram of the execution flow of the test case execution module provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0042] Example 1

[0043] The embodiment of the present invention provides an automated integrated test method for optical module firmware verification, and the automated integrated test method is applied to an automated integrated test system, see Figure 1 As shown, the automated integrated test method for optical module firmware verification includes the following steps:

[0044] Building a test environment for the optical module to be tested; in one implementation of the present embodiment, building a test environment for the optical module to be tested is divided into building a test hardware environment and a test software environment.

[0045] The specific process of building the test hardware environment includes:

[0046] Prepare optical module communication equipment, which may be a hardware device that can communicate with the optical module, such as a test board, an evaluation board, a bit error tester, a switch, and a network card; further, establish a connection between N optical modules to be tested and the optical module communication equipment, where N is a positive integer greater than or equal to 1.

[0047] Prepare a hardware platform. The hardware platform can be a computer, server, mobile phone, embedded device or other hardware platform that can run the test framework. Furthermore, establish a communication link between the optical module communication device and the hardware platform. The optical module to be tested establishes a communication link with the hardware platform through the optical module communication device to complete the construction of the test hardware environment. The optical module communication device and the hardware platform can establish a communication link through USB, serial port, Ethernet or other methods. The connection method is not limited to a specific interface, so that testers can flexibly adjust according to actual needs to meet the requirements of different test environments.

[0048] During operation, the optical module communication equipment and the hardware platform serve as the middle layer hardware to ensure that the test framework can communicate with the optical module to be tested. It should be noted that if a single optical module to be tested is tested, the optical port of the optical module to be tested can be connected using the self-loop mode, and if multiple optical modules to be tested need to be tested, the optical ports of multiple optical modules to be tested can be connected in pairs through optical fibers in a pair transmission mode or in a self-loop mode using self-loop optical fibers.

[0049] The specific process of building a test software environment includes:

[0050] The test framework program package is loaded onto the hardware platform. In one implementation of the present embodiment, the test framework program package includes a test framework, a resource directory, and a test case directory; wherein the resource directory is used to store test resources such as the MCU firmware, DSP firmware, code files, and test configuration files of the light metering module; the test case directory is used to store the directory of each test item, and the directory of each test item stores the corresponding test program and batch execution entry.

[0051] Adjustment of the test configuration file. In one implementation of this embodiment, the test content of the optical module to be tested is obtained, and the test configuration file is configured based on the test content. This embodiment can modify the test configuration file according to different test contents, set all the configuration information of the test such as: tester information, project number, optical module protocol version, firmware path and firmware author, etc., or do not adjust the test configuration file and directly use the default test configuration file.

[0052] The test framework and the test case library are detected and updated to complete the construction of the test software environment; in one implementation of the present embodiment, the test framework and the test case library are uniformly managed by the Git server; before loading the test case and executing the test, the test environment initialization module obtains the updated content of the test framework and the test case from the Git server, and detects whether the local needs to be updated. If there is an update, the test framework and the test case library are updated based on the updated content, and the test system is restarted to ensure the synchronization and timeliness of the test environment. The present invention combines the Git server with the batch script to support the automatic pulling of the latest version of the test framework and test case from the Git server, and automatically executes the test task based on the test configuration file to generate a standard test report. This mechanism reduces manual intervention and operational errors, significantly improves the automation level and test efficiency of the test system, and improves the accuracy and reliability of the test process.

[0053] In one implementation of this embodiment, the test process is as follows:

[0054] Run the batch script, retrieve the batch entry file of the test case from the test case library directory, and load the retrieved batch entry file into the execution queue; the test case is stored in the test case library in the form of a standardized batch entry file, and each test case in the test case library has a unified batch execution entry, so that different test cases only need to write standardized batch entry files, without paying attention to the way they are retrieved. The batch entry file can support test cases written in different programming languages ​​and architectures, so that they can all be retrieved, loaded and executed; and the architecture can support flexible addition, deletion and modification of test cases, significantly improving the reusability and compatibility of test cases. In addition, it also improves the integration, flexibility and scalability of the test system, and can better adapt to different types of optical modules and their diverse testing needs.

[0055] It should be noted that each test case in the test case library of the present invention has a unified batch execution entry, which not only supports the execution of a certain test case individually and the generation of an independent test report, but also can be executed as part of the overall test process to generate a complete test report, thereby further enhancing the reusability, flexibility and maintainability of the test system.

[0056] Specifically in the retrieval process of this embodiment, the batch script Run.cmd is run on the computer, and the path of the current overall test script is first obtained. Further, by traversing the folders named with the prefix "Script_" under the Scripts file in the test case library of the same level, the batch entry files with the suffixes of ".cmd", ".bat" and ".sh" in the folder are sequentially retrieved after recording the retrieval path; further, the retrieved batch entry files are added to the execution queue, and the test cases are automatically obtained from the execution queue after the retrieval is completed.

[0057] Execute test cases, load test cases according to the batch entry file in the execution queue, and load the test configuration files of the corresponding test cases into the global configuration in turn, such as whether to enable the test, test mode and test parameters, etc.; specifically in this embodiment, when the operation exceeds the set time, for example, after 10 seconds, the default test configuration file is used.

[0058] Further, the test cases are executed, and after obtaining the global configuration, the test cases are tested in sequence; specifically in this embodiment, the specific process of executing the test cases includes: parsing the test configuration files in the global configuration, determining the enabled test cases, loading the enabled test cases into the test case queue, and executing the test cases in sequence according to the order in which the test cases in the test case queue are added.

[0059] The test report integrated output and the test log output, specifically in this embodiment, the specific process of generating the test log and the test report includes: real-time capture of the data streams during the operation of all the optical modules to be tested and the test cases, and redirecting the logs, error information and exception information during the operation of the optical modules to be tested and the test cases through the standard output stream and the standard error stream output to the log management module to generate a test log; through the above method, the complete recording and display of the operation process information can be achieved, even when the test case exits abnormally, the exit code and error information can be accurately recorded, and will not affect the execution of subsequent test cases. This log processing method effectively improves the traceability of the test process and the stability of the test system.

[0060] Furthermore, a single test report of the test case generated during the test process is obtained, all single test reports are summarized, a complete test report is automatically generated, and the test report is integrated and analyzed, so that the test case does not need to process additional event logic, which significantly improves its maintainability and report generation efficiency. It should be noted that in this embodiment, the test report and the test log are generated in a unified format, ensuring the standardization, repeatability and high efficiency of the test process, and ensuring the accuracy and consistency of the test results.

[0061] The embodiment of the present invention also provides an automated integrated test system for optical module firmware verification, the automated integrated test system includes a test case library, a hardware platform and a test framework running on the test platform; the test framework includes a test environment initialization module, a test parameter configuration module, a test case retrieval module, a test case management module, a test case execution module, a report integration analysis module, a log management module and an automated execution engine, the system structure and relationship are as follows Figure 2 shown.

[0062] The test framework is the core of the test system, and the test environment initialization module is configured to interact with the Git server and update the test environment. When started, it automatically detects the version information of the test framework and test cases from the Git server, and automatically updates all programs, prepares test report technical information, and loads firmware from the Git server. The test parameter configuration module is configured to obtain the test content of the optical module to be tested and configure the test configuration file based on the test content.

[0063] It should be noted that by combining the Git server with the batch script, it is possible to automatically pull the latest version of the test framework and test cases from the Git server, automatically execute test tasks based on the test configuration files, and generate standard test reports. This mechanism reduces manual intervention and operational errors, significantly improves the automation level and test efficiency of the test system, and at the same time improves the accuracy and reliability of the test process.

[0064] The automated execution engine is configured to manage the execution of test cases, including retrieving batch execution entries in the directory, adding batch execution entries to the execution queue, obtaining test cases from the queue and starting the test, recording test data, etc. The automated execution engine can load test cases and perform verification processes based on the batch entry files in the execution queue.

[0065] The test case retrieval module is configured to run the batch script, retrieve the batch entry file of the test case from the test case library directory, and load the retrieved batch entry file into the execution queue; the test case management module is configured to load the test case according to the batch entry file in the execution queue, and load the test configuration file of the corresponding test case into the global configuration in turn; the test case execution module is configured to execute the test on the test case in turn after obtaining the global configuration. For the specific execution process, see Figure 3 As shown; the report integration analysis module and the log management module are configured to generate test logs and test reports.

[0066] It should be noted that the automated integrated test system provided above can support the verification of a single test item. After establishing a communication link between the optical module to be tested and the hardware platform, a single test item is selected. The test item can be a separate test content such as a functional verification, performance test or error check; further, the test configuration file is configured, such as the test time, communication rate, and test range; further, the test of the test item is executed by running the batch script of this test item. The batch script prepares the test environment and executes the test of the specific function; further, an independent test report and an independent test log are generated for the single test item. The test report will contain basic information of the test item, test results and other information, and the test report is only displayed for the test item and does not involve the content of other test items.

[0067] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An automated integrated test method for optical module firmware verification, characterized in that: Applied to an automated integrated test system, the automated integrated test system includes a hardware platform and a test case library, and the method includes the following steps: Build the test environment for the optical module to be tested; Run the batch processing script, retrieve the batch processing entry file of the test case from the test case library directory, and load the retrieved batch processing entry file into the execution queue, wherein the test case is stored in the test case library in the form of a standardized batch processing entry file; Load test cases according to the batch entry file in the execution queue, and load the test configuration files of the corresponding test cases into the global configuration in sequence; Execute test cases and generate test logs and test reports.

2. The automated integrated testing method for optical module firmware verification according to claim 1, characterized in that: The specific process of building a test environment for the optical module to be tested includes: establishing a connection between N optical modules to be tested and an optical module communication device, where N is a positive integer greater than or equal to 1, establishing a communication link between the optical module communication device and a hardware platform, and building a test hardware environment.

3. The automated integrated testing method for optical module firmware verification according to claim 2, characterized in that: The specific process of building a test environment for the optical module to be tested also includes: loading a test framework program package onto a hardware platform and building a test software environment, wherein the test framework program package includes a test framework, a resource directory, and a test case directory, wherein the resource directory is used to store test resources, and the test resources include MCU firmware, DSP firmware, code files, and test configuration files of the optical module to be tested, and the test case directory is used to store directories of various test items, and the directory of each test item stores a corresponding test program and a batch execution entry.

4. The automated integrated testing method for optical module firmware verification according to claim 3, characterized in that: Each test case in the test case library has a unified batch execution entry.

5. The automated integrated testing method for optical module firmware verification as claimed in claim 3, characterized in that: The specific process of building the test software environment also includes: obtaining the test content of the optical module to be tested, and configuring a test configuration file based on the test content.

6. The automated integrated testing method for optical module firmware verification as claimed in claim 3, characterized in that: The specific process of building the test software environment also includes: obtaining updated content of the test framework and test cases from the Git server, and updating the test framework and test case library based on the updated content.

7. The automated integrated testing method for optical module firmware verification according to any one of claims 1 to 6, characterized in that: The specific process of executing the test case includes: parsing the test configuration file in the global configuration, determining the enabled test cases, loading the enabled test cases into the test case queue, and executing the test cases in sequence according to the order in which the test cases in the test case queue are added.

8. The automated integrated testing method for optical module firmware verification according to claim 7, characterized in that: The specific process of generating the test log and test report includes: Capture the data flow of all optical modules to be tested and test cases in real time during operation, and generate test logs; Obtain the single test report of the test case generated during the test process, summarize all the single test reports, and generate a complete test report.

9. An automated integrated test system for optical module firmware verification, characterized in that: Using the automated integrated testing method as described in any one of claims 1 to 8, the system includes a test case library, a hardware platform, and a test framework running on the test platform; The test framework includes: A test case retrieval module is configured to run a batch processing script, retrieve a batch processing entry file of a test case from a test case library directory, and load the retrieved batch processing entry file into an execution queue; A test case management module is configured to load test cases according to the batch entry file in the execution queue, and sequentially load the test configuration files of the corresponding test cases into the global configuration; A test case execution module, configured to execute the test case; A report integration analysis module is configured to generate a test report; A log management module, configured to generate a test log; An automation execution engine is configured to manage the execution of test cases.

10. The automated integrated test system for optical module firmware verification according to claim 9, characterized in that: The test framework also includes: The test environment initialization module is configured to interact with the Git server and update the test environment; The test parameter configuration module is configured to obtain the test content of the optical module to be tested and configure a test configuration file based on the test content.

Images