Automatic test platform based on OpenOCD
Through the OpenOCD-based automatic testing platform, modular management and automated execution are adopted to solve the complexity and inefficiency problems in embedded system hardware debugging and testing, realize efficient and standardized test result recording and automatic verification of XIP mode, and improve system stability and development efficiency.
Patent Information
- Application Number
- CN202510793975.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-13
- Publication Date
- 2025-09-19
AI Technical Summary
Existing embedded system hardware debugging and testing technologies have problems such as complex operation, low efficiency, inconsistent results, low test coverage, and lack of automated tools. In particular, it is difficult to achieve scalability and automated management in the OpenOCD environment.
An automatic testing platform based on OpenOCD was designed, including a test script library, a control module, an OpenOCD debugger, a logging module, and a version update module. Through modular management of test scripts, automatic execution and result recording were achieved. Test scripts were written in TCL language and combined with the Python control framework to support batch execution of multiple test cases and standardized output of results.
It improves the efficiency and quality of testing, enhances the reusability and platform compatibility of test cases, realizes the automation of the testing process and the standardization of results, simplifies the verification process of XIP mode, and improves the robustness and development efficiency of the system.
Smart Images
Figure CN120670235A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of embedded hardware debugging and testing, and in particular to an automatic testing platform based on OpenOCD. Background Art
[0002] Hardware debugging is a critical step in the development of embedded systems. This typically involves using a hardware debugger, such as OpenOCD (Open On-Chip Debugger), to directly access and control the target system's processor through a specific interface, such as JTAG or SWD. A hardware debugger can be used to perform various debugging tasks, such as setting breakpoints, single-stepping through code, and inspecting and modifying memory and register values. Furthermore, a hardware debugger can also be used to program flash memory devices on the target system.
[0003] Existing embedded system hardware debugging and testing technologies have some shortcomings:
[0004] Complexity of manual operations: In traditional hardware debugging and testing, developers need to manually set up and manage debugging sessions, which may include complex hardware interface configuration, breakpoint setting, memory and register inspection, etc. This process often requires in-depth expertise, is complex, and prone to errors.
[0005] Low testing efficiency: Manual testing is often time-consuming, especially when the same test needs to be performed repeatedly. Each test requires manual setup and execution, which significantly reduces testing efficiency.
[0006] Inconsistent test results: Because manual testing procedures can vary from person to person, the same test case may produce different results in different executions. This inconsistency can lead to incorrect diagnosis and difficulty in resolving problems.
[0007] Low test coverage: Manual testing usually only covers a limited number of test cases, making it difficult to achieve comprehensive and systematic testing. This may result in potential problems being missed.
[0008] Lack of automated tools: Although some hardware debuggers (such as OpenOCD) provide scripting capabilities, these features usually require writing specific scripting languages, which requires high developer skills and is inconvenient to use. In addition, these tools often lack comprehensive automated testing and reporting capabilities.
[0009] Therefore, existing embedded system hardware debugging and testing technologies mostly rely on manual debugging or semi-automated script execution, which has some significant shortcomings. Better solutions are needed to improve test efficiency, quality, and reliability. In the OpenOCD environment in particular, traditional testing methods are primarily based on command-line interaction, making them difficult to scale and automate. Furthermore, test cases are often coupled to the test framework, making them difficult to maintain and port. Therefore, automated testing has important application value in this field.
[0010] This section is intended to provide a background or context for understanding the embodiments of the present application and is for reference only. The applicant should not be considered to admit that this section belongs to the prior art that has been disclosed before the filing date of this application. Summary of the Invention
[0011] The purpose of this application is to provide an automatic testing platform based on OpenOCD, which can realize modular management of test scripts, automatic control of the test process, and unified recording and analysis of results, thereby reducing manual errors, improving test efficiency, and enhancing the reusability and platform compatibility of test cases.
[0012] The present application discloses an automatic testing platform based on OpenOCD, comprising: a test script library, a control module, an OpenOCD debugger, a logging module, and a version update module, wherein the test script library, the control module, the OpenOCD debugger, and the logging module are connected in sequence;
[0013] The test script library is used to store and manage all test scripts, wherein each test script corresponds to a specific test case;
[0014] The control module is used to control the OpenOCD debugger to automatically execute all test scripts in the test script library;
[0015] The logging module is used to record and store the test results of the OpenOCD debugger executing the test case;
[0016] The version update module is used to check and download an updated version of the OpenOCD debugger and an updated version of the test script.
[0017] In a preferred embodiment, the control module further comprises: a test management module, a process management module and a Telnet client; the input of the test management module is connected to the test script library, the output is connected to the process management module and the Telnet client, and the outputs of the process management module and the Telnet client are connected to the OpenOCD debugger;
[0018] The test management module is used to obtain test cases from the test script library, create a test thread for each test case and manage the test thread;
[0019] The process management module is used to create and manage the process of the OpenOCD debugger according to the test thread;
[0020] The Telnet client is used to create and manage Telnet links and communicate with the OpenOCD debugger through the Telnet links. The test thread sends commands of the test script to the OpenOCD debugger through the Telnet links and receives test results returned by the OpenOCD debugger.
[0021] In a preferred example, the OpenOCD debugger includes a Telnet server, and the Telnet client communicates with the OpenOCD debugger through the Telnet server.
[0022] In a preferred example, the test script includes commands required to run the OpenOCD debugger.
[0023] In a preferred embodiment, the test results include the execution results of each test case and related hardware status information.
[0024] In a preferred example, the OpenOCD debugger is connected to the debugger via a USB port, and the debugger is connected to the debugging target device via a JTAG protocol.
[0025] In a preferred example, the test script is written in TCL language.
[0026] In a preferred embodiment, the test script library includes an XIP mode verification script, and the XIP mode verification script includes the following test process;
[0027] Write verification data to the flash memory of the QSPI interface in normal working mode;
[0028] If the writing is successful, loading the XIP mode check program into the random access memory, the XIP mode check program including:
[0029] Configuring the flash memory of the QSPI interface to XIP mode;
[0030] Read the flash memory of the QSPI interface in the XIP mode;
[0031] Calculate the checksum;
[0032] Determine whether the verification code is correct, and if so, return the verification result.
[0033] In a preferred embodiment, if writing verification data to the flash memory of the QSPI interface in the normal working mode fails, an error is reported during writing.
[0034] In a preferred example, if the check code is incorrect, the configuration of the XIP mode is incorrect and an error is reported.
[0035] In the implementation of this application, this method controls OpenOCD to automatically perform a series of hardware debugging and testing tasks, and records and reports the test results. This significantly improves test efficiency and quality, while also standardizing the test process and standards through test flow management. Furthermore, by managing test cases through a test script library, test cases can be flexibly combined according to actual test objectives to form different test flows.
[0036] In addition, the present invention performs XIP configuration inspection through the OpenOCD automated testing platform, provides an efficient and automated inspection solution, and improves the robustness of the system.
[0037] The various technical features disclosed in the above invention content, the various technical features disclosed in the various embodiments and examples below, and the various technical features disclosed in the accompanying drawings can be freely combined with each other to form various new technical solutions (all of which should be deemed to have been recorded in this specification), unless such a combination of technical features is technically infeasible. For example, in one example, feature A+B+C is disclosed, and in another example, feature A+B+D+E is disclosed. Features C and D are equivalent technical means that play the same role. Technically, only one of them needs to be used, and it is impossible to use them at the same time. Feature E can be technically combined with feature C. In this case, the solution of A+B+C+D should not be deemed to have been recorded because it is technically infeasible, while the solution of A+B+C+E should be deemed to have been recorded. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 It is a structural diagram of an OpenOCD-based automatic testing platform according to the first embodiment of the present application.
[0039] Figure 2 It is a flowchart of the XIP mode verification method based on OpenOCD according to the second embodiment of the present application. DETAILED DESCRIPTION
[0040] In the following description, many technical details are provided to help readers better understand this application. However, those skilled in the art will understand that even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in this application can be implemented.
[0041] Description of some concepts:
[0042] OpenOCD (Open On-Chip Debugger): An open source hardware debugging software mainly used for embedded system development.
[0043] QSPIFlash: A type of Flash memory that uses the Quad SPI (QSPI) interface. Flash memory is a non-volatile memory, which means it can retain stored data after power is removed.
[0044] XIP (Execute In Place): A common configuration in embedded systems. In an XIP configuration, the system executes code directly from non-volatile memory (such as flash memory) without first copying the code to RAM. This reduces RAM usage and allows programs to execute immediately without waiting for the code to be loaded into RAM.
[0045] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.
[0046] The first embodiment of the present application relates to an automatic test platform 100 based on OpenOCD, the block diagram of which is shown in FIG. Figure 1 As shown, the automatic testing platform 100 includes a test script library 101, a control module 102, an OpenOCD debugger (or simply OpenOCD) 105, a logging module 103, and a version update module 104. The test script library 101, the control module 102, the OpenOCD debugger 105, and the logging module 103 are connected in sequence.
[0047] The test script library 101 is used to store and manage all test scripts, where each test script corresponds to a specific test case. In one embodiment, the test scripts are written in TCL. The control module 102 is used to control the OpenOCD debugger 105 to automatically execute all test scripts in the test script library 101. The logging module 103 is used to record and store the test results of the test cases executed by the OpenOCD debugger 105. The version update module 104 is used to check for and download updated versions of the OpenOCD debugger 105 and test scripts to ensure that the test system is always up to date.
[0048] In one embodiment, the OpenOCD debugger 105 is connected to a debugger via a USB port, and the debugger is connected to a debug target device (DUT) via a JTAG protocol.
[0049] Continue to refer Figure 1As shown, the control module 102 further includes a test management module 106, a process management module 107, and a Telnet client 108. The input of the test management module 106 is connected to the test script library 101, and its output is connected to the process management module 107 and the Telnet client 108. The outputs of the process management module 107 and the Telnet client 108 are connected to the OpenOCD debugger 105. The test management module 106 is used to retrieve test cases from the test script library 101, create a test thread for each test case, and manage the test threads. The process management module 107 is used to create and manage OpenOCD debugger 105 processes based on the test threads. The Telnet client 108 is used to create and manage Telnet connections and communicate with the OpenOCD debugger 105 via the Telnet connection. The test threads send test script commands to the OpenOCD debugger 105 via the Telnet connection and receive test results returned by the OpenOCD debugger 105.
[0050] In one embodiment, the OpenOCD debugger 105 includes a Telnet server, and the Telnet client communicates with the OpenOCD debugger 105 through the Telnet server.
[0051] The workflow of the OpenOCD-based test platform is as follows:
[0052] Step 1, Test Script Preparation: First, the test engineer writes or updates the test scripts, each script corresponding to a test case. These scripts will contain all the OpenOCD commands and operations required to perform a specific test.
[0053] Step 2, version update: Before the test begins, the version update module 104 checks the latest version of the test script and OpenOCD. If an update is available, it automatically downloads and installs the latest version.
[0054] Step 3: Start OpenOCD: When the test starts, the process management module 107 starts the OpenOCD process, which is responsible for communicating with the hardware device.
[0055] Step 4, Execute Test: The test management module 106 creates a new test thread for each test script. This thread uses the Telnet client to send the commands in the script to the OpenOCD process. The execution results of each command are returned to the test thread via the Telnet connection.
[0056] Step 5, record the results: The logging module 103 will record the results of each test case and the relevant hardware status information. This information will be stored in a log file for subsequent analysis and review.
[0057] Step 6, end the test: After all test cases are executed, the process management module 107 stops the OpenOCD process.
[0058] Step 7: Analyze the test results based on the log files.
[0059] Through this workflow, the test platform can automatically execute a large number of test cases, record detailed results, and keep the test environment up to date, thereby improving test efficiency and quality. The OpenOCD-based automatic test platform of the present invention is superior to existing technologies in terms of test architecture design, execution mode, result collection and management, case maintenance, and platform compatibility. This is specifically reflected in the following aspects:
[0060] 1. Cross-language decoupling of test cases and test frameworks improves maintenance and reuse efficiency.
[0061] A common problem with traditional embedded testing methods is the deep coupling of test logic and the test framework language, making test cases difficult to port and reuse. This paper uses TCL to write test scripts, closely adhering to the native OpenOCD command format and ensuring good maintainability. The control framework, developed in Python, focuses on case scheduling, OpenOCD process control, and result parsing. Through the TCL+Python architecture, the test logic is completely decoupled from the execution framework, making test cases easier to port and replace while significantly reducing the maintenance costs of the test framework.
[0062] 2. OpenOCD's service-oriented automatic call supports automatic execution of batch use cases.
[0063] Existing OpenOCD invocation methods primarily rely on manual command line input or scripted single tests, resulting in low automation and inefficiency. This invention dynamically controls the startup and execution of OpenOCD through Python scripts, automatically importing and executing TCL test scripts, and fully capturing standard output and error information during the test process. The platform supports features such as batch traversal of multiple test cases, log comparison, and breakpoint resumption, significantly improving test efficiency and stability. This mechanism seamlessly integrates with continuous integration (CI) systems, enabling truly unattended automated testing.
[0064] 3. Test results are output in a standardized format to facilitate programmatic parsing and statistical analysis.
[0065] OpenOCD's raw output is mostly debug logs, which are unstructured and difficult to use for automatic judgment. This paper introduces a structured output format (e.g., puts "TEST_RESULT: PASS" and puts "TEST_RESULT: FAIL") into the test script, giving the test results a "result label" feature. The Python side uses regular expressions to automatically extract results and assert, enabling automatic recording and judgment of use case results. This improves the controllability, stability, and automation of the test process, and also provides a data foundation for subsequent statistics, analysis, and report generation.
[0066] 4. Centralized management of test cases, supporting fast registration and flexible expansion.
[0067] In traditional systems, test scripts are often stored in fragmented locations, making management difficult. This invention utilizes a unified .tcl file management approach, allowing the test framework to automatically register test cases based on naming conventions or path structures. New test cases can be added without modifying the framework code, and existing test cases can be directly reused. This enables modular management and rapid iteration of test cases, significantly reducing maintenance costs and enhancing system scalability and long-term sustainability.
[0068] 5. Powerful platform compatibility and hardware abstraction capabilities, adaptable to a variety of test targets.
[0069] The present invention is compatible with OpenOCD's interface.cfg and target.cfg configuration mechanisms in terms of architecture design, and can support a variety of debuggers and target chips. In terms of operating systems, the Python control framework and TCL scripts both have good cross-platform capabilities, supporting mainstream operating system environments such as Linux, Windows, and macOS, and can be migrated and executed without modifying the test logic. Compared to solutions that rely on dedicated drivers (such as pyOCD and JLink SDK), the present invention reuses OpenOCD's general hardware abstraction layer, significantly enhancing the platform's adaptability and long-term technical flexibility.
[0070] This paper builds an automated testing platform decoupled from the Python framework using TCL scripts, empowering OpenOCD with highly automated execution capabilities. This solves technical challenges inherent in traditional embedded testing, such as low efficiency, poor scalability, non-standard test outputs, and difficulty reusing test cases. This platform is suitable for testing various embedded chips and development boards, and can be widely applied to engineering processes such as R&D testing, regression verification, and continuous integration.
[0071] On the other hand, the use of QSPI Flash memory is crucial in the development of embedded systems. QSPI Flash is a non-volatile memory that uses a four-wire SPI interface and can retain data after power failure. It is widely used to store firmware or other critical data in embedded systems. The high-speed data transmission capability of QSPI Flash is a key reason for its popularity, especially when large amounts of data need to be read quickly from Flash, such as in embedded systems using the XIP configuration. In the XIP configuration, the system can execute code directly from the Flash memory without copying the code to RAM. This configuration improves system efficiency, but also brings some new challenges.
[0072] After programming the QSPI Flash, a data check is typically performed in normal operating mode to ensure that the data was written correctly. However, if the Flash is set to XIP mode, the data reading method changes (the read command is different). If the XIP mode is set incorrectly, the system may read incorrect data, causing program execution errors. Therefore, to ensure that the XIP mode is set correctly, it is necessary to perform a data check again after setting the XIP mode. This process is crucial in embedded system development, as it involves data correctness and system stability, and requires careful attention during the testing and verification process.
[0073] The existing QSPIFlash programming and verification technology has the following disadvantages:
[0074] 1. Data verification is usually performed only in normal operation mode, while data verification in XIP mode is ignored. If the XIP mode is set incorrectly, the system may read incorrect data, resulting in program errors.
[0075] 2. Due to the complex setup and verification process of XIP mode, additional time and resources may be required. This may increase development cycles and costs, especially when frequent firmware modifications and updates are required.
[0076] To this end, the second embodiment of the present application relates to an OpenOCD-based XIP mode verification method. The XIP mode verification method uses the above-mentioned OpenOCD-based automated testing platform to perform automatic verification of the XIP mode by pre-writing an XIP mode verification script and adding it to the test script library 101. The XIP mode verification process of the present invention is as follows: Figure 2 The detailed description is as follows:
[0077] 1. Normal mode programming and verification of QSPI Flash: In this step, the system first writes data to QSPI Flash in normal mode. After writing is complete, the system reads and verifies the written data to ensure that the data is written correctly. If the verification data fails to be written to QSPI Flash in normal working mode, a write error is reported.
[0078] 2. Load the XIP configuration check program (i.e., the XIP mode check script) into RAM. The XIP configuration check program can be divided into the following steps:
[0079] (1) Configuring XIP mode
[0080] (2) Read QSPIFlash in XIP mode
[0081] (3) Calculate the check code
[0082] (4) Return the result. If the checksum is incorrect, the XIP mode configuration is incorrect and an error is reported.
[0083] The above is the XIP configuration check process. This process can ensure that the QSPIFlash is correctly set up in XIP mode, thereby ensuring stable system operation.
[0084] The above process is written as a TCL script and added to the OpenOCD test script library. Through the OpenOCD automatic test platform, the XIP configuration check can be automated. The solution includes the following steps:
[0085] 1. Add the XIP configuration test script to the script library
[0086] 2. Update the OpenOCD version using the version update module 104
[0087] 3. Use the OpenOCD control module 102 to start the OpenOCD process and use the Telnet client 108 to execute the XIP configuration check script.
[0088] 4. Use the logging module 103 to record the results of the XIP configuration check and related hardware status information.
[0089] 5. Analyze and review the results in the log files.
[0090] This invention provides data verification technology for both normal operating mode and XIP mode to ensure data accuracy and system stability. Automated tools simplify the XIP mode setup and verification process, reducing development cycles and costs. Detecting and correcting errors in XIP mode enhances system robustness and reliability. By achieving these goals, this invention will help improve embedded system development efficiency and operational stability, thereby enhancing user satisfaction and product market competitiveness.
[0091] Accordingly, the embodiments of the present application also provide a computer-readable storage medium having computer-executable instructions stored therein, which implement the various method embodiments of the present application when executed by a processor. Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include transient computer-readable media (transitory media), such as modulated data signals and carrier waves.
[0092] In addition, an embodiment of the present application further provides a computer program product, which includes computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0093] It should be noted that, in this application, relational terms such as first and second are merely used to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article, or device. Without further restriction, an element defined by the phrase "comprising a" does not exclude the presence of other identical elements in the process, method, article, or device comprising the element. In this application, if it is mentioned that an action is performed according to a certain element, it means that the action is performed at least according to that element, including two situations: performing the action only according to that element, and performing the action according to that element and other elements. Expressions such as "multiple," "multiple," and "multiple" include 2, 2 times, 2 kinds, and more than 2, more than 2 times, and more than 2 kinds.
[0094] All documents mentioned in this specification are considered to be included in their entirety in the disclosure of this application so that they can be used as a basis for modification when necessary. In addition, it should be understood that the above description is only a preferred embodiment of this specification and is not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification should be included in the scope of protection of one or more embodiments of this specification.
[0095] In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. Additionally, the processes depicted in the accompanying drawings do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.
Claims
1. An automatic testing platform based on OpenOCD, characterized in that: include: A test script library, a control module, an OpenOCD debugger, a logging module, and a version update module, wherein the test script library, the control module, the OpenOCD debugger, and the logging module are connected in sequence; The test script library is used to store and manage all test scripts, wherein each test script corresponds to a specific test case; The control module is used to control the OpenOCD debugger to automatically execute all test scripts in the test script library; The logging module is used to record and store the test results of the OpenOCD debugger executing the test case; The version update module is used to check and download an updated version of the OpenOCD debugger and an updated version of the test script.
2. The automatic testing platform according to claim 1, characterized in that: The control module further includes: a test management module, a process management module and a Telnet client; the input of the test management module is connected to the test script library, the output is connected to the process management module and the Telnet client, and the outputs of the process management module and the Telnet client are connected to the OpenOCD debugger; The test management module is used to obtain test cases from the test script library, create a test thread for each test case and manage the test thread; The process management module is used to create and manage the process of the OpenOCD debugger according to the test thread; The Telnet client is used to create and manage Telnet links and communicate with the OpenOCD debugger through the Telnet links. The test thread sends commands of the test script to the OpenOCD debugger through the Telnet links and receives test results returned by the OpenOCD debugger.
3. The automatic testing platform according to claim 2, characterized in that: The OpenOCD debugger includes a Telnet server, and the Telnet client communicates with the OpenOCD debugger through the Telnet server.
4. The automatic testing platform according to claim 2, characterized in that: The test script contains the commands required to run the OpenOCD debugger.
5. The automatic testing platform according to claim 2, characterized in that: The test results include the execution results of each test case and related hardware status information.
6. The automatic testing platform according to claim 1, characterized in that: The OpenOCD debugger is connected to the debugger via a USB port, and the debugger is connected to the debugging target device via a JTAG protocol.
7. The automatic testing platform according to claim 1, characterized in that: The test script is written in TCL language.
8. The automatic testing platform according to claim 1, characterized in that: The test script library includes an XIP mode verification script, and the XIP mode verification script includes the following test process; Write verification data to the flash memory of the QSPI interface in normal working mode; If the writing is successful, loading the XIP mode check program into the random access memory, the XIP mode check program including: Configuring the flash memory of the QSPI interface to XIP mode; Read the flash memory of the QSPI interface in the XIP mode; Calculate the checksum; Determine whether the verification code is correct, and if so, return the verification result.
9. The automatic testing platform according to claim 8, characterized in that: If writing verification data to the flash memory of the QSPI interface fails in normal working mode, a write error is reported.
10. The automatic testing platform according to claim 8, characterized in that: If the check code is incorrect, the XIP mode configuration is incorrect and an error is reported.