Embedded hardware-in-the-loop automated testing method based on double board level configuration and structured serial port protocol

Abstract

The application relates to the technical field of software testing, and discloses an embedded hardware-in-the-loop automatic test method based on double-board-level configuration and a structured serial port protocol. Through the combination design of the structured serial port protocol, double-board-level configuration abstraction and independent test firmware, the generalization and decoupling of the test system are realized. The unified [TAG] payload protocol format makes a set of analysis logic adapt to all compliant firmware, and eliminates the fragmented analysis cost. The JSON board-level configuration separates the hardware parameters and the test logic, and the same test case can be adapted to different hardware platforms by switching the configuration. Through the independent acquisition board and the double-channel data fusion cross verification mechanism, the reliability and fault positioning capability of the test result are improved from the architecture level. The independent acquisition board measures external physical signals such as voltage, current and temperature through physical probes, and breaks the closed-loop defect of the traditional scheme that the DUT "measures and judges itself".

Description

Technical Field

[0001] This invention relates to the technical field of software testing, specifically to an embedded hardware-in-the-loop automated testing method based on dual-board configuration and structured serial port protocol. Background Technology

[0002] Embedded firmware testing is a core component of electronic product R&D, mass production, and quality control, directly determining the functional stability and electrical reliability of devices. Current embedded testing solutions mainly fall into three categories: first, manual testing relying on human intervention, which is inefficient and inconsistent; second, software simulation testing based on tools like QEMU, which cannot reproduce the electrical characteristics of real hardware; and third, industrial-grade hardware-in-the-loop systems such as dSPACE and NI, which offer high precision but are costly and difficult to adapt to consumer-grade embedded platforms and small-to-medium-scale R&D scenarios. With the widespread adoption of general-purpose embedded platforms such as ESP32 and STM32, and the rapid growth in the number of IoT devices and open-source hardware projects, the industry's demand for low-cost, reproducible, and reusable hardware-in-the-loop testing solutions continues to increase.

[0003] Existing embedded testing technologies have the following drawbacks: First, there is no unified standard for firmware serial port output formats, with free text, JSON, and mixed binary formats coexisting. Test scripts need to customize parsing logic for each format, making it impossible to form a universal testing framework. Second, test code is embedded in the product firmware through compiler macros, resulting in strong coupling between the two. Modifying the test logic requires recompiling the entire product code, posing a risk of introducing production defects into the test code. Third, test scripts hardcode hardware pins and peripheral parameters, requiring extensive code rewriting when changing development boards, leading to poor test case portability. Fourth, the testing process only retains the final pass / fail results, lacking end-to-end evidence such as compilation, flashing, and raw data, failing to meet quality traceability and auditing requirements. Fifth, all test data comes from self-reporting by the firmware under test, forming a closed loop of "self-testing and self-judgment," which cannot detect firmware sampling and calculation errors, makes it difficult to distinguish between software logic errors and hardware physical faults, and results in complete data loss after firmware crashes, inherently compromising test reliability. Summary of the Invention

[0004] (a) Technical problems to be solved To address the following shortcomings of existing embedded testing technologies: First, there is no unified standard for firmware serial port output formats, with free text, JSON, and mixed binary formats coexisting. Test scripts require customized parsing logic for each format, making it impossible to form a universal testing framework. Second, test code is embedded in the product firmware via compiler macros, resulting in strong coupling between the two. Modifying test logic requires recompiling the entire product code, posing a risk of introducing production defects into the test code. Third, test scripts hardcode hardware pins and peripheral parameters, requiring extensive code rewriting when changing development boards, leading to poor test case portability. Fourth, the testing process only retains the final pass / fail result, lacking end-to-end evidence such as compilation, flashing, and raw data, failing to meet quality control requirements. Fifth, all test data comes from the self-reporting of the firmware under test, forming a closed loop of "self-testing and self-judgment". It is impossible to detect firmware sampling and calculation errors, and it is difficult to distinguish between software logic errors and hardware physical failures. After the firmware crashes, the data flow is completely interrupted, and the reliability of the test is inherently insufficient. To achieve the above, the structured serial port protocol is used to unify the parsing logic, and the dual-board configuration enables the test to be ported across hardware. The independent acquisition board dual-channel cross-verification breaks the self-testing closed loop of the DUT and can accurately distinguish between firmware calculation errors and hardware failures. The structured audit package retains evidence of the entire link, and the single / dual-channel pluggable architecture takes into account cost and accuracy, which significantly improves the universality, reliability and traceability of embedded testing.

[0005] (II) Technical Solution This invention is achieved through the following technical solution: an automated hardware-in-the-loop testing method for embedded systems based on dual-board configuration and structured serial port protocol, the method comprising: Step S1: Define at least one board-level configuration file for target hardware in a structured data format. The board-level configuration file includes hardware serial port parameters, pin mapping relationships, and signal channel attributes. Step S2: Define test cases using a declarative configuration file. Each test case is associated with a corresponding board-level configuration file through a unique board-level identifier. Step S3: Locate the corresponding test firmware based on the board-level configuration file associated with the test case, and automatically execute firmware compilation and target hardware flashing; Step S4: During the test firmware operation of the target hardware, a text message following a preset structured serial port protocol is output via the serial port. The text message consists of two parts: a fixed tag and a data payload. Step S5: The serial port acquisition engine captures the text messages output by the target hardware in real time, parses them into structured data according to the tag type, and adds a collection timestamp to each data. Step S6: Read the preset assertion conditions in the test cases, match and compare the parsed structured data with the assertion conditions, and output the test judgment result.

[0006] Preferably, step S1 specifically includes: Two types of board-level configuration files are defined using JSON format: the DUT board-level configuration and the independent acquisition board configuration. Each board-level configuration file is assigned a unique board-level identifier and role identifier. The role identifier is used to distinguish between the hardware's test function and data acquisition function. DUT board-level configuration stores serial port parameters, GPIO pin mappings, logic signal channel definitions, and test firmware paths; The independent acquisition board is configured to store serial port parameters, analog measurement pins, sensor types, and physical signal channel definitions.

[0007] Preferably, step S2 specifically includes: The declarative configuration file uses YAML format and includes the test name, timeout threshold, execution steps, and assertion conditions. The assertion conditions support regular expression pattern matching and set a distinguishing marker between mandatory assertions and optional assertions. If a mandatory assertion fails, the corresponding test step is deemed to have failed.

[0008] Preferably, step S3 specifically includes: Load the corresponding DUT board configuration based on the board-level identifier associated with the test case, and read the test firmware path and compilation environment parameters from the configuration; The embedded build system is invoked to automatically compile the firmware, and the test firmware is automatically burned to the target hardware after compilation. The test firmware is independent of the product firmware code library and is a single-function test program that only calls standard hardware APIs to implement test logic.

[0009] Preferably, step S4 specifically includes: The default text message format for the structured serial port protocol is [TAG]payload, where TAG is a predefined uppercase label and payload is space-separated key-value pair data. The label includes at least a test start marker, a running status marker, a measurement data marker, and a test result marker; Each message occupies a separate line, and the collection timestamp is appended by the serial port acquisition engine at the receiving end, without occupying firmware operation and storage resources.

[0010] Preferably, step S5 specifically includes: When an independent acquisition board is configured, the serial acquisition engine simultaneously acquires data from the DUT's serial port and the independent acquisition board's serial port through dual channels. The independent acquisition board connects to the DUT's measured node through physical probes, uses analog-to-digital converters or digital sensors to measure analog physical signals such as voltage, current, and temperature, and reports them to the host computer through its own serial port according to the preset structured serial port protocol. The acquisition engine adds a source-based microsecond-level acquisition timestamp to the dual-channel data to ensure that the time base of the two data streams is consistent. When no independent acquisition board is configured, the serial port acquisition engine operates in single-channel mode, acquiring only the self-reported data from the DUT serial port and attaching the acquisition timestamp, and outputting the parsed structured data for functional logic verification.

[0011] Preferably, a dual-channel data fusion and cross-validation step is included between step S5 and step S6. This step is performed only when an independent acquisition board is configured. The deviation value is generated by subtracting the self-reported measurement value output by the DUT serial port from the actual measured value of the corresponding physical quantity output by the independent acquisition board serial port. When the deviation value exceeds the preset threshold, it is automatically attributed to firmware calculation error or hardware failure, marked as a test anomaly, and the deviation data is recorded. When the deviation value does not exceed the preset threshold, it is determined that the DUT self-reported data is consistent with the physical measured value and the data is valid. The fused and aligned dual-channel dataset is then input into step S6 to participate in the assertion determination.

[0012] Preferably, step S6 is followed by an audit package archiving step: After a single test is completed, regardless of whether the test result is pass or fail, all test lifecycle data will be packaged according to a standardized directory structure to generate a structured audit package. The structured audit package includes at least the test firmware source code, compilation log, burning log, raw serial port data, parsing result file, test result metadata, and test environment information.

[0013] Preferred, When the test result is passed, the final judgment and all process data will be solidified into a structured audit package, and the test process will end. When the test result is a failure, the iterative retry mechanism is automatically triggered to re-execute the firmware compilation, flashing and data acquisition process. The test data of all iterations are accumulated and stored in the same structured audit package. The method supports pluggable switching between single-channel and dual-channel modes. When no independent acquisition board is configured, it automatically degenerates to single-channel mode, only acquiring DUT serial port data for functional logic verification.

[0014] An automated hardware-in-the-loop testing system based on dual-board-level configuration and structured serial port protocol is provided to implement the aforementioned automated hardware-in-the-loop testing method based on dual-board-level configuration and structured serial port protocol. The system includes: The board-level configuration storage module is used to store the board-level configuration files of the target hardware in a structured data format and to manage board-level identifiers and role identifiers. The test definition parsing module is used to parse declarative test cases, associate them with the corresponding board-level configuration files through board-level unique identifiers, and extract assertion conditions. The compilation and burning engine is used to locate the test firmware based on the board configuration and automatically perform firmware compilation and target hardware burning. The serial port acquisition engine supports switchable single-channel and dual-channel modes, and is used to capture structured text messages output by the target hardware serial port in real time, parse them into structured data by tags and attach a collection timestamp. The data fusion and decision engine is used to perform dual-channel data alignment, cross-validation, and bias attribution, and to match structured data with assertion conditions to output test decision results; The audit package archiving module is used to package test lifecycle data in a standardized structure and supports the cumulative storage of data from multiple iterations.

[0015] (III) Beneficial Effects This invention provides an automated hardware-in-the-loop testing method for embedded systems based on dual-board configuration and a structured serial port protocol. It offers the following advantages: First, by combining structured serial port protocols, dual-board-level configuration abstraction, and independent test firmware, the test system achieves universality and decoupling. The unified [TAG] payload protocol format allows a single parsing logic to adapt to all compliant firmware, eliminating fragmented parsing costs. JSON board-level configuration separates hardware parameters from test logic, and the same test case can be adapted to different hardware platforms by switching configurations, greatly improving portability. Independent test firmware is completely isolated from the product firmware code library, eliminating the risk of test code introducing production defects from the root and reducing the maintenance costs of multi-project testing.

[0016] Second, by using an independent acquisition board and a dual-channel data fusion cross-validation mechanism, the reliability of test results and fault location capabilities are improved at the architectural level. The independent acquisition board measures physical signals such as voltage, current, and temperature from the outside through physical probes, breaking the closed-loop defect of the DUT's "self-testing and self-judgment" in traditional solutions. The dual-channel data based on timestamp alignment can directly compare self-reported values ​​with measured values. When the deviation exceeds the threshold, it automatically distinguishes between firmware calculation errors and hardware faults. If the deviation is within the threshold, the data is confirmed to be valid. It can identify hidden problems such as firmware ADC sampling deviation and logic calculation errors, and also covers physical signal measurements that the DUT cannot perceive, significantly improving the accuracy of electrical performance testing and the efficiency of fault diagnosis.

[0017] Third, the engineering practicality and scenario adaptability of the testing system are enhanced through structured audit packages, iterative retry mechanisms, and pluggable architecture design. The full lifecycle audit package completely retains firmware source code, compilation and burning logs, raw serial port data, and judgment results, supporting offline reproduction of the testing process and meeting the needs of quality traceability and CI / CD pipeline integration. The automatic iterative retry mechanism for test failures adapts to unstable testing scenarios, and multiple rounds of data are uniformly archived to retain the complete troubleshooting process. The single / dual channel switchable design takes into account both low-cost functional verification and high-precision performance testing scenarios. The hardware cost is far lower than that of industrial-grade hardware-in-the-loop systems, and it can be adapted to diverse application scenarios such as consumer IoT, open-source hardware, and industrial embedded systems. Attached Figure Description

[0018] Figure 1 A flowchart of the embedded hardware-in-the-loop automated testing method based on dual-board configuration and structured serial port protocol provided by the present invention; Figure 2 This is a schematic diagram of the running module of the embedded hardware-in-the-loop automated testing method based on dual-board configuration and structured serial port protocol provided by the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] An example of the embedded hardware-in-the-loop automated testing method based on dual-board configuration and structured serial port protocol is as follows: Example 1: The host computer operating environment in this embodiment 1 is Ubuntu 22.04LTS operating system. The test framework is developed based on Python 3.10 and relies on third-party libraries such as PyYAML 6.0, pyserial 3.5, and PlatformIOCore 6.1. The circuit under test (DUT, DeviceUnderTest) uses an ESP32-WROOM-32 development board, and the independent acquisition board uses the same model ESP32 development board with an external INA219 current sensor, a resistor voltage divider circuit, and an NTC thermistor. All firmware is developed based on the Arduino framework. The following describes the lower-level implementation and detailed implementation process of each technical solution in accordance with the steps of the claims.

[0021] Please see Figure 1 An automated hardware-in-the-loop testing method for embedded systems based on dual-board configuration and structured serial port protocol, the method includes: Step S1: Define at least one board-level configuration file for the target hardware in a structured data format. The board-level configuration file includes hardware serial port parameters, pin mapping relationships, and signal channel attributes. Step S1 specifically includes: Two types of board-level configuration files are defined using JSON format: the DUT board-level configuration and the independent acquisition board configuration. Set a unique board-level identifier and role identifier for each board-level configuration file. The role identifier is used to distinguish between the hardware's test function and data acquisition function. DUT board-level configuration stores serial port parameters, GPIO pin mappings, logic signal channel definitions, and test firmware paths; The independent acquisition board is configured to store serial port parameters, analog measurement pins, sensor types, and physical signal channel definitions.

[0022] The specific implementation is as follows: The test framework manages all board-level configuration files in the boards / directory. The file names are consistent with the board_id field. When the framework starts, it loads all configurations in batches and creates an index through the built-in json module of Python.

[0023] The complete content of the DUT board-level configuration file boards / dut-esp32-wroom-32.json is as follows: Where board_id is a globally unique identifier, role:dut marks the board as the object to be tested, platformio_env specifies the compilation environment, and test_firmware_path supports dynamic matching of test cases using the {test_name} placeholder.

[0024] The complete content of the independent acquisition board configuration file boards / acq-esp32-v1.json is as follows: When the framework loads the configuration, it automatically identifies the board role through the role field. When the test case contains the acq_board field, it automatically enables dual-channel mode; otherwise, it automatically degenerates into single-channel mode.

[0025] Test cases are associated with corresponding hardware configurations by referencing board-level identifiers, thereby decoupling test logic from hardware details.

[0026] Step S2: Define test cases using declarative configuration files. Test cases are associated with the corresponding board-level configuration files through a unique board-level identifier. Step S2 specifically includes: The declarative configuration file uses YAML format and includes the test name, timeout threshold, execution steps, and assertion conditions. Assertions support regular expression pattern matching and can be distinguished from mandatory assertions by setting a flag. If a mandatory assertion fails, the corresponding test step is considered to have failed.

[0027] The specific implementation is as follows: Test cases are uniformly stored in the tests / directory. The framework parses the files using the PyYAML library, converting the steps and assertions into internal execution instructions. Taking LED flicker current testing as an example, the complete content of the test case tests / led-current-test.yaml is as follows: Assertion matching is implemented using Python's built-in `re` regular expression module. The matching logic is as follows: P1. Traverse the collected structured data line by line and perform regular expression matching on the payload portion of each data line; P2. Count the number of matches for each pattern. If mandatory: true and the number of matches is 0, then this step is considered to have failed. P3. If count_min is set, the match is considered successful only if the number of matches is greater than or equal to the threshold. P4. This test step is considered passed if all assertions pass.

[0028] The test logic is declared entirely in the configuration file, without needing to be written into the firmware code, thus achieving separation between test definition and firmware implementation.

[0029] Step S3: Locate the corresponding test firmware based on the board-level configuration file associated with the test case, and automatically execute firmware compilation and target hardware flashing; Step S3 specifically includes: Load the corresponding DUT board configuration based on the board-level identifier associated with the test case, and read the test firmware path and compilation environment parameters from the configuration; The embedded build system is invoked to automatically compile the firmware, and the test firmware is automatically burned to the target hardware after compilation. The test firmware is independent of the product firmware code library. It is a single-function test program that only calls standard hardware APIs to implement test logic.

[0030] The specific implementation is as follows: The compilation and burning engine is based on the PlatformIO command-line tool, and the execution flow is as follows: M1 Firmware Location: Read the test_firmware_path template from the board configuration, substitute it with the current test name to get the complete path, for example, test-firmwares / led-current-test / main.cpp; The test firmware directory is completely independent of the product firmware directory and belongs to different code repositories, physically isolating the test code from the product code.

[0031] M2. Automatic compilation: Copy the test firmware source code to the src / directory of the PlatformIO temporary project, and call the system command piorun-eesp32dev to execute the compilation; during the compilation process, standard output and error output are captured in real time and written to the compilation log file; if the compilation return code is not 0, the test is directly judged as failed and the subsequent process is terminated.

[0032] M3 Automatic Flashing: After successful compilation, the command piorun-eesp32dev-tupload is called to perform flashing; the flashing tool downloads the firmware to the ESP32 via USB serial port, and automatically triggers a hardware reset after flashing is completed; the entire flashing process is also logged.

[0033] M4, Startup Wait: After the burning process is complete, a fixed delay of 2000ms is executed to ensure that the hardware completes reset and initialization before entering the data acquisition stage.

[0034] In dual-channel mode, the firmware of the independent acquisition board is compiled and burned independently using the same process, and the compilation and burning process of the two boards can be executed in parallel to improve efficiency.

[0035] By combining structured serial port protocols, dual-board-level configuration abstraction, and independent test firmware, the test system achieves universality and decoupling. The unified [TAG] payload protocol format allows a single parsing logic to adapt to all compliant firmware, eliminating fragmented parsing costs. JSON board-level configuration separates hardware parameters from test logic, allowing the same test case to adapt to different hardware platforms by switching configurations, significantly improving portability. Independent test firmware is completely isolated from the product firmware codebase, eliminating the risk of test code introducing production defects at the source and reducing the maintenance costs of multi-project testing.

[0036] Step S4: During the test firmware operation of the target hardware, a text message following a preset structured serial port protocol is output via the serial port. The text message consists of two parts: a fixed tag and a data payload. Step S4 specifically includes: The default text message format for the structured serial port protocol is [TAG]payload, where TAG is a predefined uppercase label and payload is space-separated key-value pair data. The label must include at least a test start marker, a running status marker, a measurement data marker, and a test result marker; Each message occupies a separate line, and the collection timestamp is appended by the serial port acquisition engine at the receiving end, without occupying firmware operation and storage resources.

[0037] The specific implementation is as follows: The DUT test firmware is developed based on the Arduino framework, and all outputs strictly follow the [TAG] payload single-line protocol. The core code is as follows: The ESP32's built-in ADC has a 12-bit resolution, a scale value of 4095, a reference voltage of 3.3V, and its calculation formula conforms to the hardware parameter specifications.

[0038] The raw serial port data stream output by the firmware during runtime is as follows: The protocol design moves the timestamp to the host computer for addition, so the firmware does not need to maintain an RTC or high-precision clock, nor does it need additional code to format the time string, which greatly reduces the firmware Flash usage and operating overhead, while ensuring that all channels have the same time base.

[0039] Step S5: The serial port acquisition engine captures the text messages output by the target hardware in real time, parses them into structured data according to the tag type, and adds a collection timestamp to each data. Step S5 specifically includes: This step supports two running modes, which are automatically switched by the testing framework based on the board configuration: When an independent acquisition board is configured, the serial acquisition engine simultaneously acquires data from the DUT's serial port and the independent acquisition board's serial port through dual channels. The independent acquisition board connects to the DUT's measured node through physical probes, uses analog-to-digital converters or digital sensors to measure analog physical signals such as voltage, current, and temperature, and reports them to the host computer through its own serial port according to a preset structured serial port protocol. The acquisition engine adds a microsecond-level acquisition timestamp to the dual-channel data to ensure that the time base of the two data streams is consistent. When no independent acquisition board is configured, the serial port acquisition engine operates in single-channel mode, acquiring only the self-reported data from the DUT serial port and attaching the acquisition timestamp, and outputting the parsed structured data for functional logic verification.

[0040] The data parsing process is as follows: read the serial port byte stream line by line, extract the tags and payloads using regular expressions; further parse the payloads of [DATA] type messages into a key-value pair dictionary; append a microsecond-level timestamp from the host computer system clock to each parsed data item and store it as a structured JSON object.

[0041] The specific implementation is as follows: The serial port acquisition engine is implemented based on the Python pyserial library, and adopts dual-thread parallel acquisition of dual-channel data. The two threads share the same system clock to ensure that the timestamps are from the same source.

[0042] F1. Hardware Connection: The DUT is connected to the host computer via a USB cable, corresponding to the serial port / dev / ttyUSB0; the independent acquisition board is connected to the host computer via another USB cable, corresponding to the serial port / dev / ttyUSB1; the INA219 current sampling terminal of the independent acquisition board is connected in series in the VCC power supply circuit of the DUT through a DuPont probe, the voltage probe is connected in parallel across the power supply pins of the DUT, and the NTC probe is attached to the surface of the DUT main control chip.

[0043] F2. Acquisition Thread: Each serial port corresponds to an independent acquisition thread, which reads serial port data line by line in a blocking manner; after each complete line of data is read, time.time() is immediately called to obtain the Unix timestamp (with microsecond precision), which is then bound to the original line data and sent to the parsing queue.

[0044] F3. Protocol parsing: Use the regular expression r'^[([AZ]+)]\s*(.*)$' to match each line of data and extract the TAG and payload parts; for the payload of the [DATA] tag, split it into multiple fields by spaces, then split it into key-value pairs by =, and convert it into a Python dictionary.

[0045] F4. Output Structure: The structured data format after parsing a single record is as follows: The data from the independent acquisition board channels uses the exact same parsing logic and timestamp mechanism, with only the `channel` field marked as `acquisition`. Without an independent acquisition board configured, only the DUT acquisition thread is started, outputting single-channel structured data.

[0046] The independent acquisition board firmware performs cyclic measurements at a sampling frequency of 10Hz, and the output format is completely consistent with the DUT. An example output is: Between steps S5 and S6, there is also a dual-channel data fusion and cross-validation step, which is only performed when configuring an independent acquisition board (executed between steps S5 and S6, effective only in dual-channel mode): The deviation value is generated by subtracting the self-reported measurement value output by the DUT serial port from the actual measured value of the corresponding physical quantity output by the independent acquisition board serial port. When the deviation value exceeds the preset threshold, it is automatically attributed to firmware calculation error or hardware failure, marked as a test anomaly, and the deviation data is recorded. When the deviation value does not exceed the preset threshold, it is determined that the DUT self-reported data is consistent with the physical measured value and the data is valid. The fused and aligned dual-channel dataset is then input into step S6 to participate in the assertion determination.

[0047] Timestamp alignment uses a nearest neighbor matching combined with a linear interpolation algorithm: for each DUT data, data from the acquisition board with a time difference of less than 10ms is searched as the matching point; if there is no precise matching point, two adjacent acquisition points are taken and linear interpolation is performed to calculate the measured value at the corresponding time.

[0048] The specific implementation details of the data fusion engine's execution flow and algorithm are as follows: U1. Data preprocessing: Sort the structured datasets of the DUT and the independent acquisition board in ascending order by timestamp, and remove invalid data with abnormal labels or failed parsing.

[0049] U2. Timestamp Alignment: A binary search method is used to find the nearest neighbor for each DUT data in the acquisition board dataset; if the time difference between two points is ≤10ms, the acquisition point is directly used as the matching value; if the time difference is >10ms, the two points before and after are taken. , Corresponding measurement value , The DUT time is calculated using a linear interpolation formula. measured value The linear interpolation formula is as follows: ; U3. Deviation Calculation: For physical quantities of the same name such as current and voltage, the relative deviation is calculated. The formula for calculating the relative deviation is as follows: ,in This indicates the self-reported physical quantity value output from the serial port after the hardware under test (DUT) has been sampled and calculated by the firmware. This indicates the measured value of the same physical quantity obtained by the independent acquisition board through physical measurement by external sensors. The preset relative deviation threshold is 10%, which is determined by superimposing the typical accuracy of ±5% of a general embedded ADC with the accuracy of ±2% of the sensor.

[0050] U4. Deviation Judgment and Attribution: If the relative deviation is ≤10%, the data is deemed valid, a fusion record is generated, and the record is marked as valid:true. If the relative deviation is >10% and the DUT value is systematically small, it is attributed to "firmware ADC sampling coefficient error or calculation logic error"; If the deviation is irregular or jumps drastically, it is attributed to "poor hardware soldering, excessive contact resistance, or sensor failure". All anomalies are recorded with deviation values ​​and attribution results, and simultaneously written into the test results.

[0051] Taking this embodiment as an example, the DUT reports that the current when the LED is lit is 18.5mA. After interpolation by the independent acquisition board, the actual measured value is 35.2mA. The calculated relative deviation is about 47.4%, which is far beyond the 10% threshold. The system automatically marks it as an abnormal item, which is attributed to the firmware ADC calculation error. The abnormal information is synchronously transmitted to step S6 to participate in the final judgment.

[0052] By employing an independent acquisition board and a dual-channel data fusion cross-validation mechanism, the reliability of test results and fault location capabilities are improved at the architectural level. The independent acquisition board measures physical signals such as voltage, current, and temperature from the outside using physical probes, breaking the closed-loop defect of the DUT's "self-testing and self-judgment" in traditional solutions. The dual-channel data, based on timestamp alignment, can directly compare self-reported values ​​with measured values. When the deviation exceeds the threshold, it automatically distinguishes between firmware calculation errors and hardware faults. If the deviation is within the threshold, the data is confirmed to be valid. This not only identifies hidden problems such as firmware ADC sampling deviation and logic calculation errors, but also covers physical signal measurements that the DUT cannot perceive, significantly improving the accuracy of electrical performance testing and the efficiency of fault diagnosis.

[0053] Step S6: Read the preset assertion conditions in the test cases, match and compare the parsed structured data with the assertion conditions, and output the test judgment result.

[0054] When the test result is passed, the final judgment and all process data will be solidified into a structured audit package, and the test process will end. When the test result is a failure, the iterative retry mechanism is automatically triggered to re-execute the firmware compilation, flashing and data acquisition process. The test data of all iterations are accumulated and stored in the same structured audit package. The method supports pluggable switching between single-channel and dual-channel modes. When no independent acquisition board is configured, it automatically degenerates to single-channel mode, only acquiring DUT serial port data for functional logic verification.

[0055] The test evaluation engine iterates through each execution step in the test case, collecting serial port data for a preset duration. It then matches the collected and parsed structured data against the assertion rules within each step. If a forced assertion fails to match, the corresponding step is considered a failure. If all forced assertions in all steps match successfully, the overall test result is output as "pass"; otherwise, a failure is output, along with the failed step and its reason. In dual-channel mode, anomalies marked by cross-validation are simultaneously included in the evaluation criteria, configurable to either directly determine failure or only record an alarm.

[0056] The specific implementation is as follows: the decision engine executes the test steps sequentially, with each step independently timed, collected data, and made a decision. Y1. For the verify_init step, start a 3000ms countdown and continuously collect and parse serial port data; after the countdown ends, check whether the two forced assertions [TEST] and [STATUS] are both successfully matched; in this embodiment, both messages are output within 100ms after startup, all assertions are hit, and the step is judged as passed.

[0057] Y2. For the verify_blink_cycle step, start an 8000ms acquisition window and count the number of times led=ON and led=OFF occur. In this embodiment, the blinking cycle is 1 second, and each occurs 4 times within 8 seconds, which meets the count_min:2 requirement, so the step passes.

[0058] Y3. For the verify_result step, start a 5000ms window to wait for [RESULT]TEST_PASS; in this embodiment, the result is output after 3 flashing cycles, the assertion is hit, and the step passes.

[0059] Y4. In dual-channel mode, additional cross-validation anomaly checks are performed. In this embodiment, if the current deviation exceeds the standard and is configured as "abnormal means failure", the overall result is set to FAIL, and an explanation of the attribution of large current deviation and firmware calculation error is added to the result. If the alarm mode is configured, the overall result is PASS, but a warning message is added.

[0060] The final output test_result.json contains complete information such as the overall result, details of each step, a list of exceptions, and time statistics.

[0061] Step S6 is followed by an audit package archiving step: After a single test is completed, regardless of whether the test result is pass or fail, all test lifecycle data will be packaged according to a standardized directory structure to generate a structured audit package. The structured audit package should include at least the test firmware source code, compilation log, burning log, raw serial port data, parsing result file, test result metadata, and test environment information.

[0062] After a single test is completed, regardless of whether the test result is pass or fail, the test lifecycle data is packaged according to a standardized directory structure to generate a structured audit package. The structured audit package includes at least the test firmware source code, compilation log, burning log, raw serial port data, parsing result file, test result metadata, and test environment information.

[0063] When the test result is passed, the final judgment and all process data are solidified into the structured audit package, and the test process ends. When the test result is failed, the iterative retry mechanism is automatically triggered, and the firmware compilation, flashing and data acquisition process is re-executed. The test data of all iterations are accumulated and stored in the same structured audit package, and the number of iterations and the result of each round are recorded.

[0064] The specific implementation is as follows: After the test is completed, the audit package archiving module automatically creates a directory named after the timestamp and the test name, such as audit_20260613_150000_led-current-test / , and stores all evidence files in it according to the specifications. The `manifest.json` file records the number of iterations. The first test has `iterations: 1`. If the test fails and the maximum number of retries (default 3) is not reached, a second round of testing is automatically triggered. New compilation, flashing, and serial port data are appended to the same audit package with the `iter2` suffix, and the `iterations` field is updated synchronously. Once the test succeeds or the maximum number of retries is reached, the structured audit package is solidified, and the testing process terminates.

[0065] Through structured audit packages, iterative retry mechanisms, and pluggable architecture design, the engineering practicality and scenario adaptability of the testing system are enhanced. The full lifecycle audit package completely retains firmware source code, compilation and burning logs, raw serial port data, and judgment results, supporting offline reproduction of the testing process and meeting the needs of quality traceability and CI / CD pipeline integration. The automatic iterative retry mechanism for test failures adapts to unstable testing scenarios, and multiple rounds of data are uniformly archived to retain the complete troubleshooting process. The single / dual-channel switchable design takes into account both low-cost functional verification and high-precision performance testing scenarios. The hardware cost is far lower than that of industrial-grade hardware-in-the-loop systems, and it can be adapted to diverse application scenarios such as consumer IoT, open-source hardware, and industrial embedded systems.

[0066] Example 2: Please see Figure 2 An automated hardware-in-the-loop testing system based on dual-board configuration and structured serial port protocol is provided to implement an automated testing method for embedded hardware-in-the-loop based on dual-board configuration and structured serial port protocol. The system includes: The board-level configuration storage module is used to store the board-level configuration files of the target hardware in a structured data format and to manage board-level identifiers and role identifiers. The test definition parsing module is used to parse declarative test cases, associate them with the corresponding board-level configuration files through board-level unique identifiers, and extract assertion conditions. The compilation and burning engine is used to locate the test firmware based on the board configuration and automatically perform firmware compilation and target hardware burning. The serial port acquisition engine supports switchable single-channel and dual-channel modes, and is used to capture structured text messages output by the target hardware serial port in real time, parse them into structured data by tags and attach a collection timestamp. The data fusion and decision engine is used to perform dual-channel data alignment, cross-validation, and bias attribution, and to match structured data with assertion conditions to output test decision results; The audit package archiving module is used to package test lifecycle data in a standardized structure and supports the cumulative storage of data from multiple iterations.

[0067] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An automated hardware-in-the-loop testing method for embedded systems based on dual-board configuration and structured serial port protocol, characterized in that: The method includes: Step S1: Define at least one board-level configuration file for target hardware in a structured data format. The board-level configuration file includes hardware serial port parameters, pin mapping relationships, and signal channel attributes. Step S2: Define test cases using a declarative configuration file. Each test case is associated with a corresponding board-level configuration file through a unique board-level identifier. Step S3: Locate the corresponding test firmware based on the board-level configuration file associated with the test case, and automatically execute firmware compilation and target hardware flashing; Step S4: During the test firmware operation of the target hardware, a text message following a preset structured serial port protocol is output via the serial port. The text message consists of two parts: a fixed tag and a data payload. Step S5: The serial port acquisition engine captures the text messages output by the target hardware in real time, parses them into structured data according to the tag type, and adds a collection timestamp to each data. Step S6: Read the preset assertion conditions in the test cases, match and compare the parsed structured data with the assertion conditions, and output the test judgment result.

2. The embedded hardware-in-the-loop automated testing method based on dual-board configuration and structured serial port protocol according to claim 1, characterized in that, Step S1 specifically includes: Two types of board-level configuration files are defined using JSON format: the DUT board-level configuration and the independent acquisition board configuration. Each board-level configuration file is assigned a unique board-level identifier and role identifier. The role identifier is used to distinguish between the hardware's test function and data acquisition function. DUT board-level configuration stores serial port parameters, GPIO pin mappings, logic signal channel definitions, and test firmware paths; The independent acquisition board is configured to store serial port parameters, analog measurement pins, sensor types, and physical signal channel definitions.

3. The embedded hardware-in-the-loop automated testing method based on dual-board configuration and structured serial port protocol according to claim 1, characterized in that, Step S2 specifically includes: The declarative configuration file uses YAML format and includes the test name, timeout threshold, execution steps, and assertion conditions. The assertion conditions support regular expression pattern matching and set a distinguishing marker between mandatory assertions and optional assertions. If a mandatory assertion fails, the corresponding test step is deemed to have failed.

4. The embedded hardware-in-the-loop automated testing method based on dual-board configuration and structured serial port protocol according to claim 1, characterized in that, Step S3 specifically includes: Load the corresponding DUT board configuration based on the board-level identifier associated with the test case, and read the test firmware path and compilation environment parameters from the configuration; The embedded build system is invoked to automatically compile the firmware, and the test firmware is automatically burned to the target hardware after compilation. The test firmware is independent of the product firmware code library and is a single-function test program that only calls standard hardware APIs to implement test logic.

5. The embedded hardware-in-the-loop automated testing method based on dual-board configuration and structured serial port protocol according to claim 1, characterized in that, Step S4 specifically includes: The default text message format for the structured serial port protocol is [TAG]payload, where TAG is a predefined uppercase label and payload is space-separated key-value pair data. The label includes at least a test start marker, a running status marker, a measurement data marker, and a test result marker; Each message occupies a separate line, and the collection timestamp is appended by the serial port acquisition engine at the receiving end, without occupying firmware operation and storage resources.

6. The embedded hardware-in-the-loop automated testing method based on dual-board configuration and structured serial port protocol according to claim 2, characterized in that, Step S5 specifically includes: When an independent acquisition board is configured, the serial acquisition engine simultaneously acquires data from the DUT's serial port and the independent acquisition board's serial port through dual channels. The independent acquisition board connects to the DUT's measured node through physical probes, uses analog-to-digital converters or digital sensors to measure analog physical signals such as voltage, current, and temperature, and reports them to the host computer through its own serial port according to the preset structured serial port protocol. The acquisition engine adds a source-based microsecond-level acquisition timestamp to the dual-channel data to ensure that the time base of the two data streams is consistent. When no independent acquisition board is configured, the serial port acquisition engine operates in single-channel mode, acquiring only the self-reported data from the DUT serial port and attaching the acquisition timestamp, and outputting the parsed structured data for functional logic verification.

7. The embedded hardware-in-the-loop automated testing method based on dual-board configuration and structured serial port protocol according to claim 6, characterized in that, Between steps S5 and S6, there is also a dual-channel data fusion and cross-validation step, which is performed only when an independent acquisition board is configured. The deviation value is generated by subtracting the self-reported measurement value output by the DUT serial port from the actual measured value of the corresponding physical quantity output by the independent acquisition board serial port. When the deviation value exceeds the preset threshold, it is automatically attributed to firmware calculation error or hardware failure, marked as a test anomaly, and the deviation data is recorded. When the deviation value does not exceed the preset threshold, it is determined that the DUT self-reported data is consistent with the physical measured value and the data is valid. The fused and aligned dual-channel dataset is then input into step S6 to participate in the assertion determination.

8. The embedded hardware-in-the-loop automated testing method based on dual-board configuration and structured serial port protocol according to claim 1, characterized in that, The step S6 is followed by an audit package archiving step: After a single test is completed, regardless of whether the test result is pass or fail, all test lifecycle data will be packaged according to a standardized directory structure to generate a structured audit package. The structured audit package includes at least the test firmware source code, compilation log, burning log, raw serial port data, parsing result file, test result metadata, and test environment information.

9. The embedded hardware-in-the-loop automated testing method based on dual-board configuration and structured serial port protocol according to claim 8, characterized in that: When the test result is passed, the final judgment and all process data will be solidified into a structured audit package, and the test process will end. When the test result is a failure, the iterative retry mechanism is automatically triggered to re-execute the firmware compilation, flashing and data acquisition process. The test data of all iterations are accumulated and stored in the same structured audit package. The method supports pluggable switching between single-channel and dual-channel modes. When no independent acquisition board is configured, it automatically degenerates to single-channel mode, only acquiring DUT serial port data for functional logic verification.

10. An embedded hardware-in-the-loop automated testing system based on dual-board configuration and structured serial port protocol, characterized in that: The system is used to implement the embedded hardware-in-the-loop automated testing method based on dual-board configuration and structured serial port protocol as described in any one of claims 1-9, the system comprising: The board-level configuration storage module is used to store the board-level configuration files of the target hardware in a structured data format and to manage board-level identifiers and role identifiers. The test definition parsing module is used to parse declarative test cases, associate them with the corresponding board-level configuration files through board-level unique identifiers, and extract assertion conditions. The compilation and burning engine is used to locate the test firmware based on the board configuration and automatically perform firmware compilation and target hardware burning. The serial port acquisition engine supports switchable single-channel and dual-channel modes, and is used to capture structured text messages output by the target hardware serial port in real time, parse them into structured data by tags and attach a collection timestamp. The data fusion and decision engine is used to perform dual-channel data alignment, cross-validation, and bias attribution, and to match structured data with assertion conditions to output test decision results; The audit package archiving module is used to package test lifecycle data in a standardized structure and supports the cumulative storage of data from multiple iterations.

Images