A heterogeneous platform embedded software test verification system and method

Abstract

The application provides a heterogeneous platform embedded software test verification system and method, which comprises a board card integration module, an interface management module, a protocol analysis module, a device management module, a use case driving module, an interface driving module, a receiving display control module; the specific process is as follows: the board card integration module is connected to a device under test, the system is started to complete self-checking of system resources, link information is added, test node interconnection is completed, a description text file is obtained, link information is matched, a script file is generated, a dictionary file is obtained, the script file is combined to determine a test case format, load data is added according to the test case format, a test case is formed, an interaction strategy is set, the test case is sent to the device under test, reported data of the device under test is received and displayed, and the test is completed. The application simplifies a test environment, has efficient, low-cost, one-to-many, integrated and platformized interface test capabilities, and meets interface test requirements of embedded software of different platform electronic devices.

Description

Technical Field

[0001] This invention relates to the field of embedded software testing, and in particular to a platform-based embedded software testing and verification system and method. Background Technology

[0002] Embedded software is an important component of various platform products such as airborne, spaceborne, vehicle-mounted, and shipborne systems, and its testing and verification work is particularly important. Currently, due to the huge number, variety, and complex interconnection of embedded software on various platforms, the debugging workload is enormous, especially the interface connection test work, which involves application scenarios such as heterogeneous platforms, multi-bus and multi-protocol conversion, and multi-channel parallel communication. The system testing is incomplete, lacking boundary and performance testing, and the testing efficiency is low. This not only consumes manpower but also leads to frequent problems when the equipment is used later.

[0003] To address the above issues, interface testing, as an important test type, is currently being implemented in the embedded software testing phase across various platforms. Numerous methods have been employed to ensure the completeness, efficiency, and robustness of interface testing, such as serial / network debugging assistants, bus interface-based test monitoring platforms, and various interface testing tools customized for different bus protocols. However, existing hardware-in-the-loop (HIL) simulation systems that integrate various types of bus communication are still lacking. Furthermore, these systems do not readily provide the ability to simulate real-world information interaction or adapt to different platform testing needs, relying primarily on secondary development and widely employing a "one-test-one-replace" testing approach. Figure 1 As shown, because this type of testing method is specifically targeted and the testing software is tightly coupled with the device under test, although the code logic is clear and the operation is simple, it lacks consideration for the actual use scenario of the system. It cannot take into account the situation of a single device communicating with multiple external buses at the same time or multiple devices communicating synchronously on the bus, making it difficult to detect and locate problems in the real working scenario of the entire system.

[0004] Meanwhile, the main reasons for the current problems of incomplete testing of interface boundaries and performance, poor scalability, limited testing scenarios, and low testing efficiency in most embedded software testing tools are as follows:

[0005] 1) Diverse bus types. Embedded software in various platform devices needs to handle a wide variety of bus communication types (such as: network, CAN bus, RS232 / 422 / 485 bus, MIL-STD-1553B bus, 1394 bus, fiber optic, etc.). The communication protocols of different buses are different, and many existing test software or semi-physical simulation test tools have only one type of bus data simulation, which cannot meet the testing of scenarios such as multi-device multi-bus data interaction in the whole system or single-device multi-bus data communication.

[0006] 2) Diverse protocol standards. Embedded software transmits data through various bus communication interfaces, and designers create corresponding communication interface protocols based on specific business requirements. Although these protocols are designed based on standard communication protocols, the organization of business data is custom-defined, and the definition of communication interface protocols between different devices may also be completely different. This leads to the need for customized handling of interface testing.

[0007] 3) Diverse device communication scenarios. Due to the different data interaction scenarios, methods, and timings between embedded software of different devices, existing testing software cannot simulate real data interaction scenarios to design test cases, that is, to simulate peripheral devices to perform testing and verification of each interface according to the actual working scenario of the system. Therefore, bus communication interface testing is incomplete, and boundary and performance testing under the corresponding functions of the entire system cannot be performed.

[0008] 4) Poor versatility of testing tools. Embedded software interface testing tools are not only meant to help developers and testers verify code interfaces, but also to simulate real peripheral devices, monitor and detect data interaction processes, and help the system troubleshoot and locate problems. However, many current testing software programs suffer from poor reusability and scalability, cannot realistically simulate peripheral devices, are simple and complex to operate, and can only support interface testing that is detached from business logic. Testers also need to be familiar with specific project business requirements, which not only seriously affects testing efficiency but also wastes a lot of human resources.

[0009] Embedded software is widely distributed across different platform devices, interconnected through various buses and networks, and provides external data communication and interaction through interfaces, exhibiting multi-source heterogeneous characteristics. Currently, in the development process of various platform devices, to adapt to the changing external environment, embedded software needs to handle a wide variety of bus types, an increasing number of buses, diverse message content, and inconsistent test performance requirements, presenting a "many-to-many" communication topology. This makes communication interface testing exhibit multi-category, multi-form, and highly complex technical characteristics. Furthermore, interface testing not only needs to inspect the communication interfaces between external systems or between internal subsystems, but also needs to check the data transmission protocols / content / interaction logic. Therefore, embedded software interface testing is one of the effective means of discovering software and system defects, making it particularly important to build an efficient, portable, and complete test and verification environment that can simulate real-world environments. Summary of the Invention

[0010] To address the problems existing in the prior art, a heterogeneous platform embedded software testing and verification system and method are provided. It can be used for communication interface testing of embedded software of various platform products such as airborne, spaceborne, vehicle-mounted, and shipborne systems. It has the ability to perform synchronous and parallel interface testing of multiple devices on heterogeneous platforms, adaptable to various testing scenarios, with an expandable bus and configurable protocols.

[0011] The technical solution adopted in this invention is as follows: A heterogeneous platform embedded software testing and verification system, comprising:

[0012] A board integration module is used to insert one or more sub-boards and connect them to the device under test.

[0013] The interface management module is used for system resource self-checking and configuring link interface parameters;

[0014] The interface driver module is used to adapt to the sub-boards in the board integration module according to the configured link interface parameters, and realize data sending and receiving between the sub-boards;

[0015] The protocol parsing module is used to match link information and generate script files based on the description text file of the predefined interface protocol, and then determine the test case format of the corresponding link based on the predefined dictionary file and script file; and complete the parsing of the reported data and send it to the receiving display and control module.

[0016] The load management module is used to add load data and generate test cases in the corresponding format based on the obtained test case format.

[0017] The test case driven module is used to set the interaction strategy and complete the sending of test cases and the receiving of reported data through the interface driven module.

[0018] The receiving and display module is used to display the reported data parsed by the protocol parsing module.

[0019] Furthermore, the system resource self-check includes checking whether the description text file, dictionary file, and dynamic link library stored in the database are complete; and whether the board driver file matches the sub-board resources; wherein, the description text file defines a variety of protocol interfaces in XML language; the dictionary file defines the description information of a variety of interfaces to be tested in txt format, and sets the test case format that meets the protocol requirements.

[0020] Furthermore, the interface driver module calls the board driver file in the dynamic link library according to the configured link interface parameters to complete the adaptation work with the corresponding paper card of the board integration module.

[0021] Furthermore, in the load management module, load data is added using either user-defined methods or by direct import.

[0022] Furthermore, the payload data supports three data formats: binary, octal, and hexadecimal, and three verification methods: parity check, summation check, and CRC check.

[0023] Furthermore, the interaction strategy in the use case driven module includes the number of transmissions, the transmission interval, and whether to transmit cyclically.

[0024] Furthermore, the description text file can be expanded with new protocol content as needed.

[0025] Furthermore, the dictionary file can be updated with new protocol content as needed.

[0026] Furthermore, the interface driver module controls the data acquisition at the board interface by calling the driver software loaded from the database.

[0027] This invention also proposes a testing and verification method based on the above-mentioned heterogeneous platform embedded software testing and verification system, comprising:

[0028] Step 1: Based on the bus interface type of the device under test, insert one or more corresponding daughter cards into the board integration module and connect them to the device under test using the corresponding vector;

[0029] Step 2: Start the system and complete the system resource self-check;

[0030] Step 3: Add link information to complete the interconnection of test nodes;

[0031] Step 4: Obtain the description text file, match the link information, and generate a script file;

[0032] Step 5: Obtain the dictionary file and combine it with the script file to determine the test case format;

[0033] Step 6: Add load data according to the test case format to form test cases;

[0034] Step 7: Set the interaction strategy and send test cases to the device under test;

[0035] Step 8: Receive and display the data reported by the device under test to complete the test.

[0036] Compared with existing technologies, the beneficial effects of adopting the above technical solution are as follows: This invention supports the function of synchronous testing of multiple bus types, multiple protocols, and multiple devices in a simulated real bus environment, freeing it from the constraints of the traditional testing method of "one test, one change", simplifying the testing environment, and possessing efficient, low-cost, one-to-many, integrated, and platform-based interface testing capabilities. It meets the interface testing needs of embedded software of electronic devices on different platforms, improves the test coverage and automation level of communication interfaces, and reduces the R&D cost and development cycle of testing work. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the existing testing method of one test and one loop.

[0038] Figure 2This is a schematic diagram of the heterogeneous platform embedded software testing and verification system proposed in this invention.

[0039] Figure 3 This is a schematic diagram of the system startup self-test in one embodiment of the present invention.

[0040] Figure 4 This is a schematic diagram of a test example in one embodiment of the present invention. Detailed Implementation

[0041] The embodiments of this application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar modules or modules having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. Rather, the embodiments of this application include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0042] This embodiment proposes a heterogeneous platform embedded software testing and verification system. It possesses the ability to describe communication protocols, establish a descriptive model of the communication protocol, and implement data encoding and decoding functions based on this model. It can simulate actual system bus communication scenarios and perform parallel testing of multi-channel bus data. This ensures that the system meets stability, reliability, and scalability requirements while simultaneously conducting interface logic, functional, boundary, and stress tests. This lays the foundation for testers to flexibly construct interface / system testing environments for embedded software on different platform devices. The specific solution is as follows:

[0043] like Figure 2 As shown, a heterogeneous platform embedded software testing and verification system includes:

[0044] A board integration module is used to insert one or more sub-boards and connect them to the device under test.

[0045] The interface management module is used for system resource self-checking and configuring link interface parameters;

[0046] The interface driver module is used to adapt to the sub-boards in the board integration module according to the configured link interface parameters, and realize data sending and receiving between the sub-boards;

[0047] The protocol parsing module is used to match link information and generate script files based on the description text file of the predefined interface protocol, and then determine the test case format of the corresponding link based on the predefined dictionary file and script file; and complete the parsing of the reported data and send it to the receiving display and control module.

[0048] The load management module is used to add load data and generate test cases in the corresponding format based on the obtained test case format.

[0049] The test case driven module is used to set the interaction strategy and complete the sending of test cases and the receiving of reported data through the interface driven module.

[0050] The receiving and display module is used to display the reported data parsed by the protocol parsing module.

[0051] In this embodiment, the system is mainly divided into two parts: software and hardware. The hardware part includes a board integration module. Based on the principle of scalability, a rack is designed to accommodate pluggable bus boards of various types. It completes the corresponding bus communication configuration by receiving configuration information from the host computer software. The software part includes an interface management module, an interface driver module, a protocol parsing module, a load management module, a test case driver module, and a receiving and display control module. Based on principles of universality, adaptability, human-computer interaction, and test completeness, the architecture is designed to be divided into a human-computer interaction layer, a data link layer, and a function-driven layer.

[0052] In terms of system architecture layer design, this embodiment divides the system into three layers: human-computer interaction layer, data link layer, and function-driven layer. Specifically:

[0053] Human-Computer Interaction Layer: This layer interacts with users through a visual interface, enabling functions such as adding test links, initializing bus configurations, configuring test cases, displaying test process data, and showing test results. The human-computer interaction layer mainly consists of basic functions (startup, self-test), core functions (link configuration, formatting, device management), and auxiliary functions (user management, data management, logs).

[0054] Data Link Layer: Connects to and loads the database, obtaining resources such as descriptive text files, dictionary files, and dynamic link libraries. Simultaneously, it interacts with the function-driven layer, issuing commands to the communication board and receiving reported board-collected and responded data; it also receives test cases output by the human-computer interaction layer and transmits board-reported data to the human-computer interaction layer.

[0055] Functional driver layer: Calls the board driver functions in the dynamic link library to adapt to the communication board. It receives and executes specific hardware operation instructions from the data link layer, then returns the execution results to the data link layer. Simultaneously, it completes the board's data acquisition and reporting.

[0056] In this embodiment, as Figure 3 As shown, the system resource self-check includes checking whether the description text file, dictionary file and dynamic link library stored in the database are complete; and whether the board driver file matches the sub-board resources. Among them, the description text file defines a variety of protocol interfaces in XML language; the dictionary file defines the description information of a variety of interfaces to be tested in txt format, and sets the test case format that meets the protocol requirements.

[0057] Preferably, the description text file can be expanded with new protocol content as needed; the dictionary file can be added with new protocol content as needed.

[0058] In this embodiment, the interface management module mainly includes adding the name of the current test link and configuring interface parameter information such as interface type, message type, message period, message ID, response flag, port number, communication rate, and IP address. Then, the interface driver module calls the board driver file in the dynamic link library according to the configured interface type to complete the adaptation work with the corresponding paper card of the board integration module.

[0059] The protocol parsing module, based on the configured interface parameters, iterates through the description text file to find the protocol content corresponding to the interface to be tested; then, it converts this XML-formatted matching content into a script file. Finally, based on the generated script file, it parses the test case format matching the interface to be tested from the dictionary file.

[0060] In this embodiment, load data is added to the load management module through user definition or direct import to generate test cases with a set format. The load data supports three data formats: binary, octal, and hexadecimal, and three verification methods: parity check, cumulative checksum, and CRC check.

[0061] In the test case-driven module, the interaction strategy is set, including the number of sends, the sending interval, and whether to send in a loop, and interacts with the interface-driven module to complete the sending of test cases and the reception of reported data. The interface-driven module controls the data collection at the interface end by calling the driver software loaded from the database.

[0062] In the embedded software testing and verification system proposed in this embodiment, the parsing process of the reported data is the reverse process of defining the test message. During the parsing process, the function of storing logs is also provided to facilitate the analysis of test problems.

[0063] like Figure 4 As shown, this embodiment uses the example of upstream device 1 and downstream device 2 being connected to the test system via network cable and RS422 cable respectively to verify the implementation effect of this test verification system. Specifically:

[0064] (1) Hardware design of the test system

[0065] The hardware components of the testing equipment include: a portable computer, integrated circuit boards (with Ethernet cards, RS422 / RS485 interface communication boards, 1553 bus boards, ARNIC429 cards, etc.), network cables, 1553b cables, RS422 cables, etc.

[0066] (2) Test system software design

[0067] The software component of the testing equipment includes: testing software and a database (including descriptive text files, dictionary files, and dynamic link libraries). The dynamic link libraries include drivers for various bus boards, such as Ethernet board drivers and RS422 board drivers.

[0068] (3) Testing the effectiveness of the system implementation

[0069] 1) Communication interface test of "Upstream device 1"

[0070] ① Connect "Upstream Device 1" and "Test Host" via network cable;

[0071] ② Add link information, and configure the link name, IP address, port number, etc.;

[0072] ③ Generate a "script file" based on the description text file and link information;

[0073] ④ Determine the test case format based on the script file and dictionary file;

[0074] ⑤ Add payload data and send it, while simultaneously enabling data reception monitoring and storage;

[0075] 2) Communication interface test of "downstream device 2"

[0076] ① Connect the "test host" and "downstream device 2" via an RS422 cable;

[0077] ② Add link information, and configure the link name, serial port number, bits per second, data bits, stop bits, etc.;

[0078] ③ Generate a "script file" based on the description text file and link information;

[0079] ④ Determine the test case format based on the script file and dictionary file;

[0080] ⑤ Add payload data and send it, while simultaneously enabling data reception monitoring and storage;

[0081] 3) Communication and interaction between upstream and downstream equipment

[0082] ① Based on the actual usage scenario of the equipment, set the time interval, number of transmissions, data length, and content format for the upstream device to send data 1;

[0083] ② Set the time interval, number of transmissions, data length, and content format of the data 2 sent by the downstream device after receiving data 1;

[0084] ③ Monitor the reception time and content of data 1 and 2 on upstream and downstream devices to ensure they are correct;

[0085] ④ Configure various data synchronization sending and receiving, real-time display and storage for upstream and downstream devices according to actual usage scenarios, and conduct boundary and stress tests to determine whether they meet the actual usage requirements of the devices.

[0086] The interface testing and verification system of this invention can realize the testing of various bus data interaction scenarios of the semi-physical simulation system to simulate the real system, including functional, performance and boundary tests, all of which are 100% covered. The testing time is reduced by 60% compared with the past, and the labor cost is reduced by 50%. It has been used in the testing phase of multiple electronic equipment systems, and has achieved the effects of convenient test environment construction, high testing efficiency and complete testing, which greatly reduces labor costs and shortens system testing time.

[0087] Example 2

[0088] This embodiment also proposes a test and verification method based on the heterogeneous platform embedded software test and verification system described in Embodiment 1, including:

[0089] Step 1: Based on the bus interface type of the device under test, insert one or more corresponding daughter cards into the board integration module and connect them to the device under test using the corresponding vector;

[0090] Step 2: Start the system and complete the system resource self-check;

[0091] Step 3: Add link information to complete the interconnection of test nodes;

[0092] Step 4: Obtain the description text file, match the link information, and generate a script file;

[0093] Step 5: Obtain the dictionary file and combine it with the script file to determine the test case format;

[0094] Step 6: Add load data according to the test case format to form test cases;

[0095] Step 7: Set the interaction strategy and send test cases to the device under test;

[0096] Step 8: Receive and display the data reported by the device under test to complete the test.

[0097] Specifically, in step 2, the system first checks whether the software resources stored in the database, such as the description text file (XML format), dictionary file (txt format), and dynamic link library, are complete. Then, it checks whether the software resources match the corresponding boards in the board integration module. This process mainly checks whether the board driver file matches the hardware board resources.

[0098] In step 3, in the interface management module, add the name of the current test link and configure interface parameters such as interface type, message type, message period, message ID, response flag, port number, communication rate, and IP address. Simultaneously, based on the configured interface type, the interface driver module calls the corresponding board driver file in the dynamic link library to complete the adaptation work with the corresponding sub-card of the board module.

[0099] In step 4, the protocol parsing module iterates through the description text file from the caller to match the protocol content corresponding to the interface to be tested, based on the interface parameters from step 3; then, it converts this XML-formatted matching content into a script file. This testing system has the ability to convert descriptive languages ​​into programming languages, that is, to convert description text files into script files.

[0100] In step 5, the protocol parsing module parses the test case format that matches the interface to be tested from the dictionary file called, based on the script file generated in step 4.

[0101] In step 6, based on the test case format and the description information of the interface to be tested in the dictionary file, load data is added to the load management module through user definition or direct import to generate test cases with a set format. The load data supports three data formats: binary, octal, and hexadecimal; and supports three verification methods: parity check, cumulative check, and CRC check.

[0102] In step 7, the interaction strategy set in the test case-driven module includes the number of sends, the sending interval, and whether to send in a loop, etc., and the interaction interface-driven module completes the sending of test cases and the reception of reported data. The interface-driven module controls the data collection at the interface end by calling the driver software loaded from the database.

[0103] The interface testing method of this invention was verified during the embedded software communication interface testing of various platform electronic devices. The results compared with the prior art are shown in Table 1 below:

[0104] Table 1. Comparison of the effects of traditional interface testing systems and the present invention.

[0105] High or low scalability Low high High degree of reusability Low high Supported protocol types 1 type Multiple Support the device under test Only the currently tested device is supported. Supports simultaneous testing of multiple devices under test. Number of supported protocols One-to-one agreement processing Supports multiple protocol configurations Does it support parallel testing? Not supported support How to parse message content Semi-automatic / manual analysis Automatic parsing

[0106] As can be seen, compared with traditional testing methods, this method supports synchronous testing of multiple bus types, multiple protocols, and multiple devices in a simulated real bus environment. It breaks free from the "one test, one change" work constraints of traditional testing methods, simplifies the testing environment, and has efficient, low-cost, one-to-many, integrated, and platform-based interface testing capabilities. It meets the interface testing needs of embedded software in electronic devices on different platforms, improves the test coverage and automation level of communication interfaces, and reduces the R&D cost and development cycle of testing work.

[0107] It should be noted that, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "set" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances. The accompanying drawings in the embodiments are used to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0108] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A heterogeneous platform embedded software testing and verification system, characterized in that, include: A board integration module is used to insert one or more sub-boards and connect them to the device under test. The interface management module is used for system resource self-checking and configuring link interface parameters; The system resource self-check includes checking whether the description text file, dictionary file, and dynamic link library stored in the database are complete; and whether the board driver file matches the sub-board resources. The description text file defines various protocol interfaces in XML format; the dictionary file defines description information for various interfaces to be tested in TXT format and sets test case formats that conform to protocol requirements. The description text file can be expanded with new protocol content as needed; the dictionary file can add new protocol content as needed. The interface driver module is used to adapt to the sub-boards in the board integration module according to the configured link interface parameters, and realize data sending and receiving between the sub-boards; The protocol parsing module is used to match link information and generate script files based on the description text file of the predefined interface protocol, and then determine the test case format of the corresponding link based on the predefined dictionary file and script file; and complete the parsing of the reported data and send it to the receiving display and control module. The load management module is used to add load data and generate test cases in the corresponding format based on the obtained test case format. The test case-driven module is used to set the interaction strategy and complete the sending of test cases and the receiving of reported data through the interface-driven module; the interaction strategy in the test case-driven module includes the number of times to send, the sending interval, and whether to send in a loop. The receiving and display module is used to display the reported data parsed by the protocol parsing module.

2. The heterogeneous platform embedded software testing and verification system according to claim 1, characterized in that, The interface driver module calls the board driver file in the dynamic link library according to the configured link interface parameters to complete the adaptation work with the corresponding paper card of the board integration module.

3. The heterogeneous platform embedded software testing and verification system according to claim 1, characterized in that, In the load management module, load data is added either by user definition or by direct import.

4. The heterogeneous platform embedded software testing and verification system according to claim 3, characterized in that, The payload data supports three data formats: binary, octal, and hexadecimal, and three verification methods: parity check, summation check, and CRC check.

5. The heterogeneous platform embedded software testing and verification system according to claim 1, characterized in that, The interface driver module controls the data acquisition at the board interface by calling the driver software loaded from the database.

6. A test and verification method based on the heterogeneous platform embedded software test and verification system according to any one of claims 1-5, characterized in that, include: Step 1: Based on the bus interface type of the device under test, insert one or more corresponding daughter cards into the board integration module and connect them to the device under test using the corresponding vector; Step 2: Start the system and complete the system resource self-check; Step 3: Add link information to complete the interconnection of test nodes; Step 4: Obtain the description text file, match the link information, and generate a script file; Step 5: Obtain the dictionary file and combine it with the script file to determine the test case format; Step 6: Add load data according to the test case format to form test cases; Step 7: Set the interaction strategy and send test cases to the device under test; Step 8: Receive and display the data reported by the device under test to complete the test.

Images