An internet of things module software automatic test system

Abstract

The application provides an Internet of Things module software automatic test system. The test system comprises: a user system module that sends network configuration data to a network configuration test module and sends a test task to a communication module. The network configuration test module performs network configuration testing on the Internet of Things module to be tested according to the network configuration data, generates a network configuration verification result, and collects device logs and sends them to the communication module. The knowledge base module of the test intelligent agent constructs a searchable knowledge base based on the original test document; the large model module receives user test requirements via the communication module, calls the searchable knowledge base to perform test operations, generates test processing results, and records process information and outputs them via the communication module. The communication module sends the test task, the network configuration verification result, the device logs, the test processing result, and the process information to a visualization platform, aggregates and arranges the test information, and displays it. The application simplifies the test process and improves the test efficiency.

Description

Technical Field

[0001] This application relates to the field of IoT testing technology, and in particular to an automated testing system for IoT module software. Background Technology

[0002] With the rapid development of IoT technology, IoT modules, as core components for network communication in smart devices, have made software function testing a crucial step in the equipment R&D and production process. System function testing of IoT module software typically involves multiple aspects, including production line testing, distribution network testing, control testing, big data testing, and additional function testing. Among these, distribution network testing and control testing are core components for verifying the module's basic communication capabilities and functional reliability, and are of great significance for ensuring the overall quality of IoT devices.

[0003] In existing technologies, system function testing of IoT module software mainly relies on manual labor. During network configuration testing, manual intervention is required to repeatedly modify router configuration parameters such as frequency band, encryption method, and channel, and to operate a mobile application to complete a series of operations including module configuration, unbinding, and control. The testing process is cumbersome and fragmented. During control testing, testers need to manually design control test cases based on product requirements and execute these cases one by one through a mobile application to traverse the device's functional points. The entire testing process involves a large number of repetitive operations, resulting in a long overall testing cycle.

[0004] However, in existing technical solutions, network configuration testing requires frequent logins to the router's web interface for manual configuration modifications, and control testing necessitates repeated execution of homogeneous operations such as network configuration binding and device control. This high reliance on manual operation leads to low efficiency and excessive time consumption. Furthermore, prolonged repetitive manual tasks easily cause fatigue, increasing the risk of errors or omissions, directly impacting the accuracy and completeness of test results. In addition, inadequate log and operation record retention during testing makes it difficult to quickly pinpoint the root cause of module network configuration or control failures, resulting in low troubleshooting efficiency and severely hindering the overall efficiency and quality of IoT module software testing. Summary of the Invention

[0005] This application provides an automated testing system for IoT module software to solve the technical problems of cumbersome testing processes, large amount of repetitive operations, and low testing efficiency in the existing IoT module software testing process.

[0006] To achieve the above objectives, this application provides an automated testing system for IoT module software, comprising: a cloud server, testing equipment, a testing agent, and a visualization platform, wherein the cloud server, testing equipment, testing agent, and visualization platform interact via network communication; The cloud server is configured to store device binding information, which records the candidate IoT modules bound to the user account. The testing equipment includes: storage module, user system module, distribution network testing module, and communication module; The storage module is configured to store multiple sets of network configuration data, which are used for network configuration testing of IoT modules. The user system module is configured to: respond to a test start command, obtain the test task selected by the user, the test task includes: IoT module under test and distribution network configuration data; send the distribution network configuration data to the distribution network test module, and send the test task to the communication module; The network distribution test module is configured to: perform network distribution tests on the IoT module under test based on the network distribution configuration data in the test task, generate network distribution verification results; collect device logs of the IoT module under test; and send the network distribution verification results and device logs to the communication module. The test agent includes: a knowledge base module and a large model module; The knowledge base module is configured to: receive original test documents uploaded by users; and build a searchable knowledge base based on the original test documents. The large model module is configured to: receive user test requirements via the communication module, invoke a searchable knowledge base to execute test operations adapted to the test requirements, generate corresponding test processing results, and record test process information during the execution of test operations; wherein, the test operations include at least one of generating scripts, retrieving scripts, or running scripts; and output the test processing results and test process information to the outside world via the communication module. The communication module is also configured to send test tasks, network configuration verification results, device logs, test processing results, and test process information to the visualization platform. The visualization platform is configured to aggregate and organize test tasks, network configuration verification results, device logs, test processing results, and test process information to obtain and display test information.

[0007] Preferably, the user system module executes the following in response to the test start command: It retrieves the test task selected by the user, specifically configured as follows: In response to a user's login action, obtain the user's identity token; Based on the identity token, retrieve device binding information from the cloud server; Receive the user's selection instruction for the IoT module in the device binding information, obtain the IoT module selected by the user, and use the IoT module selected by the user as the IoT module to be tested in this test. The system receives a selection instruction from the user for multiple sets of pre-configured distribution network configuration data for the test equipment, obtains the distribution network configuration data selected by the user, and uses the user-selected distribution network configuration data as the distribution network configuration data used in this test.

[0008] Preferably, the distribution network configuration data includes: multiple sets of test cases, each set of test cases including distribution network method and distribution network parameters; The network distribution test module performs network distribution tests on the IoT module under test based on the network distribution configuration data in the test task, and generates network distribution verification results. Specifically, it is configured as follows: Execute test cases one group at a time until all test cases have been executed. The execution operations for a single test case group include: Based on the currently executed test case, send a configuration modification command to the router to adjust the router's wireless configuration parameters, which include: encryption method, wireless mode, wireless channel, frequency band bandwidth, wireless name, wireless password, and wireless broadcast parameters. According to the distribution network method, establish a communication link with the IoT module under test, and send the distribution network parameters to the IoT module under test through the communication link to perform distribution network operation; The network distribution results are received from the IoT module under test via the communication link, and the network distribution results are verified to obtain the network distribution verification results. After each set of test cases is completed, a reset command is sent to the IoT module under test to reset it to the network configuration state, so as to proceed to the execution of the next set of test cases.

[0009] Preferably, the network configuration method includes: AP access point network configuration and Bluetooth network configuration, wherein the communication protocol corresponding to AP access point network configuration is User Datagram Protocol, and the communication protocol corresponding to Bluetooth network configuration is General Attribute Protocol; The network distribution test module establishes a communication link with the IoT module under test according to the network distribution method, and sends the network distribution parameters to the IoT module under test through the communication link to perform network distribution operations. Specifically, it is configured as follows: When the network configuration method is AP access point network configuration, the access point hotspots of the IoT module under test are scanned through the wireless communication interface, and a WiFi communication connection is established with the IoT module under test. The network distribution parameters are sent to the IoT module under test via WiFi communication connection and using the User Datagram Protocol. When the network configuration method is Bluetooth network configuration, the Bluetooth broadcast packets of the IoT module under test are scanned through the wireless communication interface, and a Bluetooth communication connection is established with the IoT module under test. The network distribution parameters are sent to the IoT module under test via Bluetooth communication connection and based on the general attribute protocol.

[0010] Preferably, the distribution network testing module receives the distribution network results returned by the IoT module under test through the communication link and verifies the distribution network results, specifically configured as follows: When the network configuration method is AP access point network configuration, the network configuration result returned by the IoT module under test is received through WiFi communication connection, and the network configuration result is verified to obtain the network configuration verification result; When the network configuration method is Bluetooth configuration, the network configuration result returned by the IoT module under test is received through Bluetooth communication connection, and the network configuration result is verified to obtain the network configuration verification result.

[0011] Preferably, the original test documents include: equipment requirements specifications, communication protocol documents, and original test scripts. The knowledge base module executes a searchable knowledge base based on the original test documents, specifically configured as follows: The equipment requirements specification, communication protocol document, and original test script are semantically sliced ​​according to a preset semantic granularity to obtain multiple document fragments. The document fragments include requirement-type document fragments formed by segmenting the equipment requirements specification, protocol instruction-type document fragments formed by segmenting the communication protocol document, and test case-type document fragments formed by segmenting the original test script. Multiple document fragments are vectorized separately to obtain the vector representation of each document fragment; According to the device category and communication protocol version, each vector representation is associated with the corresponding document fragment and stored in the vector database to form a searchable knowledge base that supports similarity retrieval. The device categories include at least air conditioners, refrigerators, washing machines and smart lights, and the communication protocol versions include Wi-Fi protocol version and Bluetooth protocol version.

[0012] Preferably, the large model module receives user test requirements via the communication module, invokes a searchable knowledge base to perform test operations adapted to the test requirements, and generates corresponding test processing results. Specifically, it is configured as follows: The system receives user test requests through the communication module, parses and identifies these requests, and obtains the execution intent. Based on the execution intent, determine the test operations to be performed; If the test operation is generated by a script, retrieve the requirement document fragments from the searchable knowledge base, parse them, and obtain the functional point descriptions. The protocol instruction set is extracted by retrieving document fragments from a searchable knowledge base and parsing them. Based on the function point description, determine the target instruction set corresponding to the function point description from the communication protocol instruction set; The target instruction set is invoked to generate a test script. The test script includes test case identifiers, preconditions, execution step sequence, expected response values, assertion rules, and post-cleanup operations. The generated test scripts are stored in a searchable knowledge base in a machine-readable format, and the test scripts are the corresponding test processing results.

[0013] Preferably, the large model module is further configured as follows: If the test operation is script retrieval, extract semantic features from the user's test requirements; based on the extracted semantic features, perform semantic similarity retrieval on the test script in the searchable knowledge base to obtain the matching test script, and the matching test script is the test processing result; If the test operation is a script execution, the test script that matches the user's test requirements is retrieved from the searchable knowledge base and the matching test script is run; during the script execution, the running data of the IoT module under test is collected; based on the running data, the running results are generated, and the running results are the test processing results.

[0014] Preferably, the visualization platform is further configured as follows: Device logs are stored in a dual-track system, which includes local log storage and platform-side log storage. Among them, the local log storage records the interaction data between the test device and the IoT module under test from the establishment of the communication connection to the disconnection of the communication connection; The platform logs are segmented and stored according to the start and end timestamps of each test case, and support independent retrieval and error log download by test case.

[0015] Preferably, the test information includes test task information and test execution information; the test execution information includes the total number of distribution network tests executed, the distribution network test pass rate, the total number of control tests executed, the control test pass rate, test case execution details, and test trends; The visualization platform aggregates and organizes test tasks, distribution network verification results, device logs, test processing results, and test process information to obtain and display test information. Specifically, it is configured as follows: The test task is parsed to obtain the test task information, which includes the test task number and the attribute information of the IoT module under test. The attribute information includes: test type, material number and firmware version. Statistical analysis of distribution network verification results yields the total number of distribution network tests performed and the distribution network test pass rate. The test results are statistically analyzed to obtain the total number of control tests executed and the control test pass rate. By correlating the device logs, the execution details of the test cases can be obtained; Analyze the test process information to obtain test trends; Display the test task number, test type, and firmware version according to the dimensions of the test task; The test case execution details are displayed according to the test case dimensions, and keyword-based log retrieval and error log download are supported; It displays the total number of distribution network tests executed, the distribution network test pass rate, the total number of control tests executed, and the control test pass rate. It also displays the trend of the total number of tests executed, the pass rate, and the failure rate within a preset time period in the form of visual statistical charts. It supports filtering by material number and firmware version number in the attribute information.

[0016] As can be seen from the above technical solutions, this application provides an automated testing system for IoT module software. Through the collaborative operation of testing equipment, a testing agent, a visualization platform, and a cloud server, the testing equipment obtains the user-selected test task through the user system module and sends network configuration data to the network testing module. The network testing module automatically performs network configuration testing on the IoT module under test based on the network configuration data. The large model module of the testing agent calls a searchable knowledge base to execute test operations adapted to the user's test requirements. The communication module aggregates the test data and sends it to the visualization platform for aggregated display. This achieves automated execution of IoT module network configuration and control testing, effectively improving testing efficiency. Attached Figure Description

[0017] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the structure of an automated testing system for IoT module software provided in an embodiment of this application; Figure label: Among them, 100 is the cloud server; 200 is the testing equipment; 210 is the storage module; 220 is the user system module; 230 is the power distribution network testing module; 240 is the communication module; 300 is the testing agent; 310 is the knowledge base module; 320 is the large model module; and 400 is the visualization platform. Detailed Implementation

[0019] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application.

[0020] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0021] In the following description, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0022] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0023] In the field of IoT testing technology, existing technologies primarily rely on manual methods for system function testing of IoT module software. Network configuration testing requires frequent logins to the router's web interface to manually modify configurations such as frequency bands, encryption methods, and channels, and repeatedly performing module network configuration binding and unbinding operations. This process is cumbersome and fragmented. Control testing requires manual design of test cases based on product requirements, traversing function points one by one via a mobile application. The repetitive execution of similar operations over long periods easily leads to fatigue, operational errors, or omissions, directly impacting the accuracy and completeness of test results. When module network configuration or control fails, incomplete log and operation record retention makes it difficult to quickly locate the root cause of the fault, resulting in low troubleshooting efficiency. These problems lead to overall low efficiency and lengthy testing time for IoT module software, severely hindering testing progress and product quality assurance.

[0024] To address the aforementioned issues, this application provides an automated testing system for IoT module software. See also... Figure 1 The system includes a cloud server 100, testing equipment 200, testing agent 300, and visualization platform 400. The cloud server 100, testing equipment 200, testing agent 300, and visualization platform 400 interact via network communication.

[0025] Cloud server 100 is used to store device binding information related to user accounts.

[0026] The cloud server 100 is configured to store device binding information, which records the candidate IoT modules bound to the user account.

[0027] Specifically, the cloud server 100 stores device binding information. This device binding information records all candidate IoT modules bound to the user's account. The device binding information is recorded and stored by the cloud server 100 after the user completes the binding operation by scanning the identification code on the IoT module through a mobile application or manually entering the module's serial number when using the system for the first time. Each candidate IoT module in the device binding information corresponds to a test type, part number, and firmware version. This information is used to identify the model and current software status of the module under test during subsequent testing.

[0028] Test device 200 is a local computing device deployed at the test site. Test device 200 is used to perform local control and data processing tasks during the IoT module distribution network testing process. Test device 200 includes a storage module 210, a user system module 220, a distribution network test module 230, and a communication module 240. Storage module 210 is a non-volatile storage unit within test device 200. Storage module 210 is used to store multiple pre-configured sets of distribution network configuration data.

[0029] Storage module 210 is configured to store multiple sets of network configuration data, which are used for network configuration testing of IoT modules.

[0030] Specifically, the multiple sets of distribution network configuration data stored in storage module 210 are pre-configured test case sets. Each set of distribution network configuration data contains a complete set of distribution network test parameters. Storage module 210 manages the multiple sets of distribution network configuration data in the form of configuration tables or configuration files. Each set of distribution network configuration data is invoked after a test task is selected, and is used by distribution network test module 230.

[0031] User system module 220 is the functional unit in test equipment 200 responsible for user interaction and task management.

[0032] User system module 220 is configured to: in response to a test start command, obtain the test task selected by the user, the test task including the IoT module under test and the distribution network configuration data; send the distribution network configuration data to the distribution network test module 230, and send the test task to the communication module 240.

[0033] Specifically, the user system module 220 receives the test start command input by the user through a human-machine interface. Upon receiving the test start command, the user system module 220 determines the target module for this test from all candidate IoT modules bound to the current user account, and determines the distribution network configuration data to be used in this test from multiple sets of distribution network configuration data. The determined target module is designated as the IoT module under test, and the determined distribution network configuration data is designated as the distribution network configuration data for this test, combining them into a complete test task. The user system module 220 sends the distribution network configuration data from this test task to the distribution network test module 230, and sends the complete test task to the communication module 240.

[0034] In some embodiments, the user system module performs the following actions in response to a test start command: It retrieves the test task selected by the user, specifically configured as follows: The user system module 220 is configured to: respond to the user's login operation and obtain the user's identity token; based on the identity token, retrieve device binding information from the cloud server 100; receive the user's selection instruction for the candidate IoT module in the device binding information, obtain the IoT module selected by the user, and use the IoT module selected by the user as the IoT module to be tested in this test; receive the user's selection instruction for multiple sets of pre-configured network configuration data for the test device 200, obtain the network configuration data selected by the user, and use the network configuration data selected by the user as the network configuration data used in this test.

[0035] Specifically, the user performs a login operation through the human-machine interface. The user system module 220 responds to the user's login operation by initiating an authentication request to the cloud server 100. The cloud server 100 verifies the login information submitted by the user and, upon successful verification, returns an identity token to the user system module 220, which then acquires the identity token. Based on the acquired identity token, the user system module 220 retrieves the device binding information under the current user account from the cloud server 100. This device binding information records all candidate IoT modules already bound to the user account. After receiving the device binding information, the user system module 220 displays all candidate IoT modules included in the device binding information on the human-machine interface. The user performs a selection operation on a candidate IoT module from the device binding information through the human-machine interface. The user system module 220 receives the user's selection instruction for the candidate IoT module, determines the selected IoT module from the device binding information based on the selection instruction, and configures the selected IoT module as the IoT module to be tested in this test. After completing the selection of the IoT module to be tested, the user performs a selection operation on the network configuration data through the human-machine interface. User system module 220 reads multiple pre-configured sets of distribution network configuration data from storage module 210 and displays all the read distribution network configuration data in the human-machine interface. User system module 220 receives a user's selection instruction for distribution network configuration data, determines the user-selected distribution network configuration data from the multiple sets of data based on the selection instruction, and uses the user-selected distribution network configuration data as the distribution network configuration data used in this test. User system module 220 combines the user-selected IoT module under test with the user-selected distribution network configuration data into a complete test task, sends the distribution network configuration data of this test task to distribution network test module 230, and sends the complete test task to communication module 240. At this point, user system module 220 has completed the acquisition and distribution operation of the test task.

[0036] The distribution network test module 230 is the core execution unit in the test equipment 200 responsible for performing distribution network tests of the Internet of Things module.

[0037] The network distribution test module 230 is configured to: perform network distribution tests on the IoT module under test based on the network distribution configuration data in the test task, and generate network distribution verification results; collect device logs from the IoT module under test; and send the network distribution verification results and device logs to the communication module 240.

[0038] Specifically, after receiving the distribution network configuration data from the user system module 220, the distribution network testing module 230 performs distribution network testing operations on the IoT module under test according to the configuration data. During the distribution network testing process, the distribution network testing module 230 synchronously collects the device logs of the IoT module under test based on the device log collection channel established by the user system module 220. The device logs of the IoT module under test record the operating status information and communication interaction records of the IoT module under test during the distribution network testing process. After completing the distribution network test, the distribution network testing module 230 sends the distribution network verification results and device logs to the communication module 240.

[0039] In some embodiments, the network distribution configuration data includes multiple sets of test cases, each set of test cases including network distribution method and network distribution parameters. The network distribution test module 230 performs network distribution testing on the IoT module under test based on the network distribution configuration data in the test task, generating network distribution verification results, specifically configured as follows: Execute test cases one group at a time until all test cases have been executed. The execution operations for a single test case group include: Based on the currently executed test case, a configuration modification command is sent to the router to adjust the router's wireless configuration parameters, which include: encryption method, wireless mode, wireless channel, frequency band bandwidth, wireless name, wireless password, and wireless broadcast parameters.

[0040] Specifically, the network configuration test module 230 executes the network configuration tests group by group according to the order of the test cases. When executing each group of test cases, the network configuration test module 230 first sends a configuration modification command to the router according to the wireless configuration parameters specified in the current test case. After receiving the configuration modification command, the router adjusts its own wireless configuration parameters to the configuration values ​​specified in the current test case. After confirming that the router has completed the configuration modification, the network configuration test module 230 continues to execute the subsequent operations of the current test case.

[0041] Based on the distribution network method, a communication link is established with the IoT module under test, and the distribution network parameters are sent to the IoT module under test through the communication link to perform the distribution network operation.

[0042] Specifically, the distribution network test module 230 determines the communication link establishment method based on the distribution network method in the current test case, and establishes a communication link corresponding to the distribution network method with the IoT module under test through the wireless communication interface of the test device 200. The distribution network test module 230 sends the distribution network parameters to the IoT module under test through the established communication link, so that the IoT module under test can perform distribution network operations according to the distribution network parameters.

[0043] In some embodiments, the network configuration method includes AP access point network configuration and Bluetooth network configuration. The communication protocol corresponding to AP access point network configuration is User Datagram Protocol (UDP), and the communication protocol corresponding to Bluetooth network configuration is General Attribute Protocol (GAP). The network configuration test module 230 establishes a communication link with the IoT module under test according to the network configuration method, and sends the network configuration parameters to the IoT module under test through the communication link to perform the network configuration operation. Specifically, it is configured as follows: When the network configuration method is AP access point configuration, the system scans the access point hotspots of the IoT module under test through the wireless communication interface and establishes a WiFi communication connection with the IoT module under test. Through the WiFi communication connection, the system sends the network configuration parameters to the IoT module under test using the User Datagram Protocol (UDP). When the network configuration method is Bluetooth configuration, the system scans the Bluetooth broadcast packets of the IoT module under test through the wireless communication interface and establishes a Bluetooth communication connection with the IoT module under test. Through the Bluetooth communication connection, the system sends the network configuration parameters to the IoT module under test based on the General Attribute Protocol (GA).

[0044] Specifically, the network configuration test module 230 determines the communication link establishment method based on the network configuration method in the current test case. If the network configuration method is AP access point configuration, the network configuration test module 230 scans for access point hotspot signals emitted by the IoT module under test through the wireless communication interface of the test device 200. After scanning for the access point hotspot signal, the network configuration test module 230 establishes a WiFi communication connection with the IoT module under test. Through this WiFi communication connection, the network configuration test module 230 sends network configuration parameters to the IoT module under test using the User Datagram Protocol (UDP). If the network configuration method is Bluetooth configuration, the network configuration test module 230 scans for Bluetooth broadcast packets emitted by the IoT module under test through the wireless communication interface of the test device 200. After scanning for the Bluetooth broadcast packets, the network configuration test module 230 establishes a Bluetooth communication connection with the IoT module under test. Through this Bluetooth communication connection, the network configuration test module 230 sends network configuration parameters to the IoT module under test based on the General Attribute Protocol (GA).

[0045] The network distribution results are received from the IoT module under test via the communication link, and the network distribution results are verified to obtain the network distribution verification results.

[0046] Specifically, after the network distribution test module 230 sends the network distribution parameters to the IoT module under test, it waits for the IoT module under test to return the network distribution result. The IoT module under test attempts to connect to the target wireless network according to the received network distribution parameters and returns the connection result as the network distribution result to the network distribution test module 230. The network distribution test module 230 receives the network distribution result returned by the IoT module under test through the established communication link. After obtaining the network distribution result, the network distribution test module 230 verifies the network distribution result to determine whether the IoT module under test has successfully connected to the target wireless network and generates a network distribution verification result.

[0047] In some embodiments, the distribution network test module 230 receives the distribution network results returned by the IoT module under test through the communication link and verifies the distribution network results, specifically configured as follows: When the network configuration method is AP access point network configuration, the network configuration result returned by the IoT module under test is received through WiFi communication connection, and the network configuration result is verified to obtain the network configuration verification result; when the network configuration method is Bluetooth network configuration, the network configuration result returned by the IoT module under test is received through Bluetooth communication connection, and the network configuration result is verified to obtain the network configuration verification result.

[0048] Specifically, after the network configuration test module 230 sends the network configuration parameters to the IoT module under test, it receives the network configuration result returned by the IoT module under test through the established communication link. If the current network configuration method is AP access point network configuration, the network configuration test module 230 receives the network configuration result returned by the IoT module under test through WiFi communication connection. If the current network configuration method is Bluetooth network configuration, the network configuration test module 230 receives the network configuration result returned by the IoT module under test through Bluetooth communication connection. After obtaining the network configuration result, the network configuration test module 230 verifies the network configuration result to determine whether the IoT module under test has successfully connected to the target wireless network and generates a network configuration verification result.

[0049] After each set of test cases is completed, a reset command is sent to the IoT module under test to reset it to the network configuration state, so as to proceed to the execution of the next set of test cases.

[0050] Specifically, after the network distribution test module 230 completes the execution of the current group of test cases and obtains the network distribution verification results, it sends a reset command to the IoT module under test. Upon receiving the reset command, the IoT module under test disconnects the currently established network connection and returns to the network distribution state. After confirming that the IoT module under test has returned to the network distribution state, the network distribution test module 230 proceeds to the execution of the next group of test cases until all test cases are completed. The network distribution test module 230 summarizes the network distribution verification results corresponding to each group of test cases and associates the device logs of the IoT module under test collected during the network distribution test execution with the network distribution verification results, then sends the network distribution verification results and device logs to the communication module 240. At this point, the network distribution test module 230 has completed all operations of the network distribution test.

[0051] In some embodiments, the network configuration data in the user-selected test task constitutes a set of test cases. The network configuration test module 230 establishes a communication link with the IoT module under test (DUT) according to the network configuration method in the test cases, and sends the network configuration parameters to the DUT through the established communication link, enabling the DUT to perform network configuration operations based on the parameters. The network configuration test module 230 receives the network configuration results returned by the DUT through the established communication link, verifies the results, and obtains the network configuration verification result. After a single set of test cases is executed, the network configuration test module 230 terminates the process without sending a reset command to the DUT or traversing subsequent test cases.

[0052] Test Agent 300 is deployed in the cloud. Test Agent 300 is used to provide AI-based test script generation and execution capabilities.

[0053] The test agent 300 includes a knowledge base module 310 and a large model module 320. The knowledge base module 310 is the functional unit in the test agent 300 responsible for receiving, processing, and managing test-related documents.

[0054] The knowledge base module 310 is configured to receive original test documents uploaded by users and build a searchable knowledge base based on the original test documents.

[0055] Specifically, the knowledge base module 310 receives the original test document uploaded by the user through the human-computer interaction interface. After obtaining the original test document, the knowledge base module 310 performs semantic segmentation and vectorization processing on the original test document, stores the processed data in a vector database, and constructs a searchable knowledge base.

[0056] In some embodiments, the original test document includes: a device requirements specification, a communication protocol document, and original test scripts. The knowledge base module 310 performs a searchable knowledge base construction based on the original test document, specifically configured as follows: The equipment requirements specification, communication protocol document, and original test script are semantically sliced ​​according to a preset semantic granularity to obtain multiple document fragments. The document fragments include requirement-type document fragments formed by segmenting the equipment requirements specification, protocol instruction-type document fragments formed by segmenting the communication protocol document, and test case-type document fragments formed by segmenting the original test script. Multiple document fragments are vectorized to obtain vector representations for each document fragment. Each vector representation is associated with its corresponding document fragment and stored in a vector database according to the device category and communication protocol version, forming a searchable knowledge base that supports similarity retrieval. The device categories include at least air conditioners, refrigerators, washing machines, and smart lights, and the communication protocol versions include Wi-Fi and Bluetooth protocols.

[0057] Specifically, the knowledge base module 310 receives the original test documents uploaded by the user. The original test documents include equipment requirement specifications, communication protocol documents, and original test scripts. The knowledge base module 310 performs semantic segmentation processing on the equipment requirement specifications, communication protocol documents, and original test scripts according to a preset semantic granularity. The knowledge base module 310 performs semantic segmentation processing on the equipment requirement specifications to obtain requirement-type document fragments. The knowledge base module 310 performs semantic segmentation processing on the communication protocol documents to obtain protocol instruction-type document fragments. The knowledge base module 310 performs semantic segmentation processing on the original test scripts to obtain test case-type document fragments. The knowledge base module 310 performs vectorization processing on all obtained document fragments, converting each document fragment into a corresponding vector representation. The knowledge base module 310 associates each vector representation with its corresponding document fragment and stores them in a vector database according to the equipment category and communication protocol version. After completing the above storage operations, the knowledge base module 310 forms a searchable knowledge base that supports similarity retrieval.

[0058] The large model module 320 is the functional unit in the test agent 300 responsible for interacting with the user and performing test operations.

[0059] The large model module 320 is configured to: receive user test requirements via the communication module 240; invoke a searchable knowledge base to execute test operations adapted to the user test requirements and generate corresponding test processing results; record test process information during the execution of test operations, which include at least one of generating scripts, retrieving scripts, or running scripts; and output the test processing results and test process information externally via the communication module 240.

[0060] Specifically, the large model module 320 receives the test requirements input by the user through the interactive interface via the communication module 240. After obtaining the user's test requirements, the large model module 320 calls the searchable knowledge base constructed by the knowledge base module 310 to execute test operations adapted to the user's test requirements. During the execution of the test operations, the large model module 320 generates corresponding test processing results and synchronously records test process information. After completing the test operations, the large model module 320 outputs the test processing results and test process information to the outside world via the communication module 240.

[0061] In some embodiments, the large model module 320 receives user test requirements via the communication module 240, invokes a searchable knowledge base to perform test operations adapted to the user test requirements, and generates corresponding test processing results, specifically configured as follows: The communication module 240 receives user test requirements and parses and identifies them to obtain the execution intent. Based on the execution intent, it determines the test operation to be executed. If the test operation is script generation, it retrieves requirement document fragments from the searchable knowledge base for parsing to obtain the function point description. It also retrieves protocol instruction document fragments from the searchable knowledge base for parsing to extract the communication protocol instruction set. Based on the function point description, it determines the target instruction set corresponding to the function point description from the communication protocol instruction set. It calls the target instruction set to generate a test script, which includes test case identifiers, preconditions, execution step sequence, expected response value, assertion rules, and post-cleanup operations. The generated test script is stored in the searchable knowledge base in a machine-readable format, and the test script is the corresponding test processing result.

[0062] Specifically, the large model module 320 receives user test requirements via the communication module 240. User test requirements are natural language descriptions input by the user through an interactive interface during the testing process. The large model module 320 parses and identifies the user test requirements, extracts semantic information from them, and determines the user's execution intent based on the extracted semantic information. The large model module 320 determines the test operations to be executed based on the execution intent. If the execution intent is to generate a script, the large model module 320 retrieves and parses requirement document fragments from the searchable knowledge base, extracting function point descriptions from the requirement document fragments. The large model module 320 retrieves and parses protocol instruction document fragments from the searchable knowledge base, extracting communication protocol instruction sets from the protocol instruction document fragments. Based on the function point descriptions, the large model module 320 determines the target instruction set corresponding to the function point descriptions from the communication protocol instruction set. The large model module 320 calls the target instruction set to generate a test script containing test case identifiers, preconditions, execution step sequences, expected response values, assertion rules, and post-cleanup operations. The above script fields are preferred implementations and can be adjusted according to test requirements, and are not limited to these. The large model module 320 stores the generated test scripts in a machine-readable format in a searchable knowledge base, and these test scripts are the corresponding test processing results.

[0063] In some embodiments, the large model module 320 is further configured as follows: If the test operation is script retrieval, extract semantic features from the user's test requirements; based on the extracted semantic features, perform semantic similarity retrieval on the test script in the searchable knowledge base to obtain the matching test script, and the matching test script is the test processing result; If the test operation is a script execution, the test script that matches the user's test requirements is retrieved from the searchable knowledge base and the matching test script is run; during the script execution, the running data of the IoT module under test is collected; based on the running data, the running results are generated, and the running results are the test processing results.

[0064] Specifically, if the execution intent is to retrieve a script, the large model module 320 extracts semantic features from the user's testing requirements. Based on the extracted semantic features, the large model module 320 performs a semantic similarity search on all stored test scripts in the searchable knowledge base. Based on the semantic similarity matching results, the large model module 320 determines the test script that matches the user's testing requirements. The large model module 320 uses the matched test script as the test processing result. If the execution intent is to run a script, the large model module 320 retrieves the test script that matches the user's testing requirements from the searchable knowledge base. The large model module 320 runs the matched test script, collecting operational data from the IoT module under test during script execution. The large model module 320 generates an operational result based on the collected operational data; this operational result is the test processing result.

[0065] The communication module 240 is a functional unit in the test equipment 200 responsible for data interaction with the test agent 300 and the visualization platform 400.

[0066] The communication module 240 is configured to send test tasks, network configuration verification results, device logs, test processing results, and test process information to the visualization platform 400.

[0067] Specifically, the communication module 240 receives test tasks from the user system module 220, distribution network verification results and device logs from the distribution network test module 230, and test processing results and test process information from the large model module 320. The communication module 240 then aggregates the received test tasks, distribution network verification results, device logs, test processing results, and test process information and sends them uniformly to the visualization platform 400. After the communication module 240 completes data transmission, the visualization platform 400 can then perform subsequent processing on the received data.

[0068] The visualization platform 400 is deployed in the cloud. It is used to aggregate and display data generated throughout the entire testing process.

[0069] The visualization platform 400 is configured to aggregate and organize test tasks, network configuration verification results, device logs, test processing results, and test process information to obtain test information, which is then displayed on the visualization platform 400.

[0070] Specifically, the visualization platform 400 receives test tasks, network configuration verification results, device logs, test processing results, and test process information sent by the communication module 240. The visualization platform 400 aggregates and organizes this data, transforming the scattered test data into structured test information according to the test task dimension. After completing the aggregation and organization, the visualization platform 400 displays this test information.

[0071] In some embodiments, the visualization platform 400 is further configured to: The device logs are stored in a dual-track manner. The visualization platform 400 stores the device logs in a dual-track manner, including local log storage and platform-side log storage. The local log storage records the interaction data between the test device 200 and the IoT module under test from the establishment of the communication connection to the disconnection of the communication connection. The platform-side logs are segmented and stored according to the start and end timestamps of each test case, and support independent retrieval and error log download by test case.

[0072] Specifically, after receiving the device logs sent by the communication module 240, the visualization platform 400 performs dual-track storage of the device logs. The local log storage records all interaction data between the test device 200 and the IoT module under test from the establishment of the communication connection to its termination. The platform-side logs are segmented and stored according to the start and end timestamps of each test case. After storage, the visualization platform 400 allows users to independently retrieve the platform-side logs by test case. The visualization platform 400 also allows users to download error logs from the retrieved logs.

[0073] In some embodiments, test information includes test task information and test execution information; test execution information includes total distribution network test execution volume, distribution network test pass rate, total control test execution volume, control test pass rate, test case execution details, and test trends; The visualization platform 400 aggregates and organizes test tasks, network configuration verification results, device logs, test processing results, and test process information to obtain and display test information. Specifically, it is configured as follows: The system parses test tasks to obtain test task information, including the test task number and attribute information of the IoT module under test, such as test type, material number, and firmware version. It also statistically analyzes distribution network verification results to obtain the total number of distribution network tests executed and the distribution network test pass rate. Furthermore, it statistically analyzes test processing results to obtain the total number of control tests executed and the control test pass rate. The system correlates device logs to obtain test case execution details. It analyzes test process information to obtain test trends. The system displays test task numbers, test types, and firmware versions according to the dimension of test tasks. It also displays test case execution details according to the dimension of test cases, and supports keyword-based log retrieval and error log download. Finally, it displays the total number of distribution network tests executed, the distribution network test pass rate, the total number of control tests executed, and the control test pass rate, and presents the trends of total test execution, pass rate, and failure rate within a preset time period in a visual statistical chart format. It supports filtering by material number and firmware version number from the attribute information.

[0074] Specifically, the visualization platform 400 parses the test tasks, extracting the test task number and attribute information of the IoT module under test, including test type, part number, and firmware version. The visualization platform 400 statistically analyzes the distribution network verification results, calculating the total number of executions and pass rates for distribution network tests, and thus the total number of distribution network test executions and the distribution network test pass rate. The visualization platform 400 also statistically analyzes the test processing results, calculating the total number of executions and pass rates for control tests, and thus the total number of control test executions and the control test pass rate. The visualization platform 400 performs correlation analysis on device logs, associating them with corresponding test cases to obtain the execution details of each test case. The visualization platform 400 analyzes the test process information, extracting the execution time sequence and status changes of each test case to obtain test trends. After completing the above data processing, the visualization platform 400 displays the test task number, test type, and firmware version according to the test task. The visualization platform 400 displays the execution details of each test case according to the test cases, and provides keyword search and error log download entry points in the display interface, supporting users to search for logs and download error logs based on keywords. The visualization platform 400 displays the total number of distribution network test executions, distribution network test pass rate, total number of control test executions, and control test pass rate. The visualization platform 400 displays the trends of total test executions, pass rate, and failure rate within a preset time period in the form of visual statistical charts, and provides material number and firmware version number filtering entry points in the chart display interface, allowing users to filter the displayed data by material number and firmware version number in the attribute information.

[0075] As can be seen from the above technical solution, this embodiment obtains the user-selected test task through the user system module 220 and automatically sends the distribution network configuration data to the distribution network test module 230. The distribution network test module 230 automatically executes the distribution network test and collects device logs according to the distribution network configuration data, realizing the automated execution of the distribution network test without the need for manual modification of router configuration and operation of mobile application. The large model module 320 of the test agent 300 receives the user's test requirements through the communication module 240 and calls the searchable knowledge base constructed by the knowledge base module 310 to execute test operations adapted to the user's test requirements, realizing the automatic generation and execution of control test scripts without the need for manual design of test cases and traversal of function points. The test task, distribution network verification result, device log, test processing result, and test process information are summarized and sent to the visualization platform 400 for aggregated display through the communication module 240, realizing the centralized display of test data and logs. This embodiment effectively solves the problems of cumbersome test processes, large amount of repetitive operations, and difficulty in tracing faults in the prior art, and improves the efficiency and traceability of IoT module software testing.

[0076] The above specific embodiments further illustrate the purpose, technical solution and beneficial effects of this application. It should be understood that the above are only specific embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of this application should be included within the scope of protection of this application.

Claims

1. An automated testing system for IoT module software, characterized in that, include: The system includes a cloud server, testing equipment, a testing agent, and a visualization platform, wherein the cloud server, the testing equipment, the testing agent, and the visualization platform interact via network communication. The cloud server is configured to store device binding information, which records the candidate IoT modules bound to the user account. The testing equipment includes: a storage module, a user system module, a distribution network testing module, and a communication module; The storage module is configured to store multiple sets of network distribution configuration data, which are used for network distribution testing of IoT modules. The user system module is configured to: in response to a test start command, obtain a test task selected by the user, the test task including: the IoT module under test and the distribution network configuration data; send the distribution network configuration data to the distribution network test module, and send the test task to the communication module; The network distribution test module is configured to: perform network distribution tests on the IoT module under test according to the network distribution configuration data in the test task, and generate network distribution verification results; collect device logs of the IoT module under test; and send the network distribution verification results and device logs to the communication module. The test agent includes: a knowledge base module and a large model module; The knowledge base module is configured to: receive original test documents uploaded by users; and construct a searchable knowledge base based on the original test documents. The large model module is configured to: receive user test requirements via the communication module, invoke the searchable knowledge base to execute test operations adapted to the test requirements, generate corresponding test processing results, and record test process information during the execution of the test operations; wherein, the test operations include at least one of generating scripts, retrieving scripts, or running scripts; and output the test processing results and the test process information externally via the communication module. The communication module is also configured to send the test task, the network configuration verification result, the device log, the test processing result, and the test process information to the visualization platform; The visualization platform is configured to aggregate and organize the test tasks, the network configuration verification results, the device logs, the test processing results, and the test process information to obtain test information, and then display the test information.

2. The automated testing system for IoT module software according to claim 1, characterized in that, The user system module performs the following actions in response to the test start command: It retrieves the test task selected by the user, specifically configured as follows: In response to a user's login action, obtain the user's identity token; Based on the identity token, device binding information is retrieved from the cloud server; Receive the user's selection instruction for the candidate IoT module in the device binding information, obtain the IoT module selected by the user, and use the IoT module selected by the user as the IoT module to be tested in this test. The system receives a selection instruction from the user for multiple sets of pre-configured distribution network configuration data for the test equipment, obtains the distribution network configuration data selected by the user, and uses the user-selected distribution network configuration data as the distribution network configuration data used in this test.

3. The automated testing system for IoT module software according to claim 2, characterized in that, The distribution network configuration data includes: multiple sets of test cases, each set of test cases including distribution network method and distribution network parameters; The network distribution test module performs network distribution tests on the IoT module under test based on the network distribution configuration data in the test task, and generates network distribution verification results. Specifically, it is configured as follows: The test cases are executed one group at a time until all test cases have been executed. The execution of a single group of test cases includes: Based on the currently executed test case, a configuration modification command is sent to the router to adjust the router's wireless configuration parameters, which include: encryption method, wireless mode, wireless channel, frequency band bandwidth, wireless name, wireless password, and wireless broadcast parameters. According to the network distribution method, a communication link is established with the IoT module under test, and the network distribution parameters are sent to the IoT module under test through the communication link to perform network distribution operation; The network distribution result returned by the IoT module under test is received through the communication link, and the network distribution result is verified to obtain the network distribution verification result; After each set of test cases is executed, a reset command is sent to the IoT module under test to reset it to the network configuration state, so as to proceed to the execution of the next set of test cases.

4. The automated testing system for IoT module software according to claim 3, characterized in that, The network configuration methods include: AP access point network configuration and Bluetooth network configuration, wherein the communication protocol corresponding to the AP access point network configuration is User Datagram Protocol, and the communication protocol corresponding to the Bluetooth network configuration is General Attribute Protocol. The network distribution test module establishes a communication link with the IoT module under test according to the network distribution method, and sends the network distribution parameters to the IoT module under test through the communication link to perform network distribution operations. Specifically, it is configured as follows: When the network configuration method is AP access point network configuration, the access point hotspots of the IoT module under test are scanned through the wireless communication interface, and a WiFi communication connection is established with the IoT module under test. The network configuration parameters are sent to the IoT module under test via the WiFi communication connection and using the User Datagram Protocol. When the network configuration method is Bluetooth network configuration, the Bluetooth broadcast packets of the IoT module under test are scanned through the wireless communication interface, and a Bluetooth communication connection is established with the IoT module under test. The network configuration parameters are sent to the IoT module under test via the Bluetooth communication connection based on the general attribute protocol.

5. The automated testing system for IoT module software according to claim 4, characterized in that, The network distribution test module receives the network distribution results returned by the IoT module under test through the communication link and verifies the network distribution results. Specifically, it is configured as follows: When the network configuration method is AP access point network configuration, the network configuration result returned by the IoT module under test is received through the WiFi communication connection, and the network configuration result is verified to obtain the network configuration verification result; When the network configuration method is Bluetooth network configuration, the network configuration result returned by the IoT module under test is received through the Bluetooth communication connection, and the network configuration result is verified to obtain the network configuration verification result.

6. The automated testing system for IoT module software according to claim 5, characterized in that, The original test documents include: equipment requirements specifications, communication protocol documents, and original test scripts. The knowledge base module executes a searchable knowledge base based on the original test documents, specifically configured as follows: The equipment requirements specification, the communication protocol document, and the original test script are semantically sliced ​​according to a preset semantic granularity to obtain multiple document fragments. The document fragments include requirement-type document fragments formed by segmenting the equipment requirements specification, protocol instruction-type document fragments formed by segmenting the communication protocol document, and test case-type document fragments formed by segmenting the original test script. The document fragments are vectorized separately to obtain the vector representation of each document fragment; According to the device category and communication protocol version, each vector representation is associated with the corresponding document fragment and stored in the vector database to form a searchable knowledge base that supports similarity retrieval. The device categories include at least air conditioners, refrigerators, washing machines and smart lights, and the communication protocol versions include Wi-Fi protocol versions and Bluetooth protocol versions.

7. The automated testing system for IoT module software according to claim 6, characterized in that, The large model module receives user test requests via the communication module, invokes the searchable knowledge base to perform test operations adapted to the test requests, and generates corresponding test processing results. Specifically, it is configured as follows: The communication module receives user test requests and parses and identifies these requests to obtain the execution intent. Based on the execution intent, determine the test operation to be executed; If the test operation is generated by a script, the requirement document fragment is retrieved from the searchable knowledge base and parsed to obtain the functional point description; The protocol instruction document fragments are retrieved from the searchable knowledge base, parsed, and the communication protocol instruction set is extracted. Based on the function point description, determine the target instruction set corresponding to the function point description from the communication protocol instruction set; The target instruction set is invoked to generate a test script, which includes test case identifiers, preconditions, execution step sequence, expected response values, assertion rules, and post-cleanup operations. The generated test script is stored in the searchable knowledge base in a machine-readable format, and the test script is the corresponding test processing result.

8. The automated testing system for IoT module software according to claim 6 or 7, characterized in that, The large model module is also configured as follows: If the test operation is script retrieval, the semantic features in the user test requirements are extracted; based on the extracted semantic features, the test script is semantically similar to the test script in the searchable knowledge base to obtain a matching test script, and the matching test script is the test processing result; If the test operation is a script execution, a test script that matches the user's test requirements is retrieved from the searchable knowledge base, and the matching test script is run. During the execution of the script, the operating data of the IoT module under test is collected; Based on the running data, running results are generated, which are the test processing results.

9. The automated testing system for IoT module software according to claim 8, characterized in that, The visualization platform is also configured to: The device logs are stored in a dual-track system, which includes local log storage and platform-side log storage. The local log storage records the interaction data between the test device and the IoT module under test from the establishment of the communication connection to the disconnection of the communication connection; The platform logs are segmented and stored according to the start and end timestamps of each test case, and support independent retrieval and error log download by test case.

10. The automated testing system for IoT module software according to claim 9, characterized in that, The test information includes test task information and test execution information; the test execution information includes the total number of distribution network tests executed, the distribution network test pass rate, the total number of control tests executed, the control test pass rate, test case execution details, and test trends; The visualization platform aggregates and organizes the test tasks, network configuration verification results, device logs, test processing results, and test process information to obtain test information, and displays the test information. Specifically, it is configured as follows: The test task is parsed to obtain the test task information, which includes the test task number and the attribute information of the IoT module under test. The attribute information includes: test type, material number and firmware version. The distribution network verification results are statistically analyzed to obtain the total number of distribution network tests performed and the distribution network test pass rate. The test results are statistically analyzed to obtain the total number of control tests executed and the control test pass rate. By correlating the device logs, the execution details of the test cases can be obtained; The test process information is analyzed to obtain test trends; Display the test task number, the test type, and the firmware version according to the dimensions of the test task; The test case execution details are displayed according to the dimensions of the test cases, and keyword-based log retrieval and error log download are supported; The system displays the total number of network distribution tests executed, the pass rate of network distribution tests, the total number of control tests executed, and the pass rate of control tests. It also displays the trends of the total number of tests executed, the pass rate, and the failure rate within a preset time period in the form of visual statistical charts. It supports filtering by material number and firmware version number in the attribute information.

Images