An internet of things device protocol consistency and compatibility automated testing system and method

Abstract

This application relates to an automated testing system for the consistency and compatibility of IoT device protocols. It constructs a protocol knowledge graph through a fusion algorithm of a large model and a protocol state machine, enabling automated and accurate parsing of heterogeneous protocol message structures. By introducing a physical environment evolution sequence model and a semantic probe feedback mechanism, it can simulate dynamic anomaly scenarios in real physical environments and automatically correct private protocol definitions, significantly improving test coverage and parsing accuracy. This effectively solves the consistency and compatibility bottlenecks in the interconnection of IoT devices, reducing labor costs and shortening product launch cycles.

Description

Technical Field

[0001] This application relates to the field of automated testing, and in particular to an automated testing system, method, computer device, and computer-readable storage medium for the conformance and compatibility of Internet of Things (IoT) devices. Background Technology

[0002] With the rapid popularization of IoT technology, the scale of IoT devices has experienced explosive growth. IoT devices are highly heterogeneous, with different manufacturers and models using different communication protocols, and multiple versions of the same protocol existing. This leads to serious protocol consistency and compatibility issues when devices connect to gateways or cloud platforms.

[0003] Currently, protocol consistency and compatibility testing for IoT devices primarily relies on manual or semi-automated testing methods. This approach has the following drawbacks: low testing efficiency, requiring manual parsing of protocol messages, writing test cases, and executing test procedures, making it difficult to adapt to the rapid iteration needs of IoT devices; insufficient test coverage, as manual testing struggles to fully cover protocol anomaly scenarios such as weak networks, packet loss, retransmission, and encryption failures, easily leading to communication interruptions or data loss after device access; and significant difficulty in cross-protocol compatibility verification. The IoT ecosystem contains various heterogeneous protocol interactions, making it difficult for manual methods to construct accurate message mapping models, resulting in low accuracy in cross-protocol interaction testing. Summary of the Invention

[0004] This application provides an automated testing system and method for IoT device protocol conformance and compatibility, which at least solves the problem of low testing efficiency in related technologies.

[0005] In a first aspect, embodiments of this application provide an automated testing system for the protocol conformance and compatibility of Internet of Things (IoT) devices, the system comprising: The protocol parsing module is used to acquire message data from the IoT device under test, and parse the message structure and protocol rules of the message data based on the fusion algorithm of large model and protocol state machine to construct a protocol knowledge graph. The test case generation module is used to generate consistency test cases and compatibility test cases based on the protocol knowledge graph and preset protocol consistency standards. An abnormal scenario injection module is used to simulate abnormal parameters of the network environment and protocol configuration, and inject the abnormal parameters into the test execution process; The test execution module is used to build an automated test environment, schedule test resources to run the test execution process according to the consistency test cases and the compatibility test cases, and collect interactive message data and test logs in real time during the test process; The cross-protocol verification module is used to construct a message mapping model between heterogeneous protocols, and to verify the correctness of cross-protocol message transmission, data format compatibility, and consistency of interaction flow based on the message mapping model.

[0006] In some embodiments, the protocol parsing module specifically includes: The parsing module is used to automatically parse the message structure, field meanings and protocol interaction rules of the message data using a large model that is lightweight and finely tuned based on an IoT protocol corpus and a configurable protocol state machine. The graph construction module is used to associate and store the parsed protocol rules, field information and interaction flow to construct the protocol knowledge graph. The parsing module is also used to identify the protocol custom extension fields of the IoT device under test, and to automatically mark abnormal messages during the parsing process and feed them back to the result analysis module.

[0007] In some embodiments, the parsing module is further configured to: when an unknown field is identified, generate a semantic candidate type for the unknown field through the large model, wherein the unknown field is a protocol-custom extended field that the large model cannot identify based on prior knowledge; The test execution module is further configured to construct a comparison probe message based on the semantic candidate type, and after changing the value of the unknown field in the comparison probe message, send the comparison probe message to the IoT device under test. The parsing module is also used to capture the physical response feedback of the IoT device under test in response to the comparison probe message, and perform response correlation analysis based on the physical response feedback and the value of the unknown field to verify and determine the true semantics of the unknown field and its logical dependency relationship with other fields. The parsing module is also used to perform real-time correction and update of field attribute definitions in the protocol knowledge graph based on the determination results verified by the parsing module.

[0008] In some embodiments, the test case generation module specifically includes: The generation module is used to generate the consistency test cases, including protocol message format verification, interaction process verification and encryption consistency verification, and to generate the compatibility test cases, including cross-protocol message mapping, version downgrade adaptation and multi-device concurrent access. The optimization module is used to optimize the coverage of the consistency test cases and the compatibility test cases based on historical test data and defect cases through reinforcement learning algorithms, and to automatically adjust the test case content according to the update of the protocol type.

[0009] In some embodiments, the abnormal scene injection module is used to inject correlated abnormal scenes based on a physical environment evolution sequence model, specifically including: The acquisition module is used to acquire a preset physical trajectory model, wherein the physical trajectory model defines the correlation and change relationship between network environment parameters and the evolution sequence of the physical environment; The execution module is used to inject interrelated network layer abnormal parameters and protocol layer abnormal parameters into the test execution process at a preset time node sequence according to the evolution sequence of the physical environment, so that the simulated abnormal scenarios change collaboratively according to physical laws.

[0010] In some embodiments, the abnormal scenario injection module is further configured to: During the execution of the correlation anomaly scenario injection, reinforcement learning algorithms are used to monitor the response status of the IoT device under test; Based on the physical trajectory model, the shortest environmental degradation path that causes the protocol stack of the IoT device under test to crash is searched, and the subsequent anomaly injection timing and parameter type are dynamically adjusted according to the shortest environmental degradation path.

[0011] In some embodiments, the test execution module is used to deploy an IoT device access unit, a gateway simulation unit, a cloud platform simulation unit, and a network environment simulation unit that works in conjunction with the abnormal scenario injection module; Furthermore, based on containerization technology, elastic scheduling of test resources is achieved, supporting multi-threaded parallel execution of the test execution process on multiple IoT devices of different types and protocols, and supporting concurrent access simulation of large-scale devices.

[0012] In some embodiments, the cross-protocol verification module specifically includes: The capture module is used to capture cross-protocol interaction message data between different heterogeneous protocols in real time; The verification execution module is used to verify the accuracy of field values ​​and the matching degree of message type after the first protocol message is converted into the second protocol message based on the message conversion rules and field mapping relationships defined by the message mapping model; if an abnormal cross-protocol interaction is found, the abnormal information is marked and synchronously transmitted to the result analysis module.

[0013] In some embodiments, the system further includes: The results analysis module is used to receive the collected interactive message data and the test logs, analyze the test results based on AI algorithms, locate the root cause of defects in the case of test failure, and generate a test report. The data storage module is used to store the protocol knowledge graph, test cases, message data, interaction logs, and test reports using a distributed storage architecture.

[0014] Secondly, embodiments of this application provide an automated testing method for the protocol conformance and compatibility of IoT devices, the method comprising: The message data of the IoT device under test is obtained, and the message structure and protocol rules of the message data are analyzed based on the fusion algorithm of large model and protocol state machine to construct a protocol knowledge graph. Based on the protocol knowledge graph and the preset protocol consistency standards, consistency test cases and compatibility test cases are generated. Simulate abnormal parameters of the network environment and protocol configuration, and inject the abnormal parameters into the test execution process; An automated testing environment is set up. Based on the consistency test cases and the compatibility test cases, test resources are scheduled to run the test execution process, and interactive message data and test logs during the test process are collected in real time. Construct a message mapping model between heterogeneous protocols, and verify the correctness of cross-protocol message transmission, data format compatibility, and interaction process consistency based on the message mapping model.

[0015] Compared to related technologies, the automated testing system for IoT device protocol consistency and compatibility provided in this application constructs a protocol knowledge graph through a fusion algorithm of a large model and a protocol state machine, enabling automated and accurate parsing of heterogeneous protocol message structures. By introducing a physical environment evolution sequence model and a semantic probe feedback mechanism, it can simulate dynamic anomaly scenarios in real physical environments and automatically correct private protocol definitions, significantly improving test coverage and parsing accuracy. This effectively solves the consistency and compatibility bottlenecks in IoT device interconnection, reduces labor costs, and shortens product launch cycles. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a structural block diagram of an automated testing system for IoT device protocol conformance and compatibility according to an embodiment of this application; Figure 2 This is a flowchart of an automated testing method for protocol conformance and compatibility of Internet of Things (IoT) devices according to an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0018] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0019] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0020] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0021] This application provides an automated testing system for the protocol conformance and compatibility of Internet of Things (IoT) devices. Figure 1 This is a structural block diagram of an automated testing system for IoT device protocol conformance and compatibility according to an embodiment of this application, such as... Figure 1 As shown, the system includes a protocol parsing module 10, a test case generation module 11, an exception scenario injection module 12, a test execution module 13, a cross-protocol verification module 14, a result analysis module 15, and a data storage module 16.

[0022] In a specific application embodiment, this system can be used for testing IoT devices in scenarios such as smart homes, covering mainstream protocols such as Bluetooth BLE 5.3, MQTT 5.0, and CoAP. Devices under test may include smart lights, temperature and humidity sensors, smart door locks, etc. The testing objective is to verify the protocol consistency and compatibility between the device under test and the IoT gateway (such as an EMQX gateway) and cloud platform, while also verifying the stability of the device under abnormal scenarios.

[0023] In addition, regarding the system deployment architecture, this system adopts a containerized deployment approach, building each functional module based on Docker containers. For example, it can be deployed on a server configured with 16 CPU cores, 32GB of memory, and 1TB of hard disk to support multi-threaded parallel testing and massive device concurrent simulation. Devices under test can access the test system's device access unit via Bluetooth, NB-IoT modules, etc. The gateway simulation unit simulates an IoT gateway, the cloud platform simulation unit simulates the cloud platform's protocol access interface and data processing logic, and the network environment simulation unit uses a network simulator to construct a realistic physical network environment.

[0024] Specifically, the protocol parsing module 10 performs message parsing and knowledge graph construction. This module acquires message data from the IoT device under test. It identifies the device's protocol type and version information (e.g., smart lights use Bluetooth BLE 5.3, temperature and humidity sensors use MQTT 5.0, smart door locks use the CoAP protocol, etc.). Furthermore, using a lightweight, finely tuned large model based on an IoT protocol corpus, combined with a configurable protocol state machine, it performs automated parsing and knowledge graph construction of the message structure, field meanings, and protocol interaction rules of the message data. Specifically, for example, it parses fields such as topic, payload, and QoS level in MQTT messages, and information such as advertising interval and pairing parameters in Bluetooth BLE messages.

[0025] Furthermore, considering the high heterogeneity of IoT device protocols, and the fact that different manufacturers often define proprietary extended fields on top of standard protocols, traditional hard-coded parsing methods are insufficient to handle unknown protocol logic. Therefore, this application identifies protocol version differences and custom extended fields of the IoT device under test during the parsing process. For unrecognized unknown fields, the module initiates a semantic probe feedback mechanism.

[0026] Specifically, the large model predicts the semantic candidate types of unknown fields, such as sequence numbers or check bits. The test execution module then constructs a comparison probe message, keeping other fields unchanged while monotonically changing only the value of the unknown field, and sends it to the device. The parsing module captures the physical response from the device, performs response correlation analysis, and verifies the true semantics and logical dependencies of the unknown field. Furthermore, the graph construction module associates and stores the parsed rules, fields, and processes to construct a protocol knowledge graph, and updates the graph definition in real time based on the physical feedback results. Simultaneously, abnormal messages occurring during the parsing process (such as format errors or missing fields) are automatically marked and fed back to the result analysis module, providing a basis for subsequent defect analysis.

[0027] Through the aforementioned semantic probe and correlation analysis mechanism, this system achieves automated reverse engineering of black-box protocols, avoiding the tedious process of manually consulting protocol documents and manually capturing and analyzing packets, greatly improving the intelligence level of protocol parsing. It can accurately uncover hidden private interaction logic of devices, ensuring that test cases can cover non-standard defined boundary scenarios and improving the accuracy of identifying unknown fields, providing basic data support for subsequent consistency verification.

[0028] The test case generation module 11 generates test cases based on the protocol knowledge graph and preset standards. It combines preset protocol conformance standards (such as Bluetooth SIG certification standard, MQTT 5.0 standard, CoAP protocol standard, etc.) and compatibility requirements to determine the test scope.

[0029] Specifically, consistency test cases include protocol message format verification, interaction process verification, and encryption consistency verification. Compatibility test cases include cross-protocol message mapping, version downgrade adaptation, and concurrent access by multiple devices. For example, verifying the correctness of message interaction between devices with different protocols (such as Bluetooth and MQTT devices) through a gateway, or verifying the stability when a large number of devices access concurrently.

[0030] This module utilizes reinforcement learning algorithms and combines historical test data to optimize test case coverage. When the protocol version is updated or a new protocol type is added, the module automatically adjusts the test case content and supports user-defined, flexible expansion of test cases, such as adding permission verification test cases for specific devices.

[0031] The anomaly scenario injection module 12 is used to simulate and configure abnormal parameters. In this embodiment, since IoT devices are often in mobile or complex physical environments, network fluctuations are often highly correlated with physical location and environmental changes in time and space. Traditional single, static anomaly injection is difficult to recreate the risks of a real production environment. To address this, this module introduces a Physical Environment Evolution Sequence Model (PEEM). The module obtains a preset physical trajectory model, which defines the correlation between network environment parameters and the evolution sequence of the physical environment.

[0032] Furthermore, based on the evolution sequence of the physical environment, the module synchronously injects network layer fluctuation anomalies and protocol layer connection anomalies at preset time nodes using a cooperative scheduling algorithm. During execution, the module can inject configured anomaly parameters (such as weak network scenarios with bandwidth of 100kbps and latency of 500ms, packet loss scenarios with a packet loss rate of 20%, and encryption anomaly scenarios with incorrect DTLS handshake keys, etc.) into the test execution process and construct and send out anomaly protocol messages.

[0033] Understandably, by introducing the PEEM model and reinforcement learning algorithm, this system can realistically simulate complex protocol anomalies in time-varying scenarios such as devices passing through tunnels or switching base stations at high speeds, automatically detecting the critical thresholds that lead to device failure. Compared to random injection, this method improves the efficiency of discovering deep protocol stack vulnerabilities, quantitatively assesses the communication robustness of devices in extreme dynamic environments, and provides practically valuable indicators for device architecture optimization.

[0034] Test execution module 13 establishes an automated testing environment. Specifically, this environment includes an IoT device access unit, a gateway simulation unit, a cloud platform simulation unit, and a network environment simulation unit. This module uses containerization technology to achieve elastic scheduling of test resources, automatically executing consistency tests, compatibility tests, and abnormal scenario tests in sequence according to the generated test cases. It supports multi-threaded parallel testing on multiple heterogeneous devices and performs concurrent access simulation verification. The module schedules the test resource execution process, collects interaction message data and test logs in real time, collects device status data (online / offline), data reporting success rate, response time, and other indicator data, and records detailed interaction logs including test steps and abnormal events, which are then synchronously transmitted to the result analysis module and data storage module.

[0035] The cross-protocol verification module 14 constructs a heterogeneous protocol message mapping model, which defines message conversion rules between different protocols (such as field mapping and data format conversion between Bluetooth messages and MQTT messages). The module captures cross-protocol interaction messages in real time and verifies the correctness of message conversion, data format compatibility, and consistency of the interaction process during cross-protocol transmission. It should be noted that if an anomaly in cross-protocol interaction is detected during capture (such as message conversion errors or data loss), the anomaly information is immediately marked and synchronously transmitted to the result analysis module.

[0036] The results analysis module 15 analyzes test results based on artificial intelligence algorithms. This module receives data collected by the execution module, compares it with preset protocol standards and compatibility requirements, and determines whether the various indicators of the device under test meet the standards. For test failure scenarios, the module extracts the characteristics of abnormal events from the logs and automatically locates the root cause of the defect. Root causes of defects include protocol field errors, cross-protocol message conversion rule errors, and abnormal environment response timeouts.

[0037] For example, if the device frequently disconnects from the gateway, log analysis reveals that the underlying advertising interval parameters do not conform to the protocol standard, indicating a protocol inconsistency defect. The module generates standardized test reports, which include basic test information, result evaluation, root causes of defects, and optimization suggestions. These reports support visualization and PDF export. Furthermore, statistical analysis of historical test data can identify high-frequency types and patterns of protocol compatibility defects, providing data support for device development.

[0038] The data storage module 16 adopts a distributed storage architecture. This module receives parsing results, test cases, logs, and reports transmitted from other modules, performs categorized storage, and builds indexes to support queries. The module performs data backup and recovery and connects to the blockchain system to store key test data on the blockchain for evidence, ensuring data immutability. At the same time, it feeds back the stored historical test data to the reinforcement learning stage of the test case generation module to continuously optimize subsequent test coverage and execution efficiency.

[0039] This system constructs a protocol knowledge graph through a fusion algorithm of a large model and a protocol state machine, enabling automated and accurate parsing of heterogeneous protocol message structures. By introducing a physical environment evolution sequence model and a semantic probe feedback mechanism, the system can simulate dynamic anomaly scenarios in real physical environments and automatically correct private protocol definitions, significantly improving test coverage and parsing accuracy. It effectively solves the consistency and compatibility bottlenecks in the interconnection of IoT devices, reducing labor costs and shortening product launch cycles.

[0040] This application also provides an automated testing method for the protocol conformance and compatibility of IoT devices. This method is applied to the aforementioned automated testing system. Figure 2 This is a flowchart of an automated testing method for the protocol conformance and compatibility of IoT devices according to an embodiment of this application, as shown below. Figure 2 As shown, the implementation steps of this method are as follows: Step S201: Obtain message data from the IoT device under test, and parse the message structure and protocol rules of the message data based on the fusion algorithm of large model and protocol state machine to construct a protocol knowledge graph.

[0041] First, the raw message data of the IoT device under test is obtained. The lightweight fine-tuning protocol model and protocol state machine are then invoked to perform in-depth analysis of the structure, field semantics, and protocol interaction logic of the message under test. During the analysis, the protocol type and version of the device are automatically identified, and custom extended fields are extracted.

[0042] If an unknown protocol field is identified, the semantic probe feedback process begins. A large model is used to predict candidate semantic types for the unknown field. Subsequently, a set of comparison probe messages is generated, in which only the value of the target unknown field is changed. These comparison probe messages are sent to the IoT device under test, and the device's response is continuously monitored. The ACK signal, transmitted data, or state transition information returned by the device are captured. By analyzing the correlation between the device's response and the changes in the unknown field's value, the true meaning of the unknown field is verified through practical testing.

[0043] For example, if changing a field value causes the device to return a verification error, then that field is determined to be related to the verification logic. Finally, the verified protocol rules, field definitions, and interaction processes are stored in the database, and a protocol knowledge graph is built and dynamically updated.

[0044] Step S202: Based on the protocol knowledge graph and the preset protocol consistency standards, generate consistency test cases and compatibility test cases.

[0045] Based on the protocol knowledge graph constructed in step S201, it is matched against preset protocol consistency standards. Test scripts covering basic protocol consistency and cross-protocol compatibility are automatically generated. The consistency test scripts verify the accuracy of message format, the legality of interaction timing, and the compliance of security encryption algorithms. The compatibility test scripts verify the field mapping between heterogeneous protocols, version compatibility, and large-scale concurrent access capabilities. The test case set is iteratively optimized through a reinforcement learning model to ensure that the test cases cover all boundary cases and state transition paths defined by the protocol.

[0046] Step S203: Simulate abnormal parameters of network environment and protocol configuration, and inject abnormal parameters into the test execution process.

[0047] Specifically, configure abnormal network environment parameters and abnormal protocol logic parameters. Start the physical environment evolution sequence model and set the path trajectory of the physical environment's evolution over time. Based on this trajectory, synchronously adjust network layer parameters and protocol layer parameters. For example, when the simulated device enters a weak network area, simultaneously inject high-latency network noise and abnormal protocol retransmission mechanisms. A cooperative scheduling algorithm ensures that multi-dimensional anomalies are accurately aligned on the timeline. Further probe the device's failure points under continuously deteriorating environmental conditions, recording the combinations of environmental parameters and state transition sequences that lead to device malfunctions.

[0048] Step S204: Build an automated testing environment, schedule test resources to run the test execution process according to consistency test cases and compatibility test cases, and collect interactive message data and test logs during the test process in real time.

[0049] The test environment, including a gateway simulator and a cloud platform simulator, is rapidly deployed using containerization technology. Computational and network resources are scheduled based on the generated test cases. Once testing begins, multiple test threads are launched simultaneously to perform parallel load testing on multiple devices under test. In this embodiment, every interaction message and internal device log is seamlessly collected during the test. Various metrics of the test environment are monitored to ensure the test process executes according to preset logic.

[0050] Step S205: Construct a message mapping model between heterogeneous protocols, and verify the correctness of cross-protocol message transmission, data format compatibility, and consistency of interaction process based on the message mapping model.

[0051] The system processes captured interaction messages in real time. For interaction flows involving protocol conversion, it verifies the semantic equivalence and format correctness of the data before and after conversion using a message mapping model. Subsequently, it scans the test data based on an AI classification algorithm. If the test case execution results are found to be inconsistent with expectations, root cause analysis logic is automatically triggered. It traces back the message flow and log characteristics before and after the anomaly and matches them against the defect feature database. It determines the specific type of defect, including protocol consistency defects, message mapping defects, and environment adaptability defects.

[0052] Step S206: Generate a test report and data storage certificate.

[0053] All test statistics are compiled to generate a standardized test report containing defect distribution charts, test pass rate curves, and optimization solutions. The final test report and key message feature summaries are sent to blockchain nodes for consensus and notarization, generating notarization credentials for subsequent traceability and verification.

[0054] The above steps together form a complete closed loop for automated testing of IoT device protocol consistency and compatibility, achieving a high degree of automation and intelligence in the testing process. This can effectively solve the consistency and compatibility bottlenecks of IoT devices in interconnection, reduce labor costs, and shorten product launch cycles.

[0055] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An automated testing system for the protocol conformance and compatibility of Internet of Things (IoT) devices, characterized in that, The system includes: The protocol parsing module is used to acquire message data from the IoT device under test, and parse the message data's message structure and protocol rules based on a fusion algorithm of a large model and a protocol state machine, in order to construct a protocol knowledge graph; the protocol parsing module specifically includes: The parsing module is used to automatically parse the message structure, field meanings and protocol interaction rules of the message data using a large model that is lightweight and finely tuned based on an IoT protocol corpus and a configurable protocol state machine. The graph construction module is used to associate and store the parsed protocol rules, field information and interaction flow to construct the protocol knowledge graph. The parsing module is also used to identify the protocol custom extension fields of the IoT device under test, and to automatically mark abnormal messages in the parsing process to feed back to the result analysis module, and when an unknown field is identified, to generate a semantic candidate type of the unknown field through the big model, wherein the unknown field is a protocol custom extension field that the big model cannot identify based on prior knowledge. The test case generation module is used to generate consistency test cases and compatibility test cases based on the protocol knowledge graph and preset protocol consistency standards. An abnormal scenario injection module is used to simulate abnormal parameters of the network environment and protocol configuration, and inject the abnormal parameters into the test execution process; The test execution module is used to build an automated test environment, schedule test resources to run the test execution process according to the consistency test cases and the compatibility test cases, and collect interactive message data and test logs in real time during the test process. It also constructs a comparison probe message according to the semantic candidate type, and after keeping other fields unchanged in the comparison probe message and only monotonically changing the value of the unknown field, it sends the comparison probe message to the IoT device under test. The cross-protocol verification module is used to construct a message mapping model between heterogeneous protocols, and to verify the correctness of cross-protocol message transmission, data format compatibility, and consistency of interaction process based on the message mapping model. The parsing module is further configured to capture the physical response feedback of the IoT device under test in response to the comparison probe message, the physical response feedback including the ACK signal returned by the device, the returned data or state transition information; and perform response correlation analysis based on the physical response feedback and the value of the unknown field to verify and determine the true semantics of the unknown field and its logical dependency relationship with other fields. The parsing module is also used to perform real-time correction and update of field attribute definitions in the protocol knowledge graph based on the determination results verified by the parsing module.

2. The system according to claim 1, characterized in that, The test case generation module specifically includes: The generation module is used to generate the consistency test cases, including protocol message format verification, interaction process verification and encryption consistency verification, and to generate the compatibility test cases, including cross-protocol message mapping, version downgrade adaptation and multi-device concurrent access. The optimization module is used to optimize the coverage of the consistency test cases and the compatibility test cases based on historical test data and defect cases through reinforcement learning algorithms, and to automatically adjust the test case content according to the update of the protocol type.

3. The system according to claim 1, characterized in that, The abnormal scenario injection module is used to inject correlated abnormal scenarios based on the physical environment evolution sequence model, specifically including: The acquisition module is used to acquire a preset physical trajectory model, wherein the physical trajectory model defines the correlation and change relationship between network environment parameters and the evolution sequence of the physical environment; The execution module is used to inject interrelated network layer abnormal parameters and protocol layer abnormal parameters into the test execution process at a preset time node sequence according to the evolution sequence of the physical environment, so that the simulated abnormal scenarios change collaboratively according to physical laws.

4. The system according to claim 3, characterized in that, The abnormal scenario injection module is also used for: During the execution of the correlation anomaly scenario injection, reinforcement learning algorithms are used to monitor the response status of the IoT device under test; Based on the physical trajectory model, the shortest environmental degradation path that causes the protocol stack of the IoT device under test to crash is searched, and the subsequent anomaly injection timing and parameter type are dynamically adjusted according to the shortest environmental degradation path.

5. The system according to claim 1, characterized in that, The test execution module is used to deploy the IoT device access unit, gateway simulation unit, cloud platform simulation unit, and network environment simulation unit that works in conjunction with the abnormal scenario injection module; Furthermore, based on containerization technology, elastic scheduling of test resources is achieved, supporting multi-threaded parallel execution of the test execution process on multiple IoT devices of different types and protocols, and supporting concurrent access simulation of large-scale devices.

6. The system according to claim 1, characterized in that, The cross-protocol verification module specifically includes: The capture module is used to capture cross-protocol interaction message data between different heterogeneous protocols in real time; The verification execution module is used to verify the accuracy of field values ​​and the matching degree of message type after the first protocol message is converted into the second protocol message based on the message conversion rules and field mapping relationships defined by the message mapping model; if an abnormal cross-protocol interaction is found, the abnormal information is marked and synchronously transmitted to the result analysis module.

7. The system according to claim 1, characterized in that, The system also includes: The results analysis module is used to receive the collected interactive message data and the test logs, analyze the test results based on AI algorithms, locate the root cause of defects in the case of test failure, and generate a test report. The data storage module is used to store the protocol knowledge graph, test cases, message data, interaction logs, and test reports using a distributed storage architecture.

8. An automated testing method for protocol conformance and compatibility of Internet of Things (IoT) devices, characterized in that, The method includes: The message data of the IoT device under test is obtained, and the message structure and protocol rules of the message data are analyzed based on the fusion algorithm of large model and protocol state machine to construct a protocol knowledge graph. Among them, a large model with lightweight fine-tuning based on an IoT protocol corpus and a configurable protocol state machine are used to automatically parse the message structure, field meaning and protocol interaction rules of the message data; The parsed protocol rules, field information, and interaction flow are associated and stored to construct the protocol knowledge graph; The system identifies the protocol-custom extended fields of the IoT device under test, automatically marks abnormal messages during the parsing process and feeds them back to the result analysis module, and when an unknown field is identified, it generates a semantic candidate type for the unknown field through the large model. The unknown field is a protocol-custom extended field that the large model cannot identify based on prior knowledge. Based on the protocol knowledge graph and the preset protocol consistency standards, consistency test cases and compatibility test cases are generated. Simulate abnormal parameters of the network environment and protocol configuration, and inject the abnormal parameters into the test execution process; An automated testing environment is set up. Based on the consistency test cases and the compatibility test cases, test resources are scheduled to run the test execution process. Interaction message data and test logs during the test process are collected in real time. A comparison probe message is constructed based on the semantic candidate type. After keeping other fields unchanged and only monotonically changing the value of the unknown field in the comparison probe message, the comparison probe message is sent to the IoT device under test. A message mapping model between heterogeneous protocols is constructed, and the correctness of cross-protocol message transmission, data format compatibility, and interaction process consistency are verified based on the message mapping model. Specifically, the physical response feedback of the IoT device under test to the comparative probe message is captured. The physical response feedback includes the ACK signal returned by the device, transmitted data, or state transition information. Response correlation analysis is performed based on the physical response feedback and the values ​​of the unknown fields to verify and determine the true semantics of the unknown fields and their logical dependencies with other fields. Based on the verification results, the field attribute definitions in the protocol knowledge graph are corrected and updated in real time.

Images