Fault injection-oriented Internet of Things reliability test method and system

Abstract

The invention relates to a fault injection-oriented Internet of Things reliability test method and system, and aims at carrying out reliability test on an existing Internet of Things service in a real operation environment under the condition of not interrupting the existing Internet of Things service. The system comprises an Internet of Things service mirroring assembly, an input data forwarding assembly and a fault injection assembly. The mirror image component generates a mirror image of the original service by copying a software and hardware environment; the input data forwarding component copies and forwards the input data of the original service to the mirror image service in real time; and the fault injection component modifies, intercepts or counterfeits the forwarded data, injects a fault and observes the abnormal state of the mirroring service. By comparing the operation difference between the mirror image service and the original service, the reliability defect of the Internet of Things service is found. According to the invention, efficient non-perceptual reliability testing is realized, and the method is suitable for various application scenes of the Internet of Things.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things and software architecture, and in particular to a method and system for performing reliability testing on Internet of Things services in terms of functionality, security, etc. by injecting faults, aiming to minimize the impact on running Internet of Things services and to achieve reliability testing of Internet of Things services without interfering with user experience. Background Art

[0002] With the rapid development of network infrastructure such as 5G and fiber-optic broadband, the popularization of mobile network technology and the improvement of chip processing capabilities, the number of terminal devices supporting IoT functions has continued to increase. IoT services have been widely used in production, manufacturing, security warning and other fields. The systems in these fields usually involve a large number of terminal nodes with a wide distribution range, and have extremely high timeliness requirements for the collection and processing of terminal data. In addition, when abnormal data is detected, the system needs to respond quickly and effectively to ensure the continuity and stability of the service.

[0003] In this case, reliability testing of IoT services faces huge challenges. On the one hand, the test needs to be fully exposed to the real environment to ensure that the system can meet the stress test requirements of massive data and high performance in the real environment; on the other hand, it is crucial to design an efficient test system without affecting the normal operation of existing services, so as to discover the potential defects of IoT services in the real environment to the greatest extent.

[0004] However, traditional IoT service testing methods usually rely on small-scale simulation data for testing. Although this method can evaluate the system to a certain extent in a laboratory environment, its results are often not accurate and comprehensive enough in the actual operating environment. Since simulation data cannot fully simulate the complex and changeable situations in the real world, there is a large difference between the test results and the actual operating conditions, which cannot meet the needs of high-precision reliability testing. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a fault injection-oriented reliability testing system for the Internet of Things. The system aims to provide an efficient reliability testing method for the Internet of Things in a real operating environment without interrupting the existing Internet of Things services. Different from the traditional testing method, the present invention performs real-time fault injection by copying and running the image of the Internet of Things service, thereby realizing the reliability evaluation of the system in a real environment.

[0006] The implementation of the present invention mainly includes the following steps, through which the reliability test of the existing IoT service can be performed without affecting its normal operation. The implementation can be adjusted and optimized according to the specific application scenario, but the overall process includes the following steps:

[0007] S1 Construction of mirror environment: In the operating environment of IoT services, build an operating environment that is consistent with the original service. In the constructed operating environment, copy the software code, configuration files, database files, and log files of the original service to generate a digital mirror that is consistent with the original service;

[0008] S2 Data forwarding and operation drive: Through data forwarding, the external input of the original IoT service (which may include network data packets, file input, sensor data, user interaction instructions, etc.) is monitored in real time, and the data is copied in real time and forwarded to the digital mirror service;

[0009] S3 Fault injection and anomaly detection: Fault injection is performed on the image service to test its behavior and performance under abnormal conditions. Fault injection methods include but are not limited to data modification, data interception, and data forgery. The implementer observes the abnormal performance of the image service under fault injection and records these abnormal performances as status indicators for analysis;

[0010] S4 Comparison of abnormal status with original service: The implementer compares the output data of the mirrored service with the output data of the original service, detects the difference between the two, and infers the specific impact of the specific fault on the IoT service;

[0011] S5 Result recording and analysis: The implementer records the abnormal performance of the mirror service after fault injection and the differences with the original service as the test conclusion and provides it to developers and testers for analyzing potential defects in the system and further optimizing the reliability of IoT services.

[0012] Based on the above steps, the fault injection-oriented IoT reliability testing system proposed by the present invention mainly includes the following core components, and their relationships can be found in the attached Figure 2 :

[0013] IoT service mirroring component: This component is responsible for duplicating the existing IoT service to be tested at the software and hardware levels. First, by building the same hardware operating environment as the original IoT service, a foundation is laid for the operation of the upper-layer software; then, by copying the upper-layer running software code, the operating system of the original IoT service is mirrored to generate a digital copy that is consistent with the original IoT service for subsequent testing.

[0014] IoT service input data forwarding component: This component is used to monitor and copy all external input data of the original IoT service and forward it to the mirror service to drive the mirror service to run in the same way as the original service.

[0015] IoT service fault injection component: This component injects faults into the replicated forwarded data received by the mirror service. This includes operations such as data modification, data interception, and data forgery. By observing the abnormal state of the mirror service under fault injection and comparing it with the operating state of the original service, the impact of the fault on the IoT service can be inferred and recorded. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Reliability test flow chart designed for the present invention.

[0017] Figure 2 The test system architecture diagram designed for the present invention. DETAILED DESCRIPTION

[0018] like Figure 1 As shown, in the execution steps of the present invention:

[0019] S1. Image environment construction, specifically, includes: 1) Software and hardware environment replication: Use container technology (such as Docker, Kubernetes) or virtualization technology (such as VMware, VirtualBox) to build a software and hardware operating environment consistent with the existing IoT service. Through image containerization, the operating system, application, dependency packages, etc. are packaged together to form a reusable service image. 2) Service image generation: In the image environment, copy the relevant software code, configuration files, databases, log files, etc. of the original IoT service to ensure that all necessary components are copied to the image environment. 3) Environment verification: Use monitoring tools (such as Nagios, Prometheus) to verify the correctness of the image environment and ensure that the operating behavior of the image service is consistent with the original service.

[0020] S2. Data forwarding and operation drive, specifically, including: 1) Data capture and replication: Use traffic analysis tools (such as Wireshark, tcpdump) to capture the real-time input data of the original service, such as network packets, sensor input, user instructions, etc. These data are copied in real time and passed as input to the mirror service to keep the input of the two consistent. 2) Data forwarding technology: You can use middleware (such as Kafka, RabbitMQ) or a custom proxy server to forward the captured data to the mirror environment. The goal of forwarding is to synchronize the input of the mirror service with the input of the original service, so as to achieve a real operating environment simulation. 3) Real-time synchronization: Use time synchronization services (such as NTP) to ensure that the data transmission timing between the original service and the mirror service is consistent to ensure the accuracy of the test.

[0021] S3. Fault injection and anomaly detection, specifically, including: 1) Use of fault injection tools: Use fault injection tools (such as Chaos Monkey, Gremlin) to inject faults into the image service. The injected fault types can include: delay injection, network packet loss, wrong data format, memory exhaustion, etc., to simulate abnormal conditions in the actual operating environment. 2) Data modification and forgery: Modify the data passed to the image service, such as changing the value of sensor data, forging data packets in transmission, and discarding some data. These operations can be implemented through Python scripts or using tools (such as Scapy). 3) Abnormal behavior monitoring: Use log analysis tools (such as ELK Stack: Elasticsearch, Logstash, Kibana) or monitoring tools (such as Zabbix, Grafana) to monitor the behavior of the image service after fault injection and record its performance under different abnormal conditions.

[0022] S4. Compare the abnormal state with the original service, specifically, including: 1) Collection of operation results: Collect the output results of the mirror service and the original service through a log collection system (such as Fluentd, Graylog) or a custom script. Different file formats (such as JSON, CSV) can be used to store this data for subsequent analysis. 2) Data comparison and analysis: Use data analysis tools (such as Pandas, NumPy) to compare the output data of the two. Analyze possible differences, such as response time, data integrity, error rate, etc., to determine the impact of the fault on system reliability. 3) Performance difference detection: Compare the performance difference between the mirror service and the original service through performance monitoring tools (such as JMeter, Gatling) to evaluate whether the system behavior after the fault injection is in line with expectations.

[0023] S5. Recording and analysis of results, specifically, including: 1) Fault analysis and recording**: Save the test results after fault injection to the test management platform (such as TestRail, JIRA) so that the development team can conduct subsequent fault analysis and repair. Record the time when the fault occurred, the specific content of the injection, the resulting system anomalies, and the scope of impact, etc. 2) System optimization suggestions**: Based on the above comparative analysis results, generate an automated analysis report, use data visualization tools (such as PowerBI, Tableau) to display the scope and severity of the fault impact, and provide intuitive system optimization suggestions for the development team. 3) Cyclic testing and feedback optimization**: According to the result analysis, repeatedly adjust the fault injection plan and conduct multiple rounds of testing. Through continuous integration systems (such as Jenkins, GitLab CI), realize automated testing and feedback, and gradually optimize the reliability of IoT services.

[0024] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention. Technical Effects

[0025] Through the above steps, the present invention realizes efficient reliability testing of IoT services. By using the comparative analysis of fault injection and mirror services, potential system defects can be discovered without interrupting IoT services, and a basis for continuous optimization of the system can be provided.

Claims

1. A fault injection-guided reliability testing method for the Internet of Things, characterized in that: The following steps are involved: S1. Construction of mirror environment: In the operating environment of the IoT service, build an operating environment that is consistent with the original service, and generate a digital mirror that is consistent with the original service by copying the software code, configuration files, database files and log files of the original service; S2. Data forwarding and operation drive: Through data forwarding, the external input of the original IoT service is monitored in real time, and the data is copied in real time and forwarded to the digital mirror service, so that the mirror service executes the same input as the original service in a consistent operating environment; S3. Fault injection and anomaly detection: Fault injection is performed on the image service, including data modification, data interception, data forgery, etc., and the abnormal state of the image service is observed and recorded as a state indicator; S4. Comparison of abnormal state with original service: Compare the output data of the mirrored service with the output data of the original service, detect the difference between the two, and infer the impact of the specific fault on the IoT service; S5. Result recording and analysis: Record the abnormal performance of the mirror service and its differences from the original service, form a test report, and use it to analyze potential defects in the system.

2. The fault injection-guided Internet of Things reliability testing method according to claim 1 is characterized in that: The construction of the mirror environment is achieved by using virtualization technology or container technology.

3. The fault injection-guided Internet of Things reliability testing method according to claim 1 is characterized in that: The external input includes but is not limited to network data packets, file input, sensor data, user interaction instructions, etc.

4. The fault injection-guided Internet of Things reliability testing method according to claim 1, characterized in that: The fault injection methods include data modification, data interception, data forgery, network delay, data loss, etc.

5. The fault injection-guided Internet of Things reliability testing method according to claim 1, characterized in that: The result record includes recording abnormal performance after fault injection, difference data from the original service, and system operation status.

6. A fault injection-guided Internet of Things reliability testing system, characterized in that: It includes the following components: IoT service mirror component, which is used to replicate the software and hardware environment of the original service and generate a mirror service that is consistent with the original service; IoT service input data forwarding component, used to monitor and replicate the external input data of the original IoT service in real time, and forward it to the mirror service to drive its operation; The IoT service fault injection component is used to inject faults into the mirror service. The injected faults include but are not limited to data modification, data interception, data forgery, etc., and is used to detect abnormal status of the mirror service.

7. The system according to claim 6, characterized in that The mirror component realizes the replication of the software and hardware environment through virtualization technology or container technology.

8. The system according to claim 6, characterized in that The fault injection component can inject network failures such as network delay and data loss to simulate real failure scenarios in the IoT environment.

9. The system according to claim 6, characterized in that The input data forwarding component can capture and copy external input data in real time, including network data packets, sensor data, user input and other forms of input.

Images