A method and system for testing an MQTT communication protocol of a substation internet of things device

Abstract

The application discloses a kind of substation internet of things equipment MQTT communication protocol test method and system, comprising: according to the first parameter of target substation internet of things equipment, establish several first equipment models, and the same communication protocol is configured to first equipment model;First test set is preset;First test environment based on first test set is established, and first equipment model is tested in first test environment in combination with first test set;Test report is generated according to test result, and substation internet of things equipment test is completed.Can automatically execute test process, significantly improve test efficiency, reduce the influence of human factors on test result, ensure the objectivity and accuracy of test result.Through the establishment of equipment model and test set, various actual operation scenarios can be simulated, and substation internet of things equipment is comprehensively tested, so as to ensure the stability and reliability of equipment in practical application.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of software testing, and particularly relates to a substation Internet of Things device MQTT communication protocol test method and system. BACKGROUND

[0002] For the increasing number of substation Internet of Things devices, in order to ensure the normal operation and efficient collaborative work between the devices, a stable and reliable communication protocol MQTT (Message Queue Telemetry Transport) is needed as a lightweight and real-time Internet of Things communication protocol, which has been widely applied in the power industry. However, the existing MQTT communication protocol test is generally in a manual test mode. Manual testing mainly relies on test personnel to test by simulating real scenes. The test personnel need to verify various functions of the protocol and record problems encountered during the test process and solutions. However, manual testing is relatively low in efficiency, and the test results are easily affected by the individual experience and technical level of the test personnel. Therefore, the existing needs are not met, and for this purpose, the present application provides a substation Internet of Things device MQTT communication protocol test system. SUMMARY

[0003] This section is intended to summarize some aspects of the embodiments of the present application and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of the specification of the present application in order to avoid obscuring the purpose of this section, the abstract and the title. Such simplifications or omissions cannot be used to limit the scope of the present application.

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a substation Internet of Things device MQTT communication protocol test method and system, which can solve the problems mentioned in the background art.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a substation Internet of Things device MQTT communication protocol test method, comprising:

[0008] A plurality of first device models are established according to the first parameters of the target substation Internet of Things device, and the first device models are all configured with the same communication protocol;

[0009] A first test set is preset, and the first test set at least includes a test scene, a test case and a test instruction;

[0010] establish a first test environment based on the first test set, and test the first device model in combination with the first test set under the first test environment;

[0011] generate a test report according to the test result, and complete the test of the substation IoT device.

[0012] As a preferred scheme of the substation IoT device MQTT communication protocol test method, the testing of the first device model in combination with the first test set under the first test environment comprises:

[0013] testing the first device model in combination with the first test set under the first test environment;

[0014] performing a first judgment on the test process;

[0015] generating a test report according to the first judgment result.

[0016] As a preferred scheme of the substation IoT device MQTT communication protocol test method, the first judgment comprises:

[0017] comparing the test process duration with an expected execution duration corresponding to the test case to obtain a first comparison result;

[0018] comparing a test process output value with an expected output value corresponding to the test case to obtain a second comparison result;

[0019] comparing a test process environment condition with an expected test environment condition corresponding to the test case to obtain a third comparison result;

[0020] obtaining a first abnormality evaluation index by using the first comparison result, the second comparison result and the third comparison result.

[0021] As a preferred scheme of the substation IoT device MQTT communication protocol test method, the establishment of the first device model according to the first parameter of the target substation IoT device comprises:

[0022] the first device model is configured with a unified MQTT protocol configuration;

[0023] the first parameter at least comprises a device name, a device type and an IP address;

[0024] the MQTT protocol configuration comprises parameter configurations of connection timeout, disconnection reconnection and message publishing frequency.

[0025] As a preferred scheme of the substation IoT device MQTT communication protocol test method, the first test set at least comprises a test scene, a test case and a test instruction.

[0026] The test scenarios each include at least predefined events and conditions;

[0027] The test cases include test names, test instructions and test data;

[0028] The test instructions are communication behaviors during testing, and at least include message types, message contents, sending frequencies and sending periods.

[0029] In a second aspect, the present application provides a substation IoT device MQTT communication protocol test method and system, comprising:

[0030] A device information management unit configured with a first device model establishment function;

[0031] A test information management unit configured with a first test set establishment function;

[0032] A protocol test unit configured with a function of testing the first device model in combination with the first test set under the first test environment;

[0033] A report generation unit configured with a test report generation function.

[0034] As a preferred scheme of the substation IoT device MQTT communication protocol test system, the system further comprises a user management unit;

[0035] The user management unit is configured to grant different operation permissions to users and provide an interactive interface for the users, so that the users can enter the device information management unit, the test information management unit, the protocol test unit and the report generation unit through the interactive interface to perform management operations.

[0036] As a preferred scheme of the substation IoT device MQTT communication protocol test system, the user management unit comprises an operation permission module and an interactive module:

[0037] The operation permission module is configured to grant different operation permissions to users according to user information, wherein the user information includes names, contact numbers and positions, and the operation permissions are divided into first-level, second-level and third-level, and specifically:

[0038] The first-level operation permission is granted to the user, and the user only performs management operations on the report generation unit;

[0039] The second-level operation permission is granted to the user, and the user performs management operations on the device information management unit, the test information management unit and the report generation unit;

[0040] Granting a third-level operation permission: the user manages and operates the device information management unit, the test information management unit, the protocol test unit and the report generation unit;

[0041] An interaction module is configured to provide an interaction interface for the user to perform the management operation corresponding to the operation permission and record each operation performed by the user.

[0042] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described above when executing the computer program.

[0043] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the method described above.

[0044] Compared with the prior art, the present application has the following beneficial effects: the present application provides a substation IoT device MQTT communication protocol test method and system, a plurality of first device models are established according to the first parameters of the target substation IoT device, and the first device models are configured with the same communication protocol; a first test set is preset, and the first test set at least includes a test scene, a test case and a test instruction; a first test environment based on the first test set is established, and the first device model is tested in the first test environment combined with the first test set; a test report is generated according to the test result, and the substation IoT device test is completed. The test process can be automatically executed, the test efficiency is significantly improved, the influence of human factors on the test result is reduced, and the objectivity and accuracy of the test result are ensured. By establishing the device model and the test set, various actual operation scenes can be simulated, and the substation IoT device can be comprehensively tested, so as to ensure the stability and reliability of the device in actual application. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. Among them:

[0046] Figure 1 A method flow chart of a substation IoT device MQTT communication protocol test method and system provided by an embodiment of the present application;

[0047] Figure 2 A system structure diagram of a substation IoT device MQTT communication protocol test method and system provided by an embodiment of the present application;

[0048] Figure 3 The internal structure diagram of the computer equipment of the substation Internet of Things device MQTT communication protocol test method and system provided for an embodiment of the present application is shown. DETAILED DESCRIPTION

[0049] In order to make the above objectives, characteristics and advantages of the present application more apparent, obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should fall within the scope of protection of the present application.

[0050] Embodiment 1

[0051] Reference Figures 1-3 For the first embodiment of the present application, the embodiment provides a substation Internet of Things device MQTT communication protocol test method and system, which comprises:

[0052] In the prior art, there are some problems, such as low efficiency of manual testing, test results are easily affected by human factors, etc.

[0053] The present application provides a method that can effectively solve the above-mentioned problems. Next, how to implement the substation Internet of Things device MQTT communication protocol test method will be described in detail in combination with multiple embodiments.

[0054] Figure 1 A method flowchart of a substation Internet of Things device MQTT communication protocol test method and system is shown, which comprises:

[0055] S101, a plurality of first device models are established according to the first parameters of the target substation Internet of Things device, and the first device models are all configured with the same communication protocol;

[0056] In an optional embodiment, different communication protocols can be selected according to different test purposes, such as MQTT communication protocol, HTTP communication protocol, etc., to adapt to different test requirements;

[0057] In the present embodiment, the MQTT communication protocol is tested. The MQTT communication protocol is a lightweight message transmission protocol, which uses the publish / subscribe mode and is very suitable for environments with limited bandwidth and high network delay, such as the communication of substation Internet of Things devices. These characteristics of the MQTT protocol make it widely used in the power industry.

[0058] In an optional embodiment, the first device model can be a virtual device model or an actual physical device model. The virtual device model is usually used to simulate the behavior of real devices, while the physical device model directly uses real substation IoT devices for testing.

[0059] In an optional embodiment, the first device model can be one or more devices in the target substation IoT device, for example, it can be a smart meter, relay, circuit breaker, etc. in the substation. By establishing the model of these devices, their behavior in actual work can be simulated, so as to test the communication protocol of the entire substation IoT system.

[0060] In an optional embodiment, the first parameter can set different parameter acquisition standards according to the selected device, for example, in this application, the target substation IoT device is considered, and the first parameter setting includes device name, device type and IP address, but is not limited thereto.

[0061] In the embodiment of the present application, a plurality of first device models are established according to the first parameters of the target substation IoT device, including:

[0062] The first device models are all configured with the same MQTT protocol configuration;

[0063] The first parameter at least includes device name, device type and IP address;

[0064] The MQTT protocol configuration includes parameter configuration of connection timeout, disconnection reconnection and message publishing frequency.

[0065] In an optional embodiment, the number of first device models depends on the number of selected target substation IoT devices and the detailed level of testing. For example, if the test needs to cover a large number of devices and complex scenarios, more device models may need to be established to ensure the comprehensiveness of the test.

[0066] It should be noted that the plurality of first device models are established according to the first parameters of the target substation IoT device, and the first device models are all configured with the same communication protocol, which can ensure the consistency of the test environment, so that the test result is more reliable and repeatable. In addition, by uniformly configuring the same communication protocol, a real network communication environment can be simulated, so as to comprehensively evaluate the communication performance of the substation IoT device.

[0067] S102, a first test set is preset, and the first test set at least includes a test scene, a test case and a test instruction;

[0068] In an optional embodiment, the first test set includes at least test scenarios, test cases, and test instructions to simulate various communication scenarios or specific operations after receiving various communication instructions during the operation of the substation. However, the first test set is not limited thereto.

[0069] In an optional embodiment, the first test set can further include test environment configuration information such as network delay, bandwidth limitation, etc. to ensure that the test environment is closer to the actual application environment.

[0070] In the embodiments of the present application, the first test set includes at least test scenarios, test cases, and test instructions.

[0071] Each test scenario includes at least predefined events and conditions.

[0072] The test case includes a test name, test instructions, and test data.

[0073] The test instructions are communication behaviors during testing, including at least message type, message content, sending frequency, and sending period.

[0074] In an optional embodiment, the test scenario is a series of predefined events and conditions that simulate various situations that the substation IoT device may encounter during actual operation. For example, a test scenario can simulate how the substation device responds to an emergency event under adverse weather conditions.

[0075] In an optional embodiment, the test case is a series of detailed steps designed for a specific test scenario, including a test name, test instructions, and test data. The test instructions define the communication behaviors that the device needs to perform during testing, such as message type, message content, sending frequency, and sending period.

[0076] In an optional embodiment, the test instructions are communication behaviors during testing, including at least message type, message content, sending frequency, and sending period. The message type can be subscription, publication, confirmation, etc., and the message content contains corresponding data or commands according to the test requirements. The sending frequency and sending period define the frequency and periodicity of message sending to simulate the communication mode in actual applications.

[0077] It should be noted that the first test set is preset, and the first test set at least includes test scenarios, test cases and test instructions to ensure that the test covers various situations that the substation Internet of Things device may encounter in actual operation, thereby improving the comprehensiveness and accuracy of the test. Through the preset test scenarios and cases, the behavior of the device in a specific environment can be simulated, and it is ensured that the test results can truly reflect the performance of the device in actual application. In addition, the detailed definition of the test instructions helps to standardize the test process, reduces the difference of human operation, and ensures the objectivity and repeatability of the test.

[0078] S103, a first test environment based on the first test set is established, and the first device model is tested in combination with the first test set in the first test environment;

[0079] In the embodiment of the present application, testing the first device model in combination with the first test set in the first test environment includes:

[0080] Testing the first device model in combination with the first test set in the first test environment;

[0081] Firstly, the test process is judged;

[0082] According to the first judgment result, a test report is generated.

[0083] In an optional embodiment, the first judgment is mainly to judge the test process, whether the test process is executed according to the test case, if not, the test process has an abnormal situation, and the warning information is sent to the management personnel in the form of short message.

[0084] In an optional embodiment, the first judgment can be compared with the expected result to determine whether the test is successful. If the test result is consistent with the expected result, it is considered that the test is successful; if it is not consistent, it is considered that the test fails, and the reason for the failure needs to be further analyzed.

[0085] In an optional embodiment, the first judgment can also monitor various parameters in the test process, such as message transmission delay, packet loss rate, etc., to evaluate the performance and stability of the communication protocol.

[0086] In the embodiment of the present application, the first judgment includes:

[0087] The test process duration is compared with the expected execution duration corresponding to the test case to obtain a first comparison result;

[0088] The test process output value is compared with the expected output value corresponding to the test case to obtain a second comparison result;

[0089] comparing the test process environment condition with the expected test environment condition corresponding to the test case, to obtain a third comparison result;

[0090] obtaining a first abnormality evaluation index by using the first comparison result, the second comparison result and the third comparison result.

[0091] For example, the first abnormality evaluation index is obtained by the following formula:

[0092]

[0093] wherein, E 10 represents the first abnormality evaluation index; R 01 represents the difference between the test duration and the expected execution duration corresponding to the test case; R 02 represents the difference between the test output value and the expected output value corresponding to the test case; R 03w represents the difference between the humidity value in the test environment condition and the humidity value in the test environment condition corresponding to the test case; R 03t represents the difference between the temperature value in the test environment condition and the temperature value in the test environment condition corresponding to the test case; R 01x represents the maximum allowed difference between the preset test duration and the expected execution duration corresponding to the test case; R 02x represents the maximum allowed difference between the preset test output value and the expected output value corresponding to the test case; R 03a represents the maximum allowed difference between the humidity value in the test environment condition and the humidity value in the test environment condition corresponding to the test case; R 03b represents the maximum allowed difference between the temperature value in the test environment condition and the temperature value in the test environment condition corresponding to the test case;

[0094] determining whether to perform retesting or abnormality alarm by using the first abnormality evaluation index.

[0095] determining whether to perform retesting or abnormality alarm by using the first abnormality evaluation index, comprising:

[0096] comparing the first abnormality evaluation index with a preset evaluation index threshold;

[0097] when the first abnormality evaluation index is higher than the preset evaluation index threshold, determining to directly perform abnormality alarm;

[0098] when the first abnormality evaluation index is not higher than the preset evaluation index threshold, determining to call a new test case to perform retesting, to obtain a retesting test result;

[0099] If the retest result indicates that the retest is not performed according to the test case, the specific data information not performed according to the test case is called, wherein the specific data information not performed according to the test case includes a test duration, a test output value and a test environment condition;

[0100] The test duration is compared with an expected execution duration corresponding to the test case to obtain a first comparison result;

[0101] The test output value is compared with an expected output value corresponding to the test case to obtain a second comparison result;

[0102] The test environment condition is compared with an expected test environment condition corresponding to the test case to obtain a third comparison result; wherein the test environment condition includes an environment temperature and an environment humidity;

[0103] An abnormality evaluation index is obtained by using the first comparison result, the second comparison result and the third comparison result, wherein the second abnormality evaluation index is obtained by the following formula:

[0104]

[0105] wherein, E 20 represents the second abnormality evaluation index; R 01 represents a difference value between the test duration and the expected execution duration corresponding to the test case; R 02 represents a difference value between the test output value and the expected output value corresponding to the test case; R 03w represents a difference value between the humidity value in the test environment condition and the humidity value in the test environment condition corresponding to the test case; R 03t represents a difference value between the temperature value in the test environment condition and the temperature value in the test environment condition corresponding to the test case; R 01x represents a maximum allowed difference value between the preset test duration and the expected execution duration corresponding to the test case; R 02x represents a maximum allowed difference value between the preset test output value and the expected output value corresponding to the test case; R 03a represents a maximum allowed difference value between the humidity value in the test environment condition and the humidity value in the test environment condition corresponding to the test case; R 03b represents a maximum allowed difference value between the temperature value in the test environment condition and the temperature value in the test environment condition corresponding to the test case; E r represents a basic index value, and the basic index value is obtained by the following formula:

[0106]

[0107] wherein, E r represents the basic index value; R 01represents the difference between the test duration and the expected execution duration corresponding to the test case; R 02 represents the difference between the test output value and the expected output value corresponding to the test case; R 03w represents the difference between the humidity value in the test environment condition and the humidity value in the test environment condition corresponding to the test case; R 03t represents the difference between the temperature value in the test environment condition and the temperature value in the test environment condition corresponding to the test case;

[0108] In an optional embodiment, the second abnormality evaluation index is compared with a preset first abnormality evaluation index;

[0109] In an optional embodiment, when the second abnormality evaluation index is lower than the adjusted evaluation index threshold, only risk marking is performed without abnormality warning;

[0110] In an optional embodiment, when the second abnormality evaluation index is not lower than the adjusted evaluation index threshold, abnormality warning is performed;

[0111] wherein the adjusted evaluation index threshold is obtained by the following formula:

[0112]

[0113] wherein E yt represents the adjusted evaluation index threshold; E y represents the preset evaluation index threshold; E 10 represents the first abnormality evaluation index.

[0114] It should be noted that establishing the first test environment based on the first test set and testing the first device model in combination with the first test set in the first test environment can ensure the standardization and automation of the testing process, thereby greatly improving the testing efficiency and accuracy. Through the preset test environment and test set, various situations that the power substation Internet of Things device may encounter in actual operation can be simulated, ensuring that all communication scenarios that the device may encounter in actual application are covered. In addition, through automated testing, differences in human operation can be reduced, ensuring the consistency and repeatability of test results.

[0115] S104, generating a test report according to the test results, completing the testing of the power substation Internet of Things device.

[0116] In an optional embodiment, the test report can record various parameters and results in the testing process in detail, including but not limited to configuration information of the test environment, execution of the test case, abnormal events in the testing process, and final test conclusion. The test report not only provides detailed test process records for the tester, but also provides important data support for subsequent analysis and improvement.

[0117] For example, the generation process of the test report includes the organization and analysis of test data, ensuring that all test cases are executed and the test results meet expectations. The test report should also include a detailed description of any abnormalities or errors that occur during testing, as well as an analysis of these abnormalities or errors and possible solutions.

[0118] When generating the test report, automated tools can be used to collect and organize test data to improve the efficiency and accuracy of report generation. Automated tools can automatically extract key information from test logs, such as the execution status of test cases, performance indicators during testing, and the time point and detailed information of any abnormal events.

[0119] In addition, the test report should provide a clear view of the test coverage and statistical information of the test results, such as pass rate, failure rate, average response time, etc., to facilitate the quick evaluation of the overall quality of the test by the test personnel and project managers.

[0120] After the test report is completed, the report can be submitted to relevant stakeholders, including the development team, quality assurance team, and project managers. The submission of the report can be in electronic form, such as email attachments or online reporting platforms, or in paper form, depending on the specific needs and preferences of the organization.

[0121] Finally, the purpose of the test report is to ensure that the communication protocol of the substation Internet of Things device meets the design requirements and can work reliably in actual application. Through detailed test reports, strong technical support can be provided for the deployment and maintenance of the device, ensuring the stable operation of the substation Internet of Things system.

[0122] In summary, the present application proposes a substation Internet of Things device MQTT communication protocol test method, a plurality of first device models are established according to the first parameters of the target substation Internet of Things device, and the first device models are configured with the same communication protocol; a first test set is preset, the first test set at least includes test scenarios, test cases and test instructions; a first test environment based on the first test set is established, and the first device model is tested in the first test environment combined with the first test set; a test report is generated according to the test result, and the test of the substation Internet of Things device is completed. The test process can be automatically executed, which significantly improves the test efficiency, reduces the influence of human factors on the test results, and ensures the objectivity and accuracy of the test results. By establishing device models and test sets, various actual operation scenarios can be simulated to comprehensively test the substation Internet of Things device, thereby ensuring the stability and reliability of the device in actual application.

[0123] Example 2

[0124] The embodiment also provides a substation Internet of Things device MQTT communication protocol test system, characterized in that comprising:

[0125] a device information management unit configured to establish a first device model;

[0126] a test information management unit configured to establish a first test set;

[0127] a protocol test unit configured to test the first device model in combination with the first test set in a first test environment;

[0128] a report generation unit configured to generate a test report.

[0129] The user management unit is used to grant different operation permissions to users and provide an interactive interface for the users, and the users enter the device information management unit, the test information management unit, the protocol test unit and the report generation unit to perform management operations through the interactive interface.

[0130] The user management unit comprises an operation permission module and an interactive module.

[0131] The operation permission module is used for a manager to grant different operation permissions to users according to user information, wherein the user information comprises a name, a contact number and a position, and the operation permissions are divided into first-level, second-level and third-level, and specifically are:

[0132] the first-level operation permission is granted to the user to perform management operations on the report generation unit only;

[0133] the second-level operation permission is granted to the user to perform management operations on the device information management unit, the test information management unit and the report generation unit;

[0134] the third-level operation permission is granted to the user to perform management operations on the device information management unit, the test information management unit, the protocol test unit and the report generation unit;

[0135] The interactive module is used to provide an interactive interface for the user, and the user performs management operations corresponding to the operation permissions through the interactive interface, and records each operation performed by the user.

[0136] Each unit module can be embedded in or independent of a processor in a computer device in a hardware form, or can be stored in a memory in the computer device in a software form, so as to be called and executed by the processor to perform operations corresponding to each module.

[0137] The embodiment also provides a computer device, which can be a terminal, and an internal structure diagram of the computer device can be as shown in Figure 3The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is configured to communicate with an external terminal in a wired or wireless manner. The wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by the processor to implement a substation Internet of Things device MQTT communication protocol testing method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0138] The embodiment also provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the following steps:

[0139] A plurality of first device models are established according to the first parameters of the target substation Internet of Things device, and the first device models are configured with the same communication protocol;

[0140] A first test set is preset, and the first test set at least includes a test scene, a test case and a test instruction;

[0141] A first test environment based on the first test set is established, and the first device models are tested in the first test environment in combination with the first test set;

[0142] A test report is generated according to the test result, and the substation Internet of Things device testing is completed.

[0143] Embodiment 3

[0144] In a preferred embodiment, a substation Internet of Things device MQTT communication protocol testing system is provided, which includes:

[0145] A device information management unit is configured to:

[0146] Store all tested substation Internet of Things device models, record the device information of each substation Internet of Things device model, and set a unified MQTT protocol configuration for each substation Internet of Things device model, wherein the device information includes a device name, a device type and an IP address, and the MQTT protocol configuration includes parameter configurations of a connection timeout, a disconnection reconnection and a message publishing frequency;

[0147] The test information management unit is configured to:

[0148] create and manage test cases, and store a plurality of test scenarios, each of which contains predefined events and conditions, wherein the test case comprises a test name, test instructions and test data, the test instructions being communication behaviors during testing, including message type, message content, sending frequency and sending period;

[0149] The protocol test unit is configured to:

[0150] select substation IoT device models, test cases and test scenarios required for protocol testing, and build a virtual IoT environment according to the selected test scenarios, load the substation IoT device models into the built virtual IoT environment, execute the test process according to the selected test cases, and monitor the test process;

[0151] The report generation unit is configured to:

[0152] record the test process and test results, and generate a detailed report for each round of testing according to the test process and test results;

[0153] The user management unit is configured to:

[0154] grant different operation permissions to users, and provide an interactive interface for users to enter the device information management unit, the test information management unit, the protocol test unit and the report generation unit corresponding to the operation permissions for management operations.

[0155] The technical effects of the above are that the device information management unit can provide substation IoT device models required for testing, so as to be selected and loaded into testing by the user, and the test information management unit can provide test cases and test scenarios required for testing, so as to be run according to specified rules during testing. Before testing, the substation IoT device model to be tested is selected, and then appropriate test cases and test scenarios are selected according to the substation IoT device model, so that a virtual IoT environment can be built through the test scenarios, and the substation IoT device model is loaded into the virtual IoT environment, and then the test process can be executed according to the test cases in the virtual IoT environment, so that the communication behaviors of the substation IoT device can be tested, and at the same time, the test process is monitored in real time. When an abnormal situation occurs during the test process, a warning message is sent, and the report generation unit can generate a test report according to the test process and test results, so as to facilitate the user to analyze and summarize later.

[0156] Embodiment 4

[0157] According to the substation Internet of Things device MQTT communication protocol test system provided in Embodiment 3, the device information management unit includes a device model library, and the device model library is used to store all substation Internet of Things device models to be tested, and the device model library stores the substation Internet of Things device models based on device information.

[0158] In the embodiments of the present application, the device model library includes:

[0159] The model management module is used to:

[0160] The management function provided for the user is used to manage the substation Internet of Things device models stored in the device model library, and the management function includes adding, editing, and deleting, and specifically includes:

[0161] The adding function is used to add a new substation Internet of Things device model in the device model library by the user;

[0162] The editing function is used to edit the substation Internet of Things device models already stored in the device model library, including editing the device information;

[0163] The deleting function is used to delete the substation Internet of Things device models already stored in the device model library.

[0164] The test information management unit includes a test case library, which is used to:

[0165] create and manage test cases, and the test cases include test names, test descriptions, and test data; and the test case library includes:

[0166] In the embodiments of the present application, the case management module is used to:

[0167] The case management function provided for the user is used to manage the test cases stored in the test case library, and specifically includes:

[0168] The new test case is added, including a test name, a test instruction, and test data; the existing test case is edited, including modifying the test name, the test instruction, and the test data;

[0169] The test case is deleted;

[0170] The test cases are classified according to device models, test scenes, and difficulty factors;

[0171] In the embodiments of the present application, the scene library is used to:

[0172] A plurality of test scenes are stored, and events and conditions of each test scene are predefined, including defining network topology, sensor parameters, types, quantities, positions of device models, and events and behaviors of device models occurring under each scene;

[0173] The scene library includes a scene management module configured to:

[0174] In an optional embodiment, a user is provided with a scene management function, and the user manages the test scenes stored in the scene library through the provided scene management function, specifically:

[0175] In an optional embodiment, network topology management is performed to add or edit network topology, including node names, connection relationships, and line parameters.

[0176] In an optional embodiment, sensor configuration is performed to add or edit various sensors, including sensor names, types, measurement ranges, and accuracies.

[0177] In an optional embodiment, device state setting is performed to set default states, running states, and fault states for each device model. The protocol test unit includes:

[0178] In an embodiment of the present application, the test preparation module is configured to:

[0179] Selecting a substation IoT device model to be tested, and selecting a test case and a test scene according to the substation IoT device model.

[0180] In an embodiment of the present application, the simulation test module is configured to:

[0181] Building a virtual IoT environment according to the selected test scene.

[0182] Loading the substation IoT device model to be tested into the built virtual IoT environment.

[0183] In an embodiment of the present application, after the above operations are completed, the test process is executed according to the test instructions in the test case; the test monitoring module is configured to:

[0184] Real-time monitoring of the test process, and comparing the monitored test process with the test case. The test monitoring module includes:

[0185] In an embodiment of the present application, the abnormality warning module is configured to:

[0186] According to the comparison result of the test monitoring module, it is determined whether the test process is executed according to the test case, and if not, the test process has an abnormal situation, and a warning message is sent to the administrator through a short message.

[0187] Specifically, if the test case is not executed, the test process is in an abnormal situation, including:

[0188] In an optional embodiment, if the test case is not executed, specific data information of the test case is not executed, wherein the specific data information of the test case not executed includes test duration, test output value and test environment condition;

[0189] In an optional embodiment, the test duration is compared with the expected execution duration corresponding to the test case to obtain a first comparison result;

[0190] In an optional embodiment, the test output value is compared with the expected output value corresponding to the test case to obtain a second comparison result;

[0191] In an optional embodiment, the test environment condition is compared with the expected test environment condition corresponding to the test case to obtain a third comparison result; wherein the test environment condition includes environmental temperature and environmental humidity;

[0192] In an optional embodiment, the first comparison result, the second comparison result and the third comparison result are used to obtain a first abnormal evaluation index, wherein the first abnormal evaluation index is obtained by the following formula:

[0193]

[0194] Wherein, E 10 represents the first abnormal evaluation index; R 01 represents the difference between the test duration and the expected execution duration corresponding to the test case; R 02 represents the difference between the test output value and the expected output value corresponding to the test case; R 03w represents the difference between the humidity value in the test environment condition and the humidity value in the test environment condition corresponding to the test case; R 03t represents the difference between the temperature value in the test environment condition and the temperature value in the test environment condition corresponding to the test case; R 01x represents the maximum allowed difference between the preset test duration and the expected execution duration corresponding to the test case; R 02x represents the maximum allowed difference between the preset test output value and the expected output value corresponding to the test case; R 03a represents the maximum allowed difference between the humidity value in the test environment condition and the humidity value in the test environment condition corresponding to the test case; R 03b represents the maximum allowed difference between the temperature value in the test environment condition and the temperature value in the test environment condition corresponding to the test case;

[0195] The first abnormality evaluation index is used to determine whether to retest or to alarm abnormally.

[0196] The technical effects of the above technical solution are: by comparing the test duration, test output value and test environment conditions (including environmental temperature and humidity) in the actual test process with the corresponding expected values in the test case, the presence or absence of abnormalities in the test process can be accurately identified. This comparison is not limited to output values, but also covers test duration and test environment conditions, providing a more comprehensive abnormality detection means. By calculating the first abnormality evaluation index (E 10 ), the technical solution can quantify the severity of the abnormality in the test process. This index takes into account the differences between the test duration, test output value and test environment conditions and the expected values, as well as the proportional relationship between these differences and the preset maximum allowable difference. This quantitative method makes the evaluation of test abnormalities more objective and accurate. According to the value of the first abnormality evaluation index, it can be determined whether to retest or trigger an abnormal alarm. This decision mechanism helps to respond to abnormal situations in the test process in a timely manner, avoiding potential quality problems. If the abnormality evaluation index is high, it indicates that there is a large deviation between the test process and the expected value, and retesting may be necessary to ensure product quality; if the abnormality evaluation index is low but still exceeds the acceptable range, an abnormal alarm can be triggered to further analyze and solve the problem. Through the process of automated comparison and calculation, the technical solution can significantly improve test efficiency and accuracy. It reduces the subjectivity and uncertainty of manual judgment, making the test process more standardized and repeatable. At the same time, by identifying and responding to test abnormalities in a timely manner, it helps to identify and solve problems early in product development, thereby reducing the cost of later repairs. The technical solution records specific data information (such as test duration, test output value and test environment conditions) that deviates from the expected value in the test process, as well as the comparison results between these data information and the expected value. These information provides traceability for the test process, which helps to find the root cause of the problem in subsequent analysis and improvement.

[0197] In summary, the technical solution provides strong support for product quality assurance through precise identification of test abnormalities, quantification of abnormality degree, guidance of subsequent actions, improvement of test efficiency and accuracy, and enhancement of traceability of the test process.

[0198] Specifically, the first abnormality evaluation index is used to determine whether to retest or to alarm abnormally, including:

[0199] Comparing the first abnormality evaluation index with a preset evaluation index threshold;

[0200] When the first abnormality evaluation index is higher than the preset evaluation index threshold, it is determined to directly alarm abnormally;

[0201] When the first abnormality evaluation index is not higher than a preset evaluation index threshold, it is determined to call a new test case for retesting, and a retesting test result is obtained;

[0202] When the retesting test result indicates that the retesting is not performed according to the test case, specific data information of the retesting not performed according to the test case is called, wherein the specific data information of the retesting not performed according to the test case includes a test duration, a test output value and a test environment condition;

[0203] The test duration is compared with an expected execution duration corresponding to the test case, and a first comparison result is obtained;

[0204] The test output value is compared with an expected output value corresponding to the test case, and a second comparison result is obtained;

[0205] The test environment condition is compared with an expected test environment condition corresponding to the test case, and a third comparison result is obtained; wherein the test environment condition includes an environment temperature and an environment humidity;

[0206] The first comparison result, the second comparison result and the third comparison result are used to obtain an abnormality evaluation index, wherein the second abnormality evaluation index is obtained by the following formula:

[0207]

[0208] wherein, E 20 represents the second abnormality evaluation index; R 01 represents a difference value between the test duration and the expected execution duration corresponding to the test case; R 02 represents a difference value between the test output value and the expected output value corresponding to the test case; R 03w represents a difference value between a humidity value in the test environment condition and a humidity value in the expected test environment condition corresponding to the test case; R 03t represents a difference value between a temperature value in the test environment condition and a temperature value in the expected test environment condition corresponding to the test case; R 01x represents a maximum allowed difference value between the preset test duration and the expected execution duration corresponding to the test case; R 02x represents a maximum allowed difference value between the preset test output value and the expected output value corresponding to the test case; R 03a represents a maximum allowed difference value between the humidity value in the test environment condition and the humidity value in the expected test environment condition corresponding to the test case; R 03b represents a maximum allowed difference value between the temperature value in the test environment condition and the temperature value in the expected test environment condition corresponding to the test case; E r represents a basic index value, and the basic index value is obtained by the following formula:

[0209]

[0210] wherein, E r represents the base index value; R 01 represents the difference between the test duration and the expected execution duration corresponding to the test case; R 02 represents the difference between the test output value and the expected output value corresponding to the test case; R 03w represents the difference between the humidity value in the test environment condition and the humidity value in the test environment condition corresponding to the test case; R 03t represents the difference between the temperature value in the test environment condition and the temperature value in the test environment condition corresponding to the test case;

[0211] In an optional embodiment, the second abnormality evaluation index is compared with a preset first abnormality evaluation index;

[0212] In an optional embodiment, when the second abnormality evaluation index is lower than the adjusted evaluation index threshold, only risk marking is performed without abnormality warning;

[0213] In an optional embodiment, when the second abnormality evaluation index is not lower than the adjusted evaluation index threshold, abnormality warning is performed;

[0214] wherein, the adjusted evaluation index threshold is obtained by the following formula:

[0215]

[0216] wherein, E yt represents the adjusted evaluation index threshold; E y represents the preset evaluation index threshold; E 10 represents the first abnormality evaluation index.

[0217] The technical effects of the above technical solution are: by introducing the comparison between the first abnormality evaluation index and the preset evaluation index threshold, the technical solution can flexibly decide whether to perform abnormality alarm or retest according to the severity of the abnormality. This decision mechanism helps to avoid unnecessary repeated testing while ensuring timely response to serious abnormalities. When the first abnormality evaluation index is not higher than the preset threshold, retesting is selected instead of direct alarm, which helps to verify and solve the problem through simple retesting when the problem is not too serious, thereby avoiding unnecessary alarms and potential resource waste. In the case where retesting is still not performed according to the test case, the technical solution further refines the abnormality processing strategy by calculating the second abnormality evaluation index and comparing it with the adjusted evaluation index threshold. This refinement helps to more accurately assess the abnormality after retesting and take appropriate measures (such as risk marking or abnormality warning) according to the assessment results. The calculation formula of the second abnormality evaluation index introduces the base index value (Er), which comprehensively considers the test duration, test output value, and difference between test environment conditions and expected values. This comprehensive consideration helps to improve the accuracy and objectivity of abnormality evaluation, thereby more accurately reflecting the actual situation in the testing process. Through the adjusted evaluation index threshold formula, the technical solution can dynamically adjust the threshold according to the value of the first abnormality evaluation index. This dynamic adjustment mechanism helps to better adapt to abnormal situations in different testing scenarios, improving the relevance and effectiveness of abnormality processing. The technical solution records specific data information that deviates from the expectation during the testing process and performs multiple comparisons and calculations. These records provide traceability for the testing process, which helps to find the root cause of problems in subsequent analysis and improvement. At the same time, by calculating the abnormality evaluation index and comparing the results, the abnormality in the testing process can be analyzed in depth, providing a basis for optimizing the testing strategy and improving product quality.

[0218] In summary, the technical solution provides strong support for product quality assurance by enhancing the flexibility of testing decisions, improving the efficiency of the testing process, introducing a second abnormality evaluation index to refine abnormality processing, improving the accuracy of abnormality evaluation, dynamically adjusting the evaluation index threshold, and enhancing the traceability and analyzability of the testing process.

[0219] In an optional embodiment, the report generation unit comprises:

[0220] The report generation module is configured to:

[0221] monitor and record the testing process and testing results of each round of protocol testing, and generate a testing report according to the obtained testing process and testing results;

[0222] The report management module is configured to:

[0223] The report management function is provided for the user to manage the generated test report through the provided report management function, specifically:

[0224] Report export: for exporting the test report;

[0225] Report preview: directly preview the report online by clicking the report link.

[0226] The technical effect of the above content is that the device information management unit can provide the substation Internet of Things device model required for testing, so as to be selected and loaded into the test by the user, and the test information management unit can provide the test case and test scene required for testing, so as to run according to the specified rules during testing. Then, the protocol test unit can realize the testing work of the substation Internet of Things device MQTT communication protocol. First, the test preparation module needs to be prepared before testing. Select the substation Internet of Things device model to be tested from the device model library, and then select the appropriate test case and test scene from the test case library and scene library, respectively, so that the test can be performed according to the rules specified in the test case and test scene during testing. After the preparation work is completed, the simulation test module performs the testing work. First, a virtual Internet of Things environment is built according to the test scene, then the substation Internet of Things device model is loaded into the virtual Internet of Things environment that has been built, and finally the test process is executed according to the test instructions in the test case, so that the communication behavior of the substation Internet of Things device can be tested, such as sending frequency and sending period, to test the performance indicators. Through testing, possible communication problems can be found and solved, reducing the occurrence of device failures, thereby improving the reliability and performance of the device. During testing, the test monitoring module monitors the test process in real time. By comparing the monitored test process with the test case, it is determined whether the test process is performed according to the test case. When there is an abnormal situation in the test process, the abnormality warning module sends warning information to the manager in the form of a short message to remind the manager or relevant personnel to pay attention and stop or temporarily test the work in a timely manner. After testing is completed, the report generation module can monitor and record the test process and test results of each round of protocol testing, and generate a test report corresponding to the test report after testing is completed, so that the user can analyze and summarize the test report in the later stage, thereby improving the possible communication problems in the test report to ensure the normal operation of the communication protocol. Moreover, the report management module provides the user with the functions of export and preview. The user can export the test report, supporting multiple formats such as HTML, PDF, CSV, etc. Alternatively, the user can directly preview the report online by clicking the report link.

[0227] Embodiment 5

[0228] According to the substation Internet of Things device MQTT communication protocol test system provided in embodiment 3, the user management unit comprises:

[0229] The operation permission module is configured to:

[0230] The manager grants different operation permissions to the user according to the user information, wherein the user information includes name, contact number and position, and the operation permissions are divided into first-level, second-level and third-level, and specifically:

[0231] The first-level operation permission is granted: the user only performs management operations on the report generation unit;

[0232] The second-level operation permission is granted: the user performs management operations on the device information management unit, the test information management unit and the report generation unit;

[0233] The third-level operation permission is granted: the user performs management operations on the device information management unit, the test information management unit, the protocol test unit and the report generation unit;

[0234] The interaction module is configured to:

[0235] The user is provided with an interaction interface through which the user performs management operations corresponding to the operation permissions.

[0236] The technical effects of the above content are as follows: the user management unit can grant different operation permissions to the user through the operation permission module. When the user is granted the first-level operation permission, the user can only perform management operations on the report generation unit, i.e., export or browse test reports. When the user is granted the second-level operation permission, the user can perform management operations such as adding, editing and deleting on the device information management unit, perform management operations such as new creation, editing, deleting and classification on the test case library of the test information management unit, and perform management operations such as network topology management, sensor configuration and device state setting on the scene library, and can also perform management operations such as exporting or browsing on the report generation unit. When the user is granted the third-level operation permission, the user can perform all management operations of the first-level and the second-level, and can also perform protocol test work through the protocol test unit. By granting different operation permissions to the user, the safety and reliability of the system can be effectively guaranteed.

[0237] Working principle: the device information management unit can provide the substation Internet of Things device model required for testing, so as to be selected and loaded into the test by the user, and the test information management unit can provide the test case and test scene required for testing, so as to be run according to the specified rules when testing, before testing, the substation Internet of Things device model to be tested is selected, and then the appropriate test case and test scene are selected according to the substation Internet of Things device model, so as to build a virtual Internet of Things environment through the test scene, and load the substation Internet of Things device model into the virtual Internet of Things environment, and then in the virtual Internet of Things environment, the test process is executed according to the test case, so as to test the communication behavior of the substation Internet of Things device, through the test, the possible communication problems can be found and solved, the occurrence of device failure is reduced, and the reliability and performance of the device are improved, at the same time, real-time monitoring is performed during the test process, when an abnormal condition occurs during the test process, a warning information is sent, and a test report is generated according to the test process and the test result, so as to be analyzed and summarized by the user later, so that the possible communication problems in the test report are improved, and the normal operation of the communication protocol is ensured.

[0238] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not limited. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

[0239] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer usable program code. The solutions in the embodiments of the present application can be implemented in various computer languages, such as object-oriented programming language Java and interpreted scripting language JavaScript.

[0240] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0241] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0242] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0243] While the preferred embodiments of the application have been described, additional variations and modifications can be employed by those skilled in the art. Therefore, the appended claims intend to cover all such modifications and variations as fall within the true spirit and scope of the application.

[0244] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for testing the MQTT communication protocol of IoT devices in substations, characterized in that, include: Several first device models are established based on the first parameters of the IoT devices in the target substation, and all first device models are configured with the same communication protocol. The first parameter includes the device name, device type, and IP address; A first test set is preset, which includes at least test scenarios, test cases, and test instructions; The test scenario includes test environment conditions and test environment configuration information. The test environment conditions include ambient temperature and ambient humidity, and the test environment configuration information includes network latency and bandwidth limitations. The test cases include test name, test instructions and test data. The test instructions are the communication behaviors during the test, and the test instructions include at least message type, message content, sending frequency and sending period. Establish a first test environment based on the first test set, and test the first device model in the first test environment in conjunction with the first test set; The testing of the first device model in the first test environment using the first test set includes: The first device model was tested in the first test environment using the first test set; Make the first judgment on the testing process; A test report is generated based on the initial assessment result; The first judgment includes: Compare the test process duration with the expected execution duration of the corresponding test cases to obtain the first comparison result; The test process output value is compared with the expected output value corresponding to the test case to obtain a second comparison result; The test environment conditions are compared with the expected test environment conditions corresponding to the test cases to obtain a third comparison result; The first anomaly evaluation index is obtained using the first comparison result, the second comparison result, and the third comparison result.

2. The MQTT communication protocol testing method for substation IoT devices as described in claim 1, characterized in that, The step of establishing several first device models based on the first parameters of the target substation IoT devices includes: All of the first device models are configured with a unified MQTT protocol. The first parameter includes at least the device name, device type, and IP address; The MQTT protocol configuration includes parameter configurations for connection timeout, reconnection after disconnection, and message publishing frequency.

3. The MQTT communication protocol testing method for substation IoT devices as described in claim 2, characterized in that, The first test set includes at least test scenarios, test cases, and test instructions; Each of the test scenarios includes at least predefined events and conditions; The test cases include test names, test instructions, and test data; The test instructions refer to the communication behavior during the test, including at least the message type, message content, sending frequency, and sending period.

4. A substation IoT device MQTT communication protocol testing system applying the method described in claim 1, characterized in that, include: The device information management unit configured with the function of establishing the first device model; The test information management unit configured with the function of establishing the first test set; A protocol test unit configured to perform testing functions on a first device model in the first test environment in conjunction with a first test set; A report generation unit configured with test report generation functionality.

5. The MQTT communication protocol testing system for substation IoT devices as described in claim 4, characterized in that, It also includes a user management unit; The user management unit is used to grant users different operation permissions and provide users with an interactive interface. Users can access the device information management unit, test information management unit, protocol test unit and report generation unit corresponding to their operation permissions through the interactive interface to perform management operations.

6. The MQTT communication protocol testing system for substation IoT devices as described in claim 5, characterized in that, The user management unit includes an operation permission module and an interaction module: The operation permission module is used by administrators to grant users different operation permissions based on user information, including name, contact number, and job title. Operation permissions are divided into three levels: Level 1, Level 2, and Level 3, as follows: Grant Level 1 access: Users can only manage the report generation unit; Grant Level 2 operation permissions: Users can perform management operations on the device information management unit, test information management unit, and report generation unit; Grant Level 3 operating permissions: Users can manage and operate the device information management unit, test information management unit, protocol test unit, and report generation unit; The interaction module provides an interactive interface for users to perform management operations corresponding to their permissions through the interface, and records all user operations.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.

Images