Substation Internet of Things equipment MQTT communication protocol test method and system

Abstract

The invention discloses a substation Internet of Things equipment MQTT communication protocol test method and system. The method comprises the following steps: establishing a plurality of first equipment models according to a first parameter of target substation Internet of Things equipment, wherein the first equipment models are configured with the same communication protocol; presetting a first test set; establishing a first test environment based on the first test set, and testing the first equipment model in combination with the first test set in the first test environment; and generating a test report according to the test result, and completing the substation Internet of Things equipment test. The test process can be automatically executed, the test efficiency is remarkably improved, the influence of human factors on the test result is reduced, and the objectivity and accuracy of the test result are ensured. By establishing the equipment model and the test set, various actual operation scenes can be simulated, and the transformer substation Internet of Things equipment is comprehensively tested, so that the stability and reliability of the equipment in actual application are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to a method and system for testing the MQTT communication protocol of a substation Internet of Things device. Background Art

[0002] In order to ensure the normal operation and efficient collaboration between these devices, a stable and reliable communication protocol MQTT (Message Queue Telemetry Transport) needs to be established to ensure the normal operation and efficient collaboration between these devices. As a lightweight and real-time IoT communication protocol, it has been widely used in the power industry. However, the existing MQTT communication protocol test is generally a manual test method. Manual testing mainly relies on testers to simulate real scenarios. Testers need to verify various functions of the protocol and record problems and solutions encountered during the test. However, manual testing is relatively inefficient and test results are easily affected by the individual experience and technical level of testers. Therefore, it does not meet the existing needs. In this regard, we propose a MQTT communication protocol test system for substation IoT devices. Summary of the invention

[0003] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method and system for testing the MQTT communication protocol of substation Internet of Things devices, which can solve the problems mentioned in the background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for testing the MQTT communication protocol of a substation IoT device, comprising:

[0008] Establishing a plurality of first device models according to the first parameters of the target substation IoT device, wherein the first device models are all configured with the same communication protocol;

[0009] Preset a first test set, the first test set at least including a test scenario, a test case and a test instruction;

[0010] Establishing a first test environment based on the first test set, and testing the first device model in the first test environment in combination with the first test set;

[0011] Generate a test report based on the test results to complete the substation IoT equipment test.

[0012] As a preferred solution of the MQTT communication protocol testing method for substation IoT devices of the present invention, the testing of the first device model in combination with the first test set under the first test environment includes:

[0013] Testing the first device model in a first test environment in combination with a first test set;

[0014] Make the first judgment on the test process;

[0015] A test report is generated according to the first determination result.

[0016] As a preferred solution of the MQTT communication protocol testing method for substation IoT devices described in the present invention, the first judgment includes:

[0017] Compare the test process duration with the expected execution duration of the test case to obtain a first comparison result;

[0018] Compare the test process output value with the expected output value corresponding to the test case to obtain a second comparison result;

[0019] Comparing the test process environment condition with the expected test environment condition corresponding to the test case to obtain a third comparison result;

[0020] A first abnormality evaluation index is acquired using the first comparison result, the second comparison result and the third comparison result.

[0021] As a preferred solution of the MQTT communication protocol testing method for substation IoT devices of the present invention, wherein: the step of establishing a plurality of first device models according to the first parameters of the target substation IoT devices includes:

[0022] The first device models are all configured with a unified MQTT protocol configuration;

[0023] The first parameter includes at least a device name, a device type and an IP address;

[0024] The MQTT protocol configuration includes parameter configurations for connection timeout, disconnection reconnection, and message publishing frequency.

[0025] As a preferred solution of the MQTT communication protocol testing method for substation IoT devices described in the present invention, wherein: the first test set at least includes a test scenario, a test case and a test instruction;

[0026] The test scenarios all include at least predefined events and conditions;

[0027] The test case includes a test name, a test instruction and a test data;

[0028] The test instruction is a communication behavior during the test, which at least includes a message type, a message content, a sending frequency and a sending period.

[0029] In a second aspect, the present invention provides a method and system for testing the MQTT communication protocol of a substation IoT device, including:

[0030] A device information management unit configured with a first device model establishment function;

[0031] A test information management unit configured with a first test set establishment function;

[0032] A protocol testing unit configured to test the function of the first device model in combination with the first test set under the first test environment;

[0033] A report generation unit is configured with a test report generation function.

[0034] As a preferred solution of the MQTT communication protocol test system for substation Internet of Things equipment of the present invention, it further includes a user management unit;

[0035] The user management unit is used to grant users different operation permissions and provide users with an interactive interface. Users enter the device information management unit, test information management unit, protocol test unit and report generation unit corresponding to the operation permissions through the interactive interface to perform management operations.

[0036] As a preferred solution of the MQTT communication protocol test system for substation IoT devices described in the present invention, the user management unit includes an operation authority module and an interaction module:

[0037] The operation authority module is used by managers to grant users different operation authorities based on user information. User information includes name, contact number and position. Operation authorities are divided into level 1, level 2 and level 3, as follows:

[0038] Granting first-level operation authority: users can only perform management operations on the report generation unit;

[0039] Granting secondary operation authority: users can manage the equipment information management unit, test information management unit and report generation unit;

[0040] Grant three levels of operation authority: users can manage the equipment information management unit, test information management unit, protocol test unit and report generation unit;

[0041] The interactive module is used to provide an interactive interface for users, through which users can perform management operations corresponding to their operating permissions, and record various operations performed by users.

[0042] In a third aspect, the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned method when executing the computer program.

[0043] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the method described above when executed by a processor.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention proposes a method and system for testing the MQTT communication protocol of a substation IoT device, wherein a plurality of first device models are established according to the first parameters of the target substation IoT device, and the first device models are all configured with the same communication protocol; a first test set is preset, and the first test set at least includes a test scenario, a test case, and a test instruction; a first test environment based on the first test set is established, and the first device model is tested in the first test environment in combination with the first test set; a test report is generated according to the test results to complete the test of the substation IoT device. The test process can be automatically executed, the test efficiency can be significantly improved, the influence of human factors on the test results can be reduced, and the objectivity and accuracy of the test results can be ensured. By establishing device models and test sets, various actual operation scenarios can be simulated, and the substation IoT devices can be comprehensively tested, thereby ensuring the stability and reliability of the devices in actual applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0046] Figure 1 A method flow chart of a method and system for testing the MQTT communication protocol of a substation Internet of Things device provided by an embodiment of the present invention;

[0047] Figure 2 A system structure diagram of a substation IoT device MQTT communication protocol testing method and system provided by an embodiment of the present invention;

[0048] Figure 3 An internal structural diagram of a computer device of a substation Internet of Things device MQTT communication protocol testing method and system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0050] Example 1

[0051] Reference Figure 1-Figure 3 , which is the first embodiment of the present invention, and provides a method and system for testing the MQTT communication protocol of a substation IoT device, including:

[0052] There are some problems in the existing related technologies, such as low efficiency of manual testing and test results being easily affected by human factors.

[0053] This application provides a method that can effectively solve the above-mentioned problems. Next, we will combine multiple embodiments to explain in detail how to implement the MQTT communication protocol test method for the substation IoT device;

[0054] Figure 1 A method flow chart of a method and system for testing the MQTT communication protocol of a substation IoT device is shown, including:

[0055] S101, establishing a plurality of first device models according to first parameters of the target substation IoT device, wherein the first device models are all configured with the same communication protocol;

[0056] In an optional embodiment, different communication protocols may be selected according to different test purposes, such as the MQTT communication protocol, the HTTP communication protocol, etc., to meet different test requirements;

[0057] In the embodiment of the present application, the MQTT communication protocol is tested. The MQTT communication protocol is a lightweight message transmission protocol that uses a publish / subscribe model and is very suitable for environments with limited bandwidth and high network latency, such as the communication of IoT devices in substations. These characteristics of the MQTT protocol have made it widely used in the power industry.

[0058] In an optional embodiment, the first device model can be a virtual device model or an actual physical device model. The virtual device model is usually used to simulate the behavior of a real device, while the physical device model is directly tested using a real substation IoT device.

[0059] In an optional embodiment, the first device model may be one or more devices in the target substation IoT device, for example, a smart meter, relay, circuit breaker, etc. in the substation. By building models of these devices, their behaviors in actual work can be simulated, thereby testing the communication protocol of the entire substation IoT system.

[0060] In an optional embodiment, the first parameter can set different parameter acquisition standards according to the selected device. For example, in this application, the target substation IoT device is considered, and the accuracy of the communication protocol is tested. Therefore, the first parameter setting includes the device name, device type and IP address, but it is not limited.

[0061] In the embodiment of the present application, establishing a plurality of first device models according to the first parameters of the target substation IoT device includes:

[0062] First, all device models are configured with a unified MQTT protocol configuration;

[0063] The first parameter includes at least the device name, device type and IP address;

[0064] MQTT protocol configuration includes parameter configuration of connection timeout, disconnection reconnection and message publishing frequency.

[0065] In an optional embodiment, the number of first device models depends on the number of target substation IoT devices selected and the level of detail of the test. For example, if the test needs to cover a large number of devices and complex scenarios, more device models may need to be established to ensure the comprehensiveness of the test.

[0066] It should be noted that, according to the first parameters of the target substation IoT device, several first device models are established, and the first device models are all configured with the same communication protocol to ensure the consistency of the test environment, making the test results more reliable and repeatable. In addition, by uniformly configuring the same communication protocol, a real network communication environment can be simulated, thereby comprehensively evaluating the communication performance of the substation IoT device.

[0067] S102, presetting a first test set, where the first test set at least includes a test scenario, a test case, and a test instruction;

[0068] In an optional embodiment, the first test set is to simulate various communication scenarios generated during the operation of the substation, or specific operations after receiving various communication instructions. Therefore, the first test set at least includes test scenarios, test cases and test instructions, but the first test set is not limited.

[0069] In an optional embodiment, the first test set may also include test environment configuration information, such as network delay, bandwidth limitation, etc., to ensure that the test environment is closer to the actual application environment.

[0070] In an embodiment of the present application, the first test set includes at least a test scenario, a test case, and a test instruction;

[0071] All test scenarios include at least predefined events and conditions;

[0072] The test case includes the test name, test instructions and test data;

[0073] The test instruction is the communication behavior during the test, which at least includes the message type, message content, sending frequency and sending cycle.

[0074] In an optional embodiment, the test scenario is a predefined series of events and conditions that simulate various situations that the substation IoT devices may encounter in actual operation. For example, a test scenario may simulate how substation equipment responds to emergency events under severe weather conditions.

[0075] In an optional embodiment, a test case is a series of detailed steps designed for a specific test scenario, including a test name, test instructions, and test data. The test instructions define the communication behavior that the device needs to perform during the test, such as message type, message content, sending frequency, and sending period.

[0076] In an optional embodiment, the test instruction is a communication behavior during the test, including at least a message type, a message content, a sending frequency, and a sending period. The message type may be subscription, publication, confirmation, etc., and the message content may include corresponding data or commands according to the test requirements. The sending frequency and the sending period define the frequency and periodicity of message sending to simulate the communication mode in actual applications.

[0077] It should be noted that the preset first test set, which at least includes test scenarios, test cases and test instructions, can ensure that the test covers various situations that the substation IoT device may encounter in actual operation, thereby improving the comprehensiveness and accuracy of the test. Through the preset test scenarios and test cases, the behavior of the device in a specific environment can be simulated to ensure that the test results can truly reflect the performance of the device in actual applications. In addition, the detailed definition of the test instructions helps to standardize the test process, reduce differences in human operations, and ensure the objectivity and repeatability of the test.

[0078] S103, establishing a first test environment based on the first test set, and testing the first device model in the first test environment in combination with the first test set;

[0079] In an embodiment of the present application, testing the first device model in a first test environment in combination with a first test set includes:

[0080] Testing the first device model in a first test environment in combination with a first test set;

[0081] Make the first judgment on the test process;

[0082] A test report is generated according to the first determination result.

[0083] In an optional embodiment, the first judgment is mainly to judge the test process, to determine whether the test process is executed according to the test case. If it is not executed according to the test case, there is an abnormality in the test process, and an early warning message is sent to the management personnel via SMS.

[0084] In an optional embodiment, the first judgment can be made by comparing the test result with the expected result to determine whether the test is successful. If the test result is consistent with the expected result, the test is considered successful; if not, the test is considered failed and the cause of the failure needs to be further analyzed.

[0085] In an optional embodiment, the first judgment may also be performed by monitoring various parameters during the test, such as message transmission delay, packet loss rate, etc., to evaluate the performance and stability of the communication protocol.

[0086] In the embodiment of the present application, the first judgment includes:

[0087] Compare the test process duration with the expected execution duration of the test case to obtain a first comparison result;

[0088] Compare the test process output value with the expected output value corresponding to the test case to obtain a second comparison result;

[0089] Comparing the test process environment condition with the expected test environment condition corresponding to the test case to obtain a third comparison result;

[0090] A first abnormality evaluation index is acquired using the first comparison result, the second comparison result, and the third comparison result.

[0091] Exemplarily, the first abnormal evaluation index is obtained by the following formula:

[0092]

[0093] Among them, E 10 Represents the first abnormal evaluation index; R 01 It represents the difference between the test duration and the expected execution duration of the test case. 02 Represents the difference between the test output value and the expected output value corresponding to the test case; R 03w Represents the difference between the humidity value in the test environment conditions and the humidity value in the test environment conditions corresponding to the test case; R 03t Represents the difference between the temperature value in the test environment conditions and the temperature value in the test environment conditions corresponding to the test case; R 01x Indicates the maximum allowable difference between the preset test duration and the expected execution duration of the test case; R 02x Indicates the maximum allowable difference between the preset test output value and the expected output value corresponding to the test case; R 03a Indicates the maximum allowable difference between the humidity value in the test environment conditions and the humidity value in the test environment conditions corresponding to the test case; R 03b Indicates the maximum allowable difference between the temperature value in the test environment conditions and the temperature value in the test environment conditions corresponding to the test case;

[0094] The first abnormal evaluation index is used to determine whether to perform a retest or issue an abnormal alarm.

[0095] The first abnormal evaluation index is used to determine whether to retest or issue an abnormal alarm, including:

[0096] Comparing the first abnormal evaluation index with a preset evaluation index threshold;

[0097] When the first abnormal evaluation index is higher than the preset evaluation index threshold, it is determined to directly issue an abnormal alarm;

[0098] When the first abnormal evaluation index is not higher than the preset evaluation index threshold, it is determined to call a new test case for retesting and obtain the test result of the retesting;

[0099] If the test result of the retest indicates that the retest was not performed according to the test case, then the specific data information of the retest not being performed according to the test case is retrieved, wherein the specific data information of the retest not being performed according to the test case includes the test duration, the test output value, and the test environment condition;

[0100] Compare the test duration with the expected execution duration of the test case to obtain a first comparison result;

[0101] Compare the test output value with the expected output value corresponding to the test case to obtain a second comparison result;

[0102] Compare the test environment conditions with the expected test environment conditions corresponding to the test case to obtain a third comparison result; wherein the test environment conditions include ambient temperature and ambient humidity;

[0103] The abnormality evaluation index is obtained by using the first comparison result, the second comparison result and the third comparison result, wherein the second abnormality evaluation index is obtained by the following formula:

[0104]

[0105] Among them, E 20 Represents the second abnormal evaluation index; R 01 It represents the difference between the test duration and the expected execution duration of the test case. 02 Represents the difference between the test output value and the expected output value corresponding to the test case; R 03w Represents the difference between the humidity value in the test environment conditions and the humidity value in the test environment conditions corresponding to the test case; R 03t Represents the difference between the temperature value in the test environment conditions and the temperature value in the test environment conditions corresponding to the test case; R 01x Indicates the maximum allowable difference between the preset test duration and the expected execution duration of the test case; R 02x Indicates the maximum allowable difference between the preset test output value and the expected output value corresponding to the test case; R 03a Indicates the maximum allowable difference between the humidity value in the test environment conditions and the humidity value in the test environment conditions corresponding to the test case; R 03b Indicates the maximum allowable difference between the temperature value in the test environment conditions and the temperature value in the test environment conditions corresponding to the test case; E r Represents the basic index value, and the basic index value is obtained by the following formula:

[0106]

[0107] Among them, E r Indicates the basic index value; R 01It represents the difference between the test duration and the expected execution duration of the test case. 02 Represents the difference between the test output value and the expected output value corresponding to the test case; R 03w Represents the difference between the humidity value in the test environment conditions and the humidity value in the test environment conditions corresponding to the test case; R 03t Indicates the difference between the temperature value in the test environment conditions and the temperature value in the test environment conditions corresponding to the test case;

[0108] In an optional embodiment, the second abnormality evaluation index is compared with a preset first abnormality evaluation index;

[0109] In an optional embodiment, when the second abnormal evaluation index is lower than the adjusted evaluation index threshold, only risk marking is performed without abnormal warning;

[0110] In an optional embodiment, when the second abnormal evaluation index is not lower than the adjusted evaluation index threshold, an abnormal warning is issued;

[0111] Among them, the adjusted evaluation index threshold is obtained by the following formula:

[0112]

[0113] Among them, E yt Represents the adjusted evaluation index threshold; E y Indicates the preset evaluation index threshold; E 10 Represents the first abnormality evaluation index.

[0114] It should be noted that establishing a first test environment based on the first test set and testing the first device model in the first test environment in combination with the first test set can ensure the standardization and automation of the test process, thereby greatly improving the test efficiency and accuracy. Through the preset test environment and test set, various situations that the substation IoT device may encounter in actual operation can be simulated to ensure that the test covers all communication scenarios that the device may encounter in actual application. In addition, through automated testing, the differences in manual operations can be reduced, ensuring the consistency and repeatability of the test results.

[0115] S104, generating a test report based on the test results, and completing the substation IoT equipment test.

[0116] In an optional embodiment, the test report can record in detail the various parameters and results during the test process, including but not limited to the configuration information of the test environment, the execution of the test cases, the abnormal events during the test process, and the final test conclusion. The test report not only provides the tester with a detailed record of the test process, but also provides important data support for subsequent analysis and improvement.

[0117] Exemplarily, the test report generation process includes the collation and analysis of test data to ensure that all test cases are executed and the test results meet expectations. The test report should also include a detailed description of any anomalies or errors that occurred during the test, as well as an analysis of these anomalies or errors and possible solution suggestions.

[0118] When generating test reports, you can use automated tools to collect and organize test data to improve the efficiency and accuracy of report generation. Automated tools can automatically extract key information from test logs, such as the execution status of test cases, performance indicators during the test, and the time and details of any abnormal events.

[0119] In addition, the test report should provide a clear view of the test coverage and statistical information of the test results, such as pass rate, failure rate, average response time, etc., so that testers and project managers can quickly evaluate the overall quality of the test.

[0120] After the test report is completed, it can be submitted to relevant stakeholders, including the development team, quality assurance team, and project management. The report can be submitted electronically, such as an email attachment or online reporting platform, or in paper form, depending on the specific needs and preferences of the organization.

[0121] Ultimately, the purpose of the test report is to ensure that the communication protocol of the substation IoT device meets the design requirements and can work reliably in actual applications. Through detailed test reports, strong technical support can be provided for the deployment and maintenance of the equipment to ensure the stable operation of the substation IoT system.

[0122] In summary, the present invention proposes a method for testing the MQTT communication protocol of a substation IoT device. According to the first parameters of the target substation IoT device, several first device models are established, and the first device models are all configured with the same communication protocol; a first test set is preset, and the first test set at least includes a test scenario, a test case, and a test instruction; a first test environment based on the first test set is established, and the first device model is tested in the first test environment in combination with the first test set; a test report is generated according to the test results to complete the substation IoT device test. The test process can be automatically executed, the test efficiency can be significantly improved, the influence of human factors on the test results can be reduced, and the objectivity and accuracy of the test results can be ensured. By establishing device models and test sets, various actual operation scenarios can be simulated, and the substation IoT devices can be comprehensively tested, thereby ensuring the stability and reliability of the equipment in actual applications.

[0123] Example 2

[0124] This embodiment also provides a substation IoT device MQTT communication protocol test system, which is characterized by comprising:

[0125] A device information management unit configured with a first device model establishment function;

[0126] A test information management unit configured with a first test set establishment function;

[0127] A protocol testing unit configured to test the function of the first device model in combination with the first test set under the first test environment;

[0128] A report generation unit is configured with a test report generation function.

[0129] The user management unit is used to grant users different operation permissions and provide users with an interactive interface. Users enter the device information management unit, test information management unit, protocol test unit and report generation unit corresponding to the operation permissions through the interactive interface to perform management operations.

[0130] The user management unit includes an operation permission module and an interaction module:

[0131] The operation authority module is used by managers to grant users different operation authorities based on user information. User information includes name, contact number and position. Operation authorities are divided into level 1, level 2 and level 3, as follows:

[0132] Granting first-level operation authority: users can only perform management operations on the report generation unit;

[0133] Granting secondary operation authority: users can manage the equipment information management unit, test information management unit and report generation unit;

[0134] Grant three levels of operation authority: users can manage the equipment information management unit, test information management unit, protocol test unit and report generation unit;

[0135] The interactive module is used to provide an interactive interface for users, through which users can perform management operations corresponding to their operating permissions, and record various operations performed by users.

[0136] The above-mentioned unit modules may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to the above-mentioned modules.

[0137] This embodiment also provides a computer device, which may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 3As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for testing the MQTT communication protocol of a substation Internet of Things device is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0138] This embodiment further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0139] Establishing a plurality of first device models according to the first parameters of the target substation IoT device, wherein the first device models are all configured with the same communication protocol;

[0140] Preset a first test set, the first test set at least including a test scenario, a test case and a test instruction;

[0141] Establishing a first test environment based on the first test set, and testing the first device model in the first test environment in combination with the first test set;

[0142] Generate a test report based on the test results to complete the substation IoT equipment test.

[0143] Example 3

[0144] In a preferred embodiment, a substation IoT device MQTT communication protocol test system is provided, comprising:

[0145] Device information management unit, used to:

[0146] Store all tested substation IoT device models, record the device information of each substation IoT device model, and set a unified MQTT protocol configuration for each substation IoT device model. The device information includes the device name, device type and IP address, and the MQTT protocol configuration includes the parameter configuration of connection timeout, disconnection reconnection and message publishing frequency.

[0147] Test information management unit, used for:

[0148] Create and manage test cases, and store multiple test scenarios. Each scenario contains predefined events and conditions. Test cases include test names, test instructions, and test data. Test instructions are communication behaviors during testing, including message type, message content, sending frequency, and sending cycle.

[0149] Protocol test unit for:

[0150] Select the substation IoT device model, test cases and test scenarios required for the protocol test, and build a virtual IoT environment based on the selected test scenarios. In the built virtual IoT environment, load the substation IoT device model, and then execute the test process according to the selected test cases and monitor the test process.

[0151] Report generation unit for:

[0152] Record the test process and test results, and generate a detailed report for each round of testing based on the test process and test results;

[0153] User management unit for:

[0154] Different operation permissions are granted to users, and an interactive interface is provided for users. Users can enter the device information management unit, test information management unit, protocol test unit and report generation unit corresponding to the operation permissions through the interactive interface to perform management operations.

[0155] The technical effect of the above content is: the device information management unit can provide the substation Internet of Things device model required for the test, so that the user can select and load it into the test, and the test information management unit can provide the test cases and test scenarios required for the test, so that it can run according to the specified rules during the test. Before the test, the substation Internet of Things device model to be tested is selected, and then the appropriate test cases and test scenarios are selected according to the substation Internet of Things device model, so that a virtual Internet of Things environment can be built through the test scenario, and the substation Internet of Things device model can be loaded into the virtual Internet of Things environment, and then the test process can be executed according to the test case in the virtual Internet of Things environment, so that the communication behavior of the substation Internet of Things device can be tested. At the same time, real-time monitoring will be carried out during the test process. When an abnormal situation occurs during the test process, an early warning message will be issued, and the report generation unit can generate a test report based on the test process and test results, so that the user can conduct subsequent analysis and summary.

[0156] Example 4

[0157] According to the substation IoT device MQTT communication protocol test system provided in Example 3, the device information management unit includes a device model library, which is used to store all substation IoT device models to be tested, and the device model library classifies and stores the substation IoT device models based on the device information.

[0158] In the embodiment of the present application, the device model library includes:

[0159] Model management module for:

[0160] Provide users with management functions. Users can manage the substation IoT device models stored in the device model library through the provided management functions. The management functions include adding, editing, and deleting. Specifically:

[0161] Added function for users to add new substation IoT device models in the device model library;

[0162] The editing function is used to edit the substation IoT device models stored in the device model library, including editing the device information;

[0163] The delete function is used to delete the substation IoT device models stored in the device model library.

[0164] Test information management unit, including: test case library, used for:

[0165] Create and manage test cases, which include test name, test description and test data; the test case library includes:

[0166] In the embodiment of the present application, the use case management module is used to:

[0167] Provide users with a case management function. Users can manage the test cases stored in the test case library through the provided case management function, specifically:

[0168] Create a new test case: add a new test case, including test name, test instructions and test data; Edit a test case: edit an existing test case, including modifying the test name, test instructions and test data;

[0169] Delete test cases: Delete useless test cases;

[0170] Test case classification: classify test cases according to device models, test scenarios, and difficulty factors;

[0171] In the embodiment of the present application, the scene library is used to:

[0172] Store multiple test scenarios and predefine the events and conditions for each test scenario, including defining the network topology, sensor parameters, the type, quantity, and location of device models, and describing the events and behaviors of the device models in each scenario;

[0173] The scene library includes a scene management module, which is used to:

[0174] In an optional embodiment, a scenario management function is provided for the user, and the user manages the test scenarios stored in the scenario library through the provided scenario management function, specifically:

[0175] In an optional embodiment, network topology management: adding or editing network topology, including node names, connection relationships, and line parameters;

[0176] In an optional embodiment, sensor configuration: adds or edits various sensors, including sensor name, type, measurement range, and accuracy;

[0177] In an optional embodiment, device state setting: setting a default state, an operating state, and a fault state for each device model. The protocol test unit includes:

[0178] In the embodiment of the present application, the test preparation module is used to:

[0179] Select the substation IoT device model that needs to be tested, and select test cases and test scenarios based on the substation IoT device model;

[0180] In the embodiment of the present application, the simulation test module is used to:

[0181] Build a virtual IoT environment based on the selected test scenario;

[0182] Load the substation IoT device model that needs to be tested into the built virtual IoT environment;

[0183] In the embodiment of the present application, after completing the above operations, the test process is executed according to the test instructions in the test case; the test monitoring module is used to:

[0184] Monitor the test process in real time and compare the monitored test process with the test case. The test monitoring module includes:

[0185] In the embodiment of the present application, the abnormal warning module is used to:

[0186] Based on the comparison results of the test monitoring module, it is determined whether the test process is executed according to the test case. If it is not executed according to the test case, there is an abnormality in the test process, and an early warning message is sent to the management personnel via SMS.

[0187] Specifically, if the test case is not followed, there will be abnormalities in the test process, including:

[0188] In an optional embodiment, if the test case is not executed, then the specific data information of not executing according to the test case, wherein the specific data information of not executing according to the test case includes the test duration, the test output value and the test environment condition;

[0189] In an optional embodiment, the test duration is compared with the expected execution duration corresponding to the test case to obtain a first comparison result;

[0190] In an optional embodiment, the test output value is compared with an expected output value corresponding to the test case to obtain a second comparison result;

[0191] In an optional embodiment, the test environment condition is compared with the expected test environment condition corresponding to the test case to obtain a third comparison result; wherein the test environment condition includes an ambient temperature and an ambient humidity;

[0192] In an optional embodiment, the first abnormality evaluation index is obtained by using the first comparison result, the second comparison result and the third comparison result, wherein the first abnormality evaluation index is obtained by the following formula:

[0193]

[0194] Among them, E 10 Represents the first abnormal evaluation index; R 01 It represents the difference between the test duration and the expected execution duration of the test case. 02 Represents the difference between the test output value and the expected output value corresponding to the test case; R 03w Represents the difference between the humidity value in the test environment conditions and the humidity value in the test environment conditions corresponding to the test case; R 03t Represents the difference between the temperature value in the test environment conditions and the temperature value in the test environment conditions corresponding to the test case; R 01x Indicates the maximum allowable difference between the preset test duration and the expected execution duration of the test case; R 02x Indicates the maximum allowable difference between the preset test output value and the expected output value corresponding to the test case; R 03a Indicates the maximum allowable difference between the humidity value in the test environment conditions and the humidity value in the test environment conditions corresponding to the test case; R 03b Indicates the maximum allowable difference between the temperature value in the test environment conditions and the temperature value in the test environment conditions corresponding to the test case;

[0195] The first abnormal evaluation index is used to determine whether to perform a retest or issue an abnormal alarm.

[0196] The technical effect of the above technical solution is: by comparing the test duration, test output value and test environment conditions (including ambient temperature and ambient humidity) in the actual test process with the corresponding values ​​expected in the test case, it is possible to accurately identify whether there are abnormalities in the test process. This comparison is not limited to the output value, but also covers the test duration and test environment conditions, thereby providing a more comprehensive anomaly detection method. By calculating the first anomaly evaluation index (E 10 ), the technical solution can quantify the severity of abnormalities in the test process. This index comprehensively considers the difference between the test duration, test output value, and test environment conditions and the expected value, as well as the proportional relationship between these differences and the preset maximum allowable difference. This quantification method makes the evaluation of test abnormalities more objective and accurate. According to the value of the first abnormal evaluation index, it can be determined whether to retest or trigger an abnormal alarm. This decision-making mechanism helps to respond to abnormal situations in the test process in a timely manner and avoid potential quality problems. If the abnormal evaluation index is high, it indicates that there is a large deviation between the test process and the expectation, and retesting may be required to ensure product quality; if the abnormal evaluation index is low but still exceeds the acceptable range, an abnormal alarm can be triggered for further analysis and problem solving. Through automated comparison and calculation processes, the technical solution can significantly improve test efficiency and accuracy. It reduces the subjectivity and uncertainty of manual judgment, making the test process more standardized and repeatable. At the same time, by timely identifying and responding to test abnormalities, it helps to discover and solve problems in the early stages of product development, thereby reducing the cost of later repairs. The technical solution records the specific data information (such as test duration, test output value, and test environment conditions) that does not meet expectations during the test process, as well as the comparison results between these data information and expected values. This information provides traceability for the testing process and helps to find the root cause of problems during subsequent analysis and improvement.

[0197] In summary, this technical solution provides strong support for product quality assurance through its technical effects in accurately identifying test anomalies, quantifying the degree of anomalies, guiding follow-up actions, improving test efficiency and accuracy, and enhancing the traceability of the test process.

[0198] Specifically, the first abnormal evaluation index is used to determine whether to perform a retest or issue an abnormal alarm, including:

[0199] Comparing the first abnormal evaluation index with a preset evaluation index threshold;

[0200] When the first abnormal evaluation index is higher than the preset evaluation index threshold, it is determined to directly issue an abnormal alarm;

[0201] When the first abnormal evaluation index is not higher than the preset evaluation index threshold, it is determined to call a new test case for retesting and obtain the test result of the retesting;

[0202] If the test result of the retest indicates that the retest was not performed according to the test case, then the specific data information of the retest not being performed according to the test case is retrieved, wherein the specific data information of the retest not being performed according to the test case includes the test duration, the test output value, and the test environment condition;

[0203] Compare the test duration with the expected execution duration of the test case to obtain a first comparison result;

[0204] Compare the test output value with the expected output value corresponding to the test case to obtain a second comparison result;

[0205] Compare the test environment conditions with the expected test environment conditions corresponding to the test case to obtain a third comparison result; wherein the test environment conditions include ambient temperature and ambient humidity;

[0206] The abnormality evaluation index is obtained by using the first comparison result, the second comparison result and the third comparison result, wherein the second abnormality evaluation index is obtained by the following formula:

[0207]

[0208] Among them, E 20 Represents the second abnormal evaluation index; R 01 It represents the difference between the test duration and the expected execution duration of the test case. 02 Represents the difference between the test output value and the expected output value corresponding to the test case; R 03w Represents the difference between the humidity value in the test environment conditions and the humidity value in the test environment conditions corresponding to the test case; R 03t Represents the difference between the temperature value in the test environment conditions and the temperature value in the test environment conditions corresponding to the test case; R 01x Indicates the maximum allowable difference between the preset test duration and the expected execution duration of the test case; R 02x Indicates the maximum allowable difference between the preset test output value and the expected output value corresponding to the test case; R 03a Indicates the maximum allowable difference between the humidity value in the test environment conditions and the humidity value in the test environment conditions corresponding to the test case; R 03b Indicates the maximum allowable difference between the temperature value in the test environment conditions and the temperature value in the test environment conditions corresponding to the test case; E r Represents the basic index value, and the basic index value is obtained by the following formula:

[0209]

[0210] Among them, E r Indicates the basic index value; R 01 It represents the difference between the test duration and the expected execution duration of the test case. 02 Represents the difference between the test output value and the expected output value corresponding to the test case; R 03w Represents the difference between the humidity value in the test environment conditions and the humidity value in the test environment conditions corresponding to the test case; R 03t Indicates the difference between the temperature value in the test environment conditions and the temperature value in the test environment conditions corresponding to the test case;

[0211] In an optional embodiment, the second abnormality evaluation index is compared with a preset first abnormality evaluation index;

[0212] In an optional embodiment, when the second abnormal evaluation index is lower than the adjusted evaluation index threshold, only risk marking is performed without abnormal warning;

[0213] In an optional embodiment, when the second abnormal evaluation index is not lower than the adjusted evaluation index threshold, an abnormal warning is issued;

[0214] Among them, the adjusted evaluation index threshold is obtained by the following formula:

[0215]

[0216] Among them, E yt Represents the adjusted evaluation index threshold; E y Indicates the preset evaluation index threshold; E 10 Represents the first abnormality evaluation index.

[0217] The technical effect of the above technical solution is: by introducing the comparison between the first abnormal evaluation index and the preset evaluation index threshold, the technical solution can flexibly decide whether to perform abnormal alarm or retest according to the severity of the abnormality. This decision-making mechanism helps to avoid unnecessary repeated tests while ensuring timely response to serious abnormalities. When the first abnormal evaluation index is not higher than the preset threshold, retesting is selected instead of direct alarm, which helps to verify and solve the problem by simple retesting when the problem is not too serious, thereby avoiding unnecessary alarms and potential waste of resources. In the case where the retest is still not performed according to the test case, the technical solution further refines the abnormal handling strategy by calculating the second abnormal evaluation index and comparing it with the adjusted evaluation index threshold. This refinement helps to more accurately evaluate the abnormal situation after retesting and take corresponding measures (such as risk marking or abnormal warning) according to the evaluation results. The basic index value (Er) is introduced into the calculation formula of the second abnormal evaluation index, which comprehensively considers the difference between the test duration, the test output value and the test environment conditions and the expected value. This comprehensive consideration helps to improve the accuracy and objectivity of abnormal evaluation, thereby more accurately reflecting the actual situation during the test process. Through the adjusted evaluation index threshold formula, the technical solution can dynamically adjust the threshold according to the value of the first abnormal evaluation index. This dynamic adjustment mechanism helps to better adapt to abnormal situations in different test scenarios and improve the pertinence and effectiveness of abnormal handling. The technical solution records the specific data information that does not meet expectations during the test process, and performs multiple comparisons and calculations. These records provide traceability for the test process and help find the root cause of the problem in subsequent analysis and improvement. At the same time, by calculating the abnormal evaluation index and comparing the results, the abnormal situations in the test process can be analyzed more deeply, providing a basis for optimizing the test strategy and improving product quality.

[0218] In summary, this technical solution provides strong support for product quality assurance by enhancing the flexibility of test decisions, improving the efficiency of the test process, introducing a second anomaly evaluation index to refine anomaly handling, improving the accuracy of anomaly assessment, dynamically adjusting the evaluation index threshold, and enhancing the traceability and analyzability of the test process.

[0219] In an optional embodiment, the report generating unit includes:

[0220] Report generation module for:

[0221] Monitor and record the test process and test results of each round of protocol testing, and generate a test report based on the obtained test process and test results;

[0222] Report management module for:

[0223] Provide users with report management functions. Users can manage the generated test reports through the provided report management functions, specifically:

[0224] Report export: used to export test reports;

[0225] Report Preview: Preview the report directly online by clicking on the report link.

[0226] The technical effect of the above content is: the device information management unit can provide the substation IoT device model required for the test so that the user can select and load it into the test, and the test information management unit can provide the test cases and test scenarios required for the test so that it can run according to the specified rules during the test. Then, the test work of the substation IoT device MQTT communication protocol can be realized through the protocol test unit. First, the test preparation module needs to be prepared before the test. The substation IoT device model to be tested is selected from the device model library, and then the appropriate test cases and test scenarios are selected from the test case library and the scenario library respectively, so that the test can be carried out according to the rules specified by the test cases and test scenarios during the test. After the preparation work is completed, the simulation test module performs the test work. First, a virtual IoT environment is built according to the test scenario, and then the substation IoT device model is loaded into the built virtual IoT environment. Finally, the test process is executed according to the test instructions in the test case, so that the communication behavior of the substation IoT device can be tested, such as the sending frequency and sending cycle, to test the performance index. Through testing, possible communication problems can be discovered and solved, the occurrence of equipment failures can be reduced, and the reliability and performance of the equipment can be improved. During the test, the test monitoring module will monitor the test process in real time, and compare the monitored test process with the test case to determine whether the test process is executed in accordance with the test case. When there is an abnormality in the test process, the abnormal warning module will send a warning message to the management personnel via SMS to remind the management personnel or relevant personnel to pay attention and stop or temporarily test the work in time. After the test is completed, the report generation module can monitor and record the test process and test results of each round of protocol testing, and generate a corresponding test report after the test is completed, so that users can conduct subsequent analysis and summary, so as to improve the communication problems that may exist in the test report and ensure the normal operation of the communication protocol. The report management module provides users with export and preview functions. Users can export test reports and support export in multiple formats, such as HTML, PDF, CSV, etc., or users can preview the report online directly by clicking the report link.

[0227] Example 5

[0228] According to the substation IoT device MQTT communication protocol testing system provided in Example 3, the user management unit includes:

[0229] Operation permission module, used for:

[0230] Administrators grant users different operating permissions based on user information, including name, contact number, and position. Operating permissions are divided into level 1, level 2, and level 3, as follows:

[0231] Granting first-level operation authority: users can only perform management operations on the report generation unit;

[0232] Granting secondary operation authority: users can manage the equipment information management unit, test information management unit and report generation unit;

[0233] Grant three levels of operation authority: users can manage the equipment information management unit, test information management unit, protocol test unit and report generation unit;

[0234] Interaction modules for:

[0235] Provide users with an interactive interface through which they can perform management operations corresponding to their operating permissions.

[0236] The technical effect of the above content is: the user management unit can grant different operation permissions to the user through the operation permission module. When the user is granted the first-level operation permission, the user can only perform management operations on the report generation unit, that is, export or browse the test report. When the user is granted the second-level operation permission, the user can add, edit, delete and other management operations on the device information management unit, create, edit, delete and classify the test case library of the test information management unit, and perform network topology management, sensor configuration and device status setting management operations on the scenario library. It can also export or browse the report generation unit. When the user is granted the third-level operation permission, all management operations of the first and second levels can be realized, and protocol testing can be performed through the protocol testing unit. By granting different operation permissions to users, the security and reliability of the system can be effectively guaranteed.

[0237] Working principle: The device information management unit can provide the substation IoT device model required for the test so that the user can select and load it into the test, and the test information management unit can provide the test cases and test scenarios required for the test so that it can run according to the specified rules during the test. Before the test, select the substation IoT device model to be tested, and then select the appropriate test case and test scenario based on the substation IoT device model, so that a virtual IoT environment can be built through the test scenario, and the substation IoT device model can be loaded into the virtual IoT environment. Then, the test process can be executed according to the test case in the virtual IoT environment, so that the communication behavior of the substation IoT device can be tested. Through the test, possible communication problems can be discovered and solved, the occurrence of equipment failures can be reduced, and the reliability and performance of the equipment can be improved. At the same time, real-time monitoring will be carried out during the test. When an abnormal situation occurs during the test, an early warning message will be issued, and a test report will be generated based on the test process and test results, so that users can conduct subsequent analysis and summary, so as to improve the possible communication problems in the test report and ensure the normal operation of the communication protocol.

[0238] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0239] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0240] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0241] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0242] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0243] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0244] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for testing the MQTT communication protocol of a substation Internet of Things device, characterized in that: include: Establishing a plurality of first device models according to the first parameters of the target substation IoT device, wherein the first device models are all configured with the same communication protocol; Preset a first test set, the first test set at least including a test scenario, a test case and a test instruction; Establishing a first test environment based on the first test set, and testing the first device model in the first test environment in combination with the first test set; Generate a test report based on the test results to complete the substation IoT equipment test.

2. The MQTT communication protocol testing method for substation Internet of Things equipment according to claim 1, characterized in that: The testing of the first device model in combination with the first test set under the first test environment includes: Testing the first device model in a first test environment in combination with a first test set; Make the first judgment on the test process; A test report is generated according to the first determination result.

3. The MQTT communication protocol testing method for substation Internet of Things equipment according to claim 2, characterized in that: The first judgment includes: Compare the test process duration with the expected execution duration of the test case to obtain a first comparison result; Compare the test process output value with the expected output value corresponding to the test case to obtain a second comparison result; Comparing the test process environment condition with the expected test environment condition corresponding to the test case to obtain a third comparison result; A first abnormality evaluation index is acquired using the first comparison result, the second comparison result and the third comparison result.

4. The MQTT communication protocol testing method for substation Internet of Things equipment according to claim 3, characterized in that: The step of establishing a plurality of first device models according to the first parameters of the target substation IoT device includes: The first device models are all configured with a unified MQTT protocol configuration; The first parameter includes at least a device name, a device type and an IP address; The MQTT protocol configuration includes parameter configurations for connection timeout, disconnection reconnection, and message publishing frequency.

5. The MQTT communication protocol testing method for substation Internet of Things equipment according to claim 4, characterized in that: The first test set includes at least a test scenario, a test case and a test instruction; The test scenarios all include at least predefined events and conditions; The test case includes a test name, a test instruction and a test data; The test instruction is a communication behavior during the test, which at least includes a message type, a message content, a sending frequency and a sending period.

6. A substation IoT device MQTT communication protocol test system, characterized in that: include: A device information management unit configured with a first device model establishment function; A test information management unit configured with a first test set establishment function; A protocol testing unit configured to test the function of the first device model in combination with the first test set under the first test environment; A report generation unit is configured with a test report generation function.

7. The substation Internet of Things device MQTT communication protocol test system according to claim 6, characterized in that: Also includes a user management unit; The user management unit is used to grant users different operation permissions and provide users with an interactive interface. Users enter the device information management unit, test information management unit, protocol test unit and report generation unit corresponding to the operation permissions through the interactive interface to perform management operations.

8. The substation Internet of Things device MQTT communication protocol test system according to claim 7, characterized in that: The user management unit includes an operation authority module and an interaction module: The operation authority module is used by managers to grant users different operation authorities based on user information. User information includes name, contact number and position. Operation authorities are divided into level 1, level 2 and level 3, as follows: Granting first-level operation authority: users can only perform management operations on the report generation unit; Granting secondary operation authority: users can manage the equipment information management unit, test information management unit and report generation unit; Grant three levels of operation authority: users can manage the equipment information management unit, test information management unit, protocol test unit and report generation unit; The interactive module is used to provide an interactive interface for users, through which users can perform management operations corresponding to their operating permissions, and record various operations performed by users.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images