A message conversion plug-in tool that supports device-to-cloud

Abstract

The present invention provides a message conversion plug-in tool for supporting equipment on the cloud, including a message input module, a message conversion module, a message output module, a message log module, a message diversion module, a message current limiting module, and a message merging module; the plug-in tool runs between the Internet of Things platform and the edge device / gateway, and manages, controls, and coordinates the message sending process from the perspective of a middleman, relying on the message input module, the message conversion module, and the message output module; according to the specific usage scenario, the message log module, the message diversion module, the message current limiting module, and the message merging module are selected to cooperate with the sending work. The present invention can ensure that the message sending between the edge device and the Internet of Things platform is quickly realized under the premise of meeting the core business requirements of dynamic message conversion, supporting the switching of existing devices to the Internet of Things platform, and the access of incremental new devices to the Internet of Things platform through the joint action of multiple modules.

Description

Technical Field

[0001] The present invention relates to a message conversion technology, and in particular to a message conversion plug-in tool supporting a cloud on a device. Background Art

[0002] Putting devices on the cloud means that the objectively existing edge devices can report their status and issue control instructions to the cloud server through sensors, PLCs and network communications.

[0003] Generally, the communication protocol between edge devices and cloud servers is MQTT (Message Queue Telemetry Transport). During the communication process, the edge device acts as the MQTT client, and the IoT platform acts as the MQTT server. MQTT is a message protocol based on the client-server publish / subscribe paradigm under the ISO standard (ISO / IEC PRF 20922). It is a publish / subscribe message protocol designed for remote devices with low hardware performance and poor network conditions. As a lightweight transmission protocol developed for the Internet of Things, MQTT aims to provide reliable network services for IoT devices in low-bandwidth and unstable network environments. It has the following main features:

[0004] 1. Use the publish / subscribe messaging model to provide one-to-many message publishing and decouple applications;

[0005] 2. Message transmission with shielded payload content;

[0006] 3. Use TCP / IP to provide network connection;

[0007] 4. There are three types of message publishing service quality;

[0008] 5. Small transmission, small overhead (fixed-length header is 2 bytes), and minimized protocol exchange to reduce network traffic;

[0009] 6. Use the Last Will and Testament features to notify all parties of abnormal client interruptions.

[0010] Among them, in the process of one-to-many message publishing, due to the different themes and formats of message communications on different IoT platforms (such as Alibaba Cloud IOT platform, Huawei Cloud IOTDA platform, Dongfang Electric Tongchuang IOT platform, Baidu Cloud, China Mobile Onenet, etc.), edge devices often only support communication in a certain specified message communication theme and format in real production scenarios. For example, a Chinese patent document with a publication date of July 24, 2020 and a publication number of CN112003686A discloses a message format negotiation method and device. The method is applied to edge devices. In this scheme, the edge device sends a connection request message to the cloud platform, and the edge device receives a confirmation connection request message sent by the cloud platform, wherein the extended payload of the confirmation connection request message carries a message format supported by the cloud platform; the edge device sends the collected data to the cloud platform based on the message format. In this scheme, the message formats between the edge device and the cloud platform are limited to those supported by the cloud platform.

[0011] It can be seen that due to this single message format, the biggest limitation is that edge devices and cloud platforms cannot directly communicate with each other in multiple formats. Therefore, when migrating (accessing) the IoT platform, the subject and format of the message need to be readjusted one by one.

[0012] There are three quality of service (QOS) for transmission messages: At most once, at this level, message loss or duplication may occur, and message publishing depends on the underlying TCP / IP network. That is: <=1. At most once, at this level, the message will be guaranteed to arrive, but the message may be duplicated. That is: >=1. Only once, ensure that the message arrives only once. That is: =1.

[0013] Data transmission and protocol exchange are minimized (the protocol header is only 2 bytes) to reduce network traffic.

[0014] At present, the key to solving the above problems lies in realizing the runtime rewriting of messages by the IoT platform, but the relevant research on this issue is still in its infancy. Using an extensible, customizable, and highly available message conversion plug-in to realize the dynamic conversion of communication messages is a new perspective to solve this problem. Summary of the invention

[0015] The purpose of the present invention is to provide a message conversion plug-in tool that supports devices going to the cloud, which runs between the Internet of Things platform and the edge device / gateway, and manages, controls, and coordinates the message sending process as an intermediary; the message conversion plug-in tool works together through multiple modules to ensure that the core business needs of dynamic message conversion are met, and that the existing devices can switch to the Internet of Things platform and the incremental new devices can be connected to the Internet of Things platform. Under the premise, the rewriting and retransmission of messages between the edge device and the Internet of Things platform can be quickly realized.

[0016] The technical solution adopted by the present invention to solve its technical problem is:

[0017] A message conversion plug-in tool supporting device-to-cloud, comprising a message input module, a message conversion module, and a message output module;

[0018] The message input module is essentially an MQTT server, which is used to receive messages uploaded by edge devices and instructions (also messages) issued by the Internet of Things platform, parse the received messages or instructions and pass them to the message conversion module.

[0019] The message conversion module, as the downstream of the message input module, receives the message transmitted after successful parsing by the message input module, and converts the MQTT protocol communication topic and MQTT protocol communication format in the message information. The converted MQTT protocol communication topic and MQTT protocol communication format conform to the target Internet of Things platform uploaded by the edge device, and then passes the conversion result to the message output module.

[0020] The message output module is essentially an MQTT client. As the downstream of the message conversion module, it is used to receive the converted MQTT message sent by the message conversion module, and then output the received MQTT message to the target Internet of Things platform; the MQTT message refers to the conversion result transmitted by the message conversion module.

[0021] Based on the architecture of the above message conversion plug-in tool, the message input module is further designed as follows:

[0022] The messages received by the message input module include MQTT messages that the Internet of Things platform cannot directly parse and forward, that is, messages that the migrated or connected edge devices cannot directly send to the Internet of Things platform, and instructions that the Internet of Things platform cannot directly issue to the migrated or connected edge devices.

[0023] The message input module has built-in message topics and formats of multiple IoT platforms, such as Alibaba Cloud, Huawei Cloud, Dongfang Electric Tongchuang, Baidu Cloud, China Mobile OneNet, etc.

[0024] The message input module can also customize non-standardized message subjects and formats.

[0025] According to the architecture of the above-mentioned message conversion plug-in tool, the specific conversion process of the message conversion module is as follows:

[0026] (1) Although there are slight differences in the naming of topics among various IoT platforms, they are all designed according to the standard communication protocol of MQTT. Therefore, the conversion of MQTT protocol communication topics mainly involves mapping according to the corresponding relationship between the topics before and after the conversion.

[0027] (2) The conversion of the MQTT protocol communication format is to rearrange the key-value pairs of the Map structure obtained by parsing the message input module, and then create a new Map key-value pair that conforms to the target IoT platform.

[0028] In view of the architecture of the above-mentioned message conversion plug-in tool, based on the high availability mechanism characteristics of the MQTT protocol, when the message output module outputs to the outside (i.e., the IoT platform, or the next nested plug-in), different service quality levels (QOS) can be set to ensure that the message is accurately delivered.

[0029] Furthermore, with the development of the IoT platform, the number of edge devices connected to the IoT platform will continue to grow, which will cause the number of MQTT communication messages on the IoT platform to increase exponentially. In order to cope with the massive communication messages, the message conversion plug-in tool is also equipped with a message log module, a message diversion module, a message current limiting module, and a message merging module. The corresponding module can be selected to work according to the specific usage scenario.

[0030] The message log module is used to record the work log of the message conversion plug-in tool, and supports the log storage mode based on local files, the log storage mode based on MySql, and the storage mode based on EFK (Elasticserach + Filebeat + Kibana), ensuring that log information in business scenarios with different concurrency levels can be recorded. Among them, the storage mode based on EFK can bear the highest concurrency pressure, but the plug-in tool defaults to the log storage mode based on local files, which can bear the lowest concurrency and has the least external dependence.

[0031] In order to cope with the massive amount of communication messages, the IoT platform usually chooses to build an MQTT service cluster and formulate a corresponding load balancing mechanism. When the edge device reports data to the IoT platform, the message diversion module can be selected; when the message diversion module is selected, the message diversion module intercepts the conversion result output by the message conversion module and matches the destination address where the output message is to be sent; if the match is successful, the target address of the message will be reset according to the IP list and load balancing strategy of the configured MQTT load cluster, forming a new conversion result and passing it to the message output module to execute the message output.

[0032] The message diversion module supports at least four load balancing strategies: polling, random, weight, and hash; the polling strategy uses a polling method to send messages to different MQTT servers in the MQTT service cluster in turn; the random strategy sends messages to different MQTT servers in the MQTT service cluster randomly according to the random idea; the weight strategy uses a weighted random algorithm as a guide to randomly send messages to different MQTT servers in the MQTT service cluster; the hash strategy calculates the hash value of the message input, and determines the corresponding MQTT server in the MQTT service cluster as the forwarding address based on the hash value.

[0033] In order to cope with the massive amount of communication messages, it is necessary to make reasonable message flow limiting. Because not all messages reported at all times are important and worthy of attention, the attributes of edge devices often fluctuate within a certain range, which does not change the current state of the device, so an optional message flow limiting module is designed. In the stage where the edge device reports data to the IoT platform, the message flow limiting module is selected; when the message diversion module is selected, the message flow limiting module intercepts the message output by the message input module and determines the importance of the message; usually, the importance of the message is judged based on the relationship between the field threshold and the time threshold of the message. At the same time, in order to avoid normal messages being discarded, a minimum guarantee mechanism of sending at least one message per unit time is also set. A logical judgment is made based on the set field threshold and time threshold. The judgment principle is: if the current value of a field of an output message at a certain moment in a unit time exceeds the set field threshold, the message is immediately passed to the message conversion module, and the message conversion module converts the message and passes it to the message output module; otherwise, it is judged whether it is the message at the final moment at the end of the unit time period. If it meets the requirements of the final moment, the message is immediately passed to the message conversion module for conversion, otherwise the message is discarded.

[0034] In order to cope with the massive amount of communication messages, merging messages is also an optional solution. In the production process, a key device may be connected to multiple sensors, and these sensors each initiate communication messages to the IoT platform (for example: for a welding machine, there are electrical sensors and temperature sensors, and the values ​​of all welding sensors are merged through the message conversion plug-in tool and uploaded uniformly). When the edge device reports data to the IoT platform, the message merging module creates a buffer area between the message input module and the message conversion module, uses the buffer area to aggregate the message information of each component of the same edge device, and transmits values ​​to the message conversion module at a certain frequency.

[0035] By selecting the message flow limiting module, the message diversion module, and the message merging module, the service quality effect can be significantly improved and the number of network access times can be reduced.

[0036] Furthermore, the root directory of the message conversion plug-in tool contains three key folders, namely the core directory, the modules directory and the conf directory.

[0037] (1) The core directory is the core directory and serves as the entry point for this message conversion plug-in tool.

[0038] In the core directory, the Main.java file is used to define the main process of the message conversion plug-in tool and set the global state machine;

[0039] AOP.java in the core directory provides the extension of aspect function points for the Main.java file from the aspect perspective, ensuring that the module can enrich the main process defined in Main.java;

[0040] The IOC.xml file in the core directory defines the module loading and usage logic, and creates instance objects of each module based on the Inversion of Control container provided by the Spring framework for AOP.java to call.

[0041] (2) The modules directory is the module folder of the message conversion plug-in tool.

[0042] The modules directory contains at least sub-directories such as input, transform, output, logs, load-balancing, current-limiting, and merge. These sub-directories correspond to the message input module, message conversion module, message output module, message log module, message diversion module, message current limiting module, and message merging module respectively.

[0043] (3) The conf directory is the configuration folder of the message conversion plug-in tool.

[0044] The application.yml file contained in the conf directory is used to define the system configuration and business configuration of the message conversion plug-in tool. The system configuration mainly sets the JVM (Java Virtual Machine) parameters, and the business configuration mainly configures the module usage.

[0045] The specific implementation steps of the present invention include:

[0046] ①Create a core directory, write the main process logic, state transition machine and plug-in entry function; then jump to step ②.

[0047] ② Create the modules directory, and create the corresponding subdirectories of each module one by one according to the corresponding relationship: input, transform, output, logs, load-balancing, current-limiting, merge. Then, define the standard interface of each module in the subdirectory of each module, and provide a default implementation class based on the interface; then jump to step ③.

[0048] ③Create a conf directory, write the configuration of the middleware that the message conversion plug-in tool depends on into the application.yml file, and migrate the common configuration parameters of each module of the plug-in to application.yml; finally, host it to the configuration center to ensure that changes to application.yml can be dynamically perceived; then jump to step ④.

[0049] The configuration center can be used to dynamically modify configurations for users. For example, the application.yml in the associated configuration folder of Alibaba Nacos erueka and Google zookeeper can be used to perceive application.yml in real time.

[0050] ④Package the developed message conversion plug-in tool and deploy it on the server; then jump to step ⑤.

[0051] ⑤Select an edge device and modify its message sending address to the server where the message conversion plug-in tool is deployed. Then jump to step ⑥.

[0052] ⑥ When the network is unobstructed, the message conversion plug-in tool can receive the message sent by the edge device. Before formally receiving it, it is necessary to determine whether the functions of each module of the message conversion plug-in tool are sufficient to support the migration or access process of the edge device; if the judgment result is "sufficient", jump to step ⑦, otherwise jump to step ⑧.

[0053] When judging, it is to judge whether the message conversion plug-in tool can support the migration or access process of the theme and format conversion.

[0054] ⑦ Modify the configuration of application.yml in the configuration center according to actual needs. The modified configuration includes: the IP, port, identity authentication information, message mapping theme and format of the cloud platform on the device to be transferred, and the selection of each module of the plug-in; then jump to step ⑩.

[0055] ⑧It is necessary to enhance each module of the message conversion plug-in tool; specifically, it can be re-implemented according to the standard interface provided by each module, or the default implementation class can be integrated to perform targeted enhancement on the specific functions of the default module. More often, the message conversion module is enhanced; then jump to step ⑨.

[0056] ⑨Upload the extended implementation class through the upload interface of the message conversion plug-in tool (that is, place it in the folder corresponding to the module), and dynamically load it through the Java reflection mechanism to achieve a smooth upgrade of the message conversion plug-in tool; then jump to step ⑦.

[0057] ⑩ The request sent by the edge device is dynamically rewritten by the message conversion plug-in tool and can be efficiently sent to the cloud server on the target device, successfully realizing low-cost migration of the device to the cloud platform.

[0058] Through the implementation of the above technical solution, the beneficial effects of the present invention are:

[0059] 1. This message conversion plug-in tool can meet the dynamic conversion requirements of messages from devices to the cloud, and quickly realize the rewriting and publishing of messages between edge devices and the IoT platform.

[0060] 2. Under the premise of supporting the switching of existing devices to the IoT platform and the access of new incremental devices to the IoT platform, it can achieve multi-module scalability. Users can match modules according to actual scenarios to expand plug-in functions;

[0061] 3. All modules of this message conversion plug-in tool support custom configuration. Most of the customization requirements can be realized through configuration (whether this function is realized by the configuration folder). At the same time, the message conversion plug-in tool defines a standard interface. Users can use the implementation class of the interface to achieve complete customization at the module level.

[0062] 4. Effectively solve the problem that the continuous and high-frequency data reporting messages may cause the MQTT server to crash and thus cause the entire IoT platform to fuse with the increasing number of access devices. Optional coordinated message conversion and output are performed from the perspectives of load balancing, intelligent filtering, and message merging to improve the stability of the IoT platform;

[0063] 5. This message conversion plug-in tool can configure multiple alternatives for a module. These alternatives are different implementations of the standard interface of the module, so there is a natural tree structure relationship of implementation (or inheritance). Before running, you only need to specify the corresponding implementation class for each module. If you do not specify it, the module will be executed according to the default configuration. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1This is a diagram showing the positioning of the present invention in the equipment data acquisition process, that is, a specific module architecture.

[0065] Figure 2 It is a schematic diagram of the cooperation mode of each module in the present invention.

[0066] Figure 3 It is a schematic diagram of the file structure of the present invention.

[0067] Figure 4 It is a schematic diagram of the entire life cycle of the present invention.

[0068] Figure 5 This is a schematic diagram of the wiring method of the temperature and humidity sensor in Example 1.

[0069] Figure 6 This is a schematic diagram of the process of using the traditional method in Example 1.

[0070] Figure 7 This is a comparison table of MQTT communication message examples and translation results in Example 1.

[0071] Figure 8 This is a schematic diagram of the format of the resource field of an example record in Example 2.

[0072] Fig. 9 This is a working principle diagram of the message current limiting module of the present invention.

[0073] Fig.10 This is a working principle diagram of the message merging module of the present invention. DETAILED DESCRIPTION

[0074] like Figure 1-2 As shown, this embodiment provides a message conversion plug-in tool that supports the cloud of devices, including a message input module, a message conversion module, a message output module, a message log module, a message diversion module, a message flow limiting module, and a message merging module. According to the specific process, the message log module, the message diversion module, the message flow limiting module, and the message merging module can be selected to work.

[0075] The message input module is essentially an MQTT server, which is used to receive messages uploaded by edge devices and instructions (also messages) issued by the Internet of Things platform, parse the received messages or instructions and pass them to the message conversion module. The received messages include MQTT messages that cannot be directly parsed and forwarded by the Internet of Things platform, that is, messages that cannot be directly sent to the Internet of Things platform by the migrated or connected edge devices, and instructions that cannot be directly issued by the Internet of Things platform to the migrated or connected edge devices. The message input module has built-in message themes and formats of multiple Internet of Things platforms, such as Alibaba Cloud, Huawei Cloud, Dongfang Electric Tongchuang, Baidu Cloud, China Mobile Onenet, etc., and can also customize non-standardized message themes and formats. The message input module parses the message to obtain a key-value pair of a Map structure, and the key-value pair structure can be as follows:

[0076] {

[0077] Topic: aaa,

[0078] Data:

[0079] }

[0080] }

[0081] The message conversion module, as the downstream of the message input module, receives the message transmitted after the message input module successfully parses it, and converts the MQTT protocol communication topic and the MQTT protocol communication format in the message information. The converted MQTT protocol communication topic and the MQTT protocol communication format conform to the target IoT platform uploaded by the edge device, and then passes the conversion result to the message output module; the conversion module can further reduce network traffic through specific topic conversion and MQTT protocol communication format conversion; the conversion result includes the communication topic and message content that can be correctly received by the target IoT platform, for example:

[0082] Topic conversion: / sys / a1rSPQr0tkP / D001 / thing / service / property / set ->

[0083] / dec / a1rSPQr0tkP / D001 / thing / service / property / set

[0084] Data conversion {name: sensor01, …} -> {Dname: sensor01, …}

[0085] The specific conversion process of the message conversion module is as follows:

[0086] (1) Although there are slight differences in the naming of topics among various IoT platforms, they are all designed according to the standard communication protocol of MQTT. Therefore, the conversion of MQTT protocol communication topics mainly involves mapping according to the corresponding relationship between the topics before and after the conversion.

[0087] (2) The conversion of the MQTT protocol communication format is to rearrange the key-value pairs of the Map structure obtained by parsing the message input module, and then create a new Map key-value pair that conforms to the target IoT platform.

[0088] The message output module is essentially an MQTT client. As the downstream of the message conversion module, it is used to receive the converted MQTT message sent by the message conversion module, and then output the received MQTT message to the target IoT platform; the MQTT message includes the conversion result transmitted by the message conversion module. Based on the high availability mechanism characteristics of the MQTT protocol, different service quality levels can be set when the message output module outputs to the outside to ensure that the message reaches accurately.

[0089] The message log module is used to record the work log of the message conversion plug-in tool, supports a local file-based log storage mode, a MySql-based log storage mode, and an EFK-based storage mode, ensuring that log information can be recorded in business scenarios with different concurrency levels.

[0090] Based on the IoT platform with multiple MQTT server-side diversions, build MQTT server clusters, MQTT load clusters and formulate corresponding load balancing mechanisms. When the edge device reports data to the IoT platform, the message diversion module can be selected; when the message diversion module is selected, the message diversion module intercepts the conversion result output by the message conversion module and matches the destination address where the output message is to be sent; if the match is successful, the target address of the message will be reset according to the IP list and load balancing strategy of the configured MQTT load cluster, forming a new conversion result and passing it to the message output module to execute the message output.

[0091] The message diversion module supports at least four load balancing strategies: polling, random, weight, and hash; the polling strategy uses a polling method to send messages to different MQTT servers in the MQTT service cluster in turn. This strategy is the default strategy after the message diversion module is enabled; the random strategy randomly sends messages to different MQTT servers in the MQTT service cluster according to the random idea; the weight strategy is guided by a weighted random algorithm to randomly send messages to different MQTT servers in the MQTT service cluster; the hash strategy calculates the hash value of the message input, and determines the corresponding MQTT server in the MQTT service cluster as the forwarding address based on the hash value.

[0092] In the stage when the edge device reports data to the IoT platform, a message current limiting module is selected; when the message diversion module is selected, the message current limiting module intercepts the message output by the message input module and determines the importance of the message; usually, the importance of the message is judged based on the relationship between the field threshold and the time threshold of the message. At the same time, in order to avoid the normal messages being discarded, a guarantee mechanism is also set to send at least one message per unit time. Logical judgment is made according to the set field threshold and time threshold. The judgment principle is: if the current value of a field of an output message at a certain moment in a unit time exceeds the set field threshold, the message is immediately passed to the message conversion module, and the message conversion module converts the message and passes it to the message output module; otherwise, it is judged whether it is the message at the final moment at the end of the unit time period. If it meets the requirements of the final moment, the message is immediately passed to the message conversion module for conversion, otherwise the message is discarded (for example, a message is sent every 5 minutes); the working principle of the message current limiting module is as follows Fig. 9 shown.

[0093] In order to cope with the massive amount of communication messages, merging messages is also an optional solution. In the production process, a key device may be connected to multiple sensors, and these sensors each initiate communication messages to the IoT platform. When the edge device reports data to the IoT platform, the message merging module creates a buffer between the message input module and the message conversion module, uses the buffer to aggregate the message information of each component of the same edge device, and transmits the value to the message conversion module at a certain frequency. This process is as follows Fig.10 shown.

[0094] exist Fig.10In the example, there are three sensing devices working. Now assume that these three sensors are respectively measuring the status of the same large equipment. For example, in the process of equipment welding, there is a temperature sensor to measure the temperature of the welding point, a Hall sensor to measure the welding current and voltage of the welding machine, and a position sensor to measure the current working position of the welding machine. All three sensors need to transmit information to the Internet of Things platform through the Internet of Things gateway. The present invention proposes that the message merging module in the message conversion plug-in supporting the device on the cloud can define a complex object of a device in the memory. In this example, the object has at least four attribute fields: current temperature, current current, current voltage, and current position coordinates. The complex object in the memory is then updated using the value uploaded by the message, and the message is discarded after the update is completed. At the same time, the message merging module also presets an asynchronous timed task, which can regularly send the complex object in the memory to the message conversion module. The message merging module merges multiple synchronous messages into an asynchronous message, which can effectively improve the availability of the Internet of Things platform.

[0095] By selecting the message flow limiting module, message diversion module, and message merging module, the service quality effect can be significantly improved and network traffic can be reduced.

[0096] Furthermore, if Figure 3 As shown, the root directory of the message conversion plug-in tool contains three key folders, namely the core directory, the modules directory and the conf directory. Among them:

[0097] (1) The core directory is the core directory and serves as the entry point for this message conversion plug-in tool.

[0098] In the core directory, the Main.java file is used to define the main process of the message conversion plug-in tool and set the global state machine;

[0099] AOP.java in the core directory provides the extension of aspect function points for the Main.java file from the aspect perspective, ensuring that the module can enrich the main process defined in Main.java;

[0100] The IOC.xml file in the core directory defines the module loading and usage logic, and creates instance objects of each module based on the inversion of control container provided by the Spring framework for AOP.java to call.

[0101] (2) The modules directory is the module folder of the message conversion plug-in tool.

[0102] The modules directory contains at least sub-directories such as input, transform, output, logs, load-balancing, current-limiting, and merge. These sub-directories correspond to the message input module, message conversion module, message output module, message log module, message diversion module, message current limiting module, and message merging module respectively.

[0103] Taking the transform directory and the message conversion module as an example, the transform directory contains: the standard interface TransInterface.java of the message conversion module, the default implementation class TransImplDefault.java of the message conversion module, and the custom implementation classes TransImpl1.java and TransImpl2.java of the message conversion module.

[0104] (3) The conf directory is the configuration folder of the message conversion plug-in tool.

[0105] The application.yml file contained in the conf directory is used to define the system configuration and business configuration of the message conversion plug-in tool. The system configuration mainly sets the JVM (Java Virtual Machine) parameters, and the business configuration mainly configures the module usage.

[0106] Combination Figure 4 As shown, the specific implementation steps of the present invention include:

[0107] ①Create a core directory, write the main process logic, state transition machine and plug-in entry function; then jump to step ②.

[0108] ② Create the modules directory, and create the corresponding subdirectories of each module one by one according to the corresponding relationship: input, transform, output, logs, load-balancing, current-limiting, merge. Then, define the standard interface of each module in the subdirectory of each module, and provide a default implementation class based on the interface; then jump to step ③.

[0109] ③Create a conf directory, write the configuration of the middleware that the message conversion plug-in tool depends on into the application.yml file, and migrate the common configuration parameters of each module of the plug-in to application.yml; finally, host it to the configuration center to ensure that changes to application.yml can be dynamically perceived; then jump to step ④.

[0110] The configuration center can be used to dynamically modify configurations for users. For example, the application.yml in the associated configuration folder of Alibaba Nacos erueka and Google zookeeper can be used to perceive application.yml in real time.

[0111] ④Package the developed message conversion plug-in tool and deploy it on the server; then jump to step ⑤.

[0112] ⑤Select an edge device and modify its message sending address to the server where the message conversion plug-in tool is deployed. Then jump to step ⑥.

[0113] ⑥ When the network is unobstructed, the message conversion plug-in tool can receive the message sent by the edge device. Before formally receiving it, it is necessary to determine whether the functions of each module of the message conversion plug-in tool are sufficient to support the migration or access process of the edge device; if the judgment result is "sufficient", jump to step ⑦, otherwise jump to step ⑧.

[0114] When judging, it is to judge whether the message conversion plug-in tool can support the migration or access process of the theme and format conversion.

[0115] ⑦ Modify the configuration of application.yml in the configuration center according to actual needs. The modified configuration includes: the IP, port, identity authentication information, message mapping theme and format of the cloud platform on the device to be transferred, and the selection of each module of the plug-in; then jump to step ⑩.

[0116] ⑧It is necessary to enhance each module of the message conversion plug-in tool; specifically, it can be re-implemented according to the standard interface provided by each module, or the default implementation class can be integrated to perform targeted enhancement on the specific functions of the default module. More often, the message conversion module is enhanced; then jump to step ⑨.

[0117] ⑨Upload the extended implementation class through the upload interface of the message conversion plug-in tool (that is, place it in the folder corresponding to the module), and dynamically load it through the Java reflection mechanism to achieve a smooth upgrade of the message conversion plug-in tool; then jump to step ⑦.

[0118] ⑩ The request sent by the edge device is dynamically rewritten by the message conversion plug-in tool and can be efficiently sent to the cloud server on the target device, successfully realizing low-cost migration of the device to the cloud platform.

[0119] Example 1

[0120] The core function of the message conversion tool on the cloud supporting device corresponding to the above setting is to realize dynamic message conversion. As for this plug-in, since there is a clear division of functional modules, the demand for dynamic message conversion is directly realized by the message conversion module.

[0121] Taking the RS-WS-N01-2-4 temperature and humidity sensor produced by Jianda Renke and the G3557 IoT gateway device produced by Yunken Technology as an example of connecting to Dongfang Electric's "Tongchuang" IoT platform, the message conversion tool proposed in the present invention to support devices going to the cloud can realize and greatly simplify the above-mentioned access process.

[0122] Jianda Renke RS-WS-N01-2-4 temperature and humidity sensor supports communication in the form of modbus. Figure 5 After completing the wiring, you can send modbus RTU communication messages to the outside to report the temperature and humidity values ​​collected by the sensor.

[0123] However, the Jianda Renke RS-WS-N01-2-4 temperature and humidity sensor does not have a network module, so it needs an IoT gateway as an intermediary to achieve two-way communication with the IoT platform. In this case, the IoT gateway selected is the Yunken Technology G3557 IoT gateway. The gateway supports the conversion of edge device modbus RTU communication messages into MQTT communication messages, and has built-in communication topics and formats for platforms such as Alibaba Cloud IOT and Huawei Cloud IOTDA. Combining edge devices and edge gateways to access the IoT platform is a common method for device access and data collection at this stage.

[0124] The original intention of Dongfang Electric's "Tongchuang" IoT platform was to serve the energy equipment manufacturing process. Since the process has the characteristics of "small batch", "high value" and "one-time completion", more mandatory constraints are set on the communication process when designing the communication rules of the IoT platform. For example, most public cloud platforms believe that it is not necessary to digitally sign data for MQTT message communication to ensure data integrity; however, energy equipment is a very important security device. If the data is tampered with, it may cause heavy economic losses and even threaten the lives of the people. Therefore, Dongfang Electric's "Tongchuang" IoT platform requires that each message be accompanied by a digital signature of the communication data. IoT platforms such as China Mobile Onenet, Haier CoaX D3OS, and Aerospace Cloud Network INDICS have also referred to the communication themes and protocols of the IoT platforms of leading Internet companies, and formulated a set of private communication standards based on business scenarios.

[0125] If the traditional solution is adopted, the process of connecting the RS-WS-N01-2-4 temperature and humidity sensor produced by Jianda Renke and the G3557 IoT gateway device produced by Yunken Technology to the Dongfang Electric "Tongchuang" IoT platform can be expressed as follows: Figure 6 shown.

[0126] Compared with the traditional method, the IoT gateway will first initiate an inquiry frame to the temperature and humidity sensor. In this example, the message of the inquiry frame is "01 03 00 00 00 02 c4 0b". The analysis of the message is shown in Table 1.

[0127]

[0128] Table 1. Comparison table of inquiry frame examples

[0129] For the traditional method, when the temperature and humidity sensor receives the inquiry, it will respond to the IoT gateway with a response frame. In this case, the message of the response frame is "01 03 04 02 92 ff 9b 5a 3d". The analysis of the message is shown in Table 2.

[0130]

[0131] Table 2 Response frame example comparison table

[0132] For the traditional method, after the gateway receives the response from the temperature and humidity sensor, it will send an MQTT message to the Dongdian "Tongchuang" IoT platform. Before formal communication, the IoT gateway will negotiate with the "Tongchuang" IoT platform to create an MQTT communication channel through the MQTT protocol, and then communicate according to the changed channel. The present invention does not improve the negotiation process, so it is assumed that the channel has been created and communication is carried out directly. During the communication process, the IoT gateway will send an attribute reporting message "30 99 01 00 30 73 2f 61 31 72 53 50 51 72 30 74 6b 50 2f 44 30 30 31 2f74 68 69 6e67 2f 73 65 72 76 69 63 65 2f 70 72 6f 70 65 72 74 79 2f 73 65 747b 22 6d 65 74 68 6f 64 22 3a 22 74 68 69 6e 67 2e 73 65 72 76 69 63 65 2e 7072 6f 70 6572 74 79 2e 73 65 74 22 2c 22 69 64 22 3a 22 39 32 38 33 35 31 3435 36 22 2c 22 70 61 72 61 6d 73 22 3a 7b 22 74 65 6d 70 65 72 61 74 75 72 6522 3a 2d 3130 31 2c 22 68 75 6d 69 64 69 74 79 22 3a 36 35 38 2c 22 74 69 6d65 73 74 61 6d 70 22 3a 20 31 36 37 31 31 31 37 37 36 30 39 32 31 7d 7d", the analysis of this message is as follows Figure 7 shown.

[0133] For traditional methods, the subject and format of the message sent by the IoT gateway do not meet the communication specifications of Dongfang Electric's "Tongchuang" IoT platform, so Dongfang Electric's "Tongchuang" IoT platform will regard it as a junk message and discard it, which will lead to data reporting failure. Summarizing the reasons for the failure, it can be found that the failure is caused by the inconsistency of the specifications between the sender and the receiver.

[0134] In order to dynamically solve the above problems, the present invention adds a message conversion plug-in tool on the cloud supporting device to the traditional method to dynamically rewrite and resend the message. The specific process is as follows: Figure 6 shown.

[0135] With the present invention, the contents of "gateway-initiated inquiry frame" and "device-response reply frame" are exactly the same as those of the traditional method. Subsequently, the gateway does not communicate directly with the IoT platform through MQTT, but communicates with the message conversion plug-in tool on the cloud supporting the device proposed by the present invention. After the message conversion plug-in tool receives the message transmitted by the gateway, it will rewrite the message content based on the message conversion module. In this example, firstly, according to the mapping relationship of the communication topic, the original topic " / sys / a1rSPQr0tkP / D001 / thing / service / property / set" is changed to " / dec / a1rSPQr0tkP / D001 / thing / service / property / set" through regular matching; secondly, in order to avoid floating point numbers in the measured temperature and humidity, the temperature and humidity sensor amplifies the result by 10 times. The gateway did not process the result when forwarding it, so it was inconsistent with the unit of Dongfang Electric's "Tongchuang" IoT platform, so it needed to be reduced by 10 times. Therefore, the temperature processed by the plug-in was -10.1℃ (unit omitted), and the humidity was 65.8% (percent sign omitted); finally, because Dongfang Electric's "Tongchuang" IoT platform attaches importance to data integrity and enforces the constraint that each message must provide a signature, the plug-in will add a "sign" field to the message, and its value is the hash value of the data after hashing using the md5 algorithm, which is "827FAC6A4D016D53B6874573CF37FC05" in this example.

[0136] After the message conversion plug-in supporting the device to go to the cloud is completed, the message that can be correctly received by the IoT platform can be passed to the Dongfang Electric "Tongchuang" IoT platform to complete the communication.

[0137] Example 2

[0138] After the message conversion module completes the conversion, if the message conversion plug-in tool enables the message diversion module, the conversion result of the message conversion module will not be directly transmitted to the message output module, but will be hijacked by the message diversion module.

[0139] In the message distribution module, a data table named load_balance_record is maintained. The scheme structure of the table is shown in Table 3.

[0140]

[0141] Table 3 Scheme structure of load_balance_record in the message distribution module

[0142] The id field in the scheme structure is the primary key of the table, and its type is string. The specific value of this field in a record is the timestamp of creation, which can ensure that the id in the table is monotonically non-decreasing and unique, such as "1671173380171"; the traget_host field type in the scheme structure is a string, and the specific value of this field in a record is a certain IP+port number, such as "10.18.30.118:1883"; the strategy_type field type in the scheme structure is a string, and the specific value of this field in a record is any one of polling, random, weight, and hash (this range can be expanded by rewriting the message diversion module, and only four strategies of polling, random, weight, and hash are supported by default). The resource field type in the scheme structure is JSON (essentially still a string). The resource is an array composed of multiple host objects, and each host is composed of attributes such as url and weight. The resource field of a record is generally Figure 8 The format shown.

[0143] For messages sent to the message splitting module, the message splitting module obtains the target IP and port of the message and matches the target_host column of the load_balance_record table to see if there are the same records:

[0144] If not, it will be forwarded directly to the IoT platform of the destination IP;

[0145] If yes, the real address to which the MQTT message is sent is determined according to the strategy corresponding to the strategy_type column of the load_balance_record table and the resource JSON in the resource column of the load_balance_record table.

[0146] The default implementation of the message diversion module supports four load balancing strategies: polling, random, weight, and hash. The polling strategy uses a round-robin method to send messages to different hosts in the resource squadron in turn. This strategy is the default strategy after the message diversion module is enabled; the random strategy randomly sends messages to different hosts in the resource squadron according to the random idea; the weight strategy uses a weighted random algorithm as a guide to randomly send messages to different hosts in the resource squadron; the hash strategy calculates the hash value of the message input, and determines the corresponding host in the resource as the forwarding address based on the hash value.

Claims

1. A message conversion plug-in tool that supports devices on the cloud, characterized in that: It includes a message input module, a message conversion module, and a message output module; The message input module is used to receive messages uploaded by edge devices and instructions issued by the Internet of Things platform, which are also messages; parse the received messages and pass them to the message conversion module; The message conversion module is used to receive the message transmitted after the message input module successfully parses it, and convert the MQTT protocol communication subject and the MQTT protocol communication format in the message to obtain a conversion result; the MQTT protocol communication subject and the MQTT protocol communication format of the MQTT message in the conversion result both conform to the target Internet of Things platform uploaded by the edge device, and then pass the conversion result to the message output module; The message output module is used to receive the converted MQTT message sent by the message conversion module, and then output the received MQTT message to the target Internet of Things platform; the MQTT message includes the conversion result transmitted by the message conversion module; According to the specific usage scenario, the plug-in tool is also equipped with a message log module, a message diversion module, a message flow limiting module, and a message merging module as optional modules in usage scenarios with higher availability requirements: (1) When it is necessary to record and monitor the message conversion plug-in tool, a message log module is selected; different message log storage methods are specified according to business scenarios with different concurrency levels to meet the work log recording requirements of the message conversion plug-in tool in various usage scenarios; the message log module supports a log storage mode based on local files, a log storage mode based on MySql, and a storage mode based on EFK; Among them, the storage mode based on EFK can bear the highest concurrency pressure, but the plug-in tool adopts the log storage mode based on local files by default, which can bear the lowest concurrency and has the least external dependence; (2) When there are many communication messages, one or more of the message diversion module, message current limiting module, and message merging module are selected; For an IoT platform with an MQTT server cluster and an MQTT load cluster, a message diversion module may be used; the message diversion module is used to intercept messages output by the message conversion module and match the address of the destination IoT platform where the output message is to be sent; if the match is successful, the target address of the message is reset according to the IP list and load balancing strategy of the configured MQTT load cluster, a new conversion result is formed and passed to the message output module to execute the message output; In the stage when the edge device reports data to the IoT platform, a message flow limiting module can be selected; when the message diversion module is selected, the message flow limiting module intercepts the message output by the message input module and determines the importance of the message; logical judgment is performed according to the set field threshold and time threshold, and the judgment principle is: if the current value of a field of a certain output message at a certain moment in a unit time exceeds the set field threshold, the message is immediately passed to the message conversion module, and the message conversion module converts the message and passes it to the message output module; otherwise, it is judged whether it is the message at the final moment at the end of the unit time period. If it meets the requirements of the final moment, the message is immediately passed to the message conversion module for conversion, otherwise the message is discarded; When the edge device reports data to the Internet of Things platform, a message merging module can be selected; the message merging module creates a buffer area between the message input module and the message conversion module, uses the buffer area to aggregate the message information of various components of the same edge device, and transmits values ​​to the message conversion module at a certain frequency.

2. The message conversion plug-in tool according to claim 1, characterized in that: The messages received by the message input module include MQTT messages that cannot be directly parsed and forwarded by the Internet of Things platform.

3. The message conversion plug-in tool according to claim 1, characterized in that: The message input module has built-in message topics and formats of multiple Internet of Things platforms.

4. The message conversion plug-in tool according to claim 1, characterized in that: The message input module has the function of customizing non-standardized message subjects and formats.

5. The message conversion plug-in tool according to claim 1, characterized in that: The specific conversion process of the message conversion module is as follows: (1) For the conversion of MQTT protocol communication topics, mapping is performed according to the corresponding relationship between the topics before and after the conversion; (2) The conversion of the MQTT protocol communication format is to rearrange the key-value pairs of the Map structure obtained by parsing the message input module, and then create a new Map key-value pair that conforms to the subject and format of the target IoT platform.

6. The message conversion plug-in tool according to claim 1, characterized in that: For the IoT platform that builds an MQTT server cluster, it is required to have multiple MQTT server-side traffic diversions.

7. The message conversion plug-in tool according to claim 1, characterized in that: The load balancing includes multiple levels of load balancing.

8. The message conversion plug-in tool according to claim 1 or 7, characterized in that: The message diversion module supports at least four load balancing strategies: polling, random, weight, and hash; the polling strategy uses a polling method to send messages to different MQTT servers in the MQTT service cluster in turn; the random strategy sends messages to different MQTT servers in the MQTT service cluster randomly according to the random idea; the weight strategy uses a weighted random algorithm as a guide to randomly send messages to different MQTT servers in the MQTT service cluster; the hash strategy calculates the hash value of the message input, and determines the corresponding MQTT server in the MQTT service cluster as the forwarding address based on the hash value.

9. The message conversion plug-in tool according to claim 1, characterized in that: The root directory of the message conversion plug-in tool contains three folders: core directory, modules directory and conf directory; among them: (1) The core directory is the core directory and serves as the entry point for the message conversion plug-in tool; In the core directory, the Main.java file is used to define the main process of the message conversion plug-in tool and set the global state machine; AOP.java in the core directory provides the Main.java file with an extension of aspect function points from the aspect perspective, ensuring that each module meets the main process defined in Main.java. The IOC.xml file in the core directory defines the loading and use logic of each module. Based on the inversion of control container provided by the Spring framework, instance objects of each module are created for AOP.java to call. (2) The modules directory is the module folder of the message conversion plug-in tool; The modules directory contains at least input, transform, output, logs, load-balancing, current-limiting, and merge subdirectories, which correspond to the message input module, message conversion module, message output module, message log module, message diversion module, message current limiting module, and message merging module respectively; (3) The conf directory is the configuration folder of the message conversion plug-in tool; The application.yml file contained in the conf directory is used to agree on the system configuration and business configuration of the message conversion plug-in tool; among them, the system configuration is used to set the JVM parameters, and the business configuration is used to configure the usage of each module.

10. The message conversion plug-in tool according to claim 9, characterized in that The specific implementation steps are as follows: ①Create the core directory, write the main process logic, state transition machine and plug-in entry function; then jump to step ②; ② Create a modules directory, and create subdirectories for each module one by one according to the corresponding relationship: input, transform, output, logs, load-balancing, current-limiting, merge; then, define the standard interface of each module in the subdirectory of each module, and provide a default implementation class based on the standard interface; then jump to step ③; ③Create a conf directory, write the configuration of the middleware that the message conversion plug-in tool depends on into the application.yml file, and migrate the common configuration parameters of each module to the application.yml file; Finally, it is hosted in the configuration center of the message conversion plug-in tool to ensure that changes to application.yml can be dynamically perceived; Then jump to step ④; ④Package the developed message conversion plug-in tool and deploy it on the server; Then jump to step ⑤; ⑤ Select an edge device and modify the message sending address of the edge device to the server where the message conversion plug-in tool is deployed; Then jump to step ⑥; ⑥ When the network is unobstructed, the message conversion plug-in tool receives the message sent by the edge device. Before formally receiving the message, it is necessary to determine whether the various modules of the message conversion plug-in tool are sufficient to support the migration or access process of the edge device; if the judgment result is "sufficient", jump to step ⑦, otherwise jump to step ⑧; ⑦ Modify the configuration of application.yml in the configuration center of the message conversion plug-in tool according to actual needs. The modified configuration includes: the IP, port, identity authentication information, message subject and format, and the selection of each module of the plug-in on the edge device to be migrated or transferred to the cloud platform; then jump to step ⑩; ⑧Enhance each module of the message conversion plug-in tool; then jump to step ⑨; ⑨Upload the extended implementation class through the upload interface of the message conversion plug-in tool, and dynamically load it through the Java reflection mechanism to achieve a smooth upgrade of the message conversion plug-in tool; then jump to step ⑦; ⑩ The message sent by the edge device is dynamically rewritten by the message conversion plug-in tool and sent to the cloud server on the target device, successfully realizing the low-cost migration of the edge device to the cloud platform.

11. The message conversion plug-in tool according to claim 10, characterized in that: In step ⑧, re-implement the standard interfaces provided by each module, or integrate the default implementation class to perform targeted enhancement on the specific functions of the default module.

12. The message conversion plug-in tool according to claim 10 or 11, characterized in that: Step ⑧ is to enhance the message conversion module.

Images