Data processing system based on edge gateway accessing multiple sub-devices

Abstract

The application discloses a kind of data processing systems based on edge gateway access multiple sub-devices, belong to internet technology field, the technical problem that the present application wants to solve is how to realize flexible configuration data acquisition and analysis, avoid data loss while reducing labor cost, the technical scheme for using is: web interface, for providing management interactive page, directly input register information and offset configuration data acquisition and data analysis;Data acquisition module is used to automatically generate acquisition instruction according to the register information configured by user, finally obtains the reply message of device, and realizes data storage using sqlite technology, supports sub-device configuration import and export;Data reporting module is used to automatically generate analysis rule according to the register information configured by user, obtains the sensing data of device and reports json type data to cloud platform, integrates heartbeat monitoring and data retransmission, maximizes the reliability and efficiency of data transmission.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of Internet of Things, in particular to a data processing system based on edge gateway access to multiple sub-devices. BACKGROUND

[0002] With the rapid development of 5G technology, the Internet of Things technology has also developed rapidly in the industrial field. At the same time, how to quickly, accurately and conveniently process and apply data has become a hot topic, and a new device is needed to share part of the functions of the core network.

[0003] Industrial Internet of Things gateway has gradually stood in the center of the industrial application stage in recent years. It integrates a series of traditional data processing technologies such as data collection, data analysis, data analysis, data reporting, etc. Under the support of 5G, it is favored by more and more manufacturing enterprises. Although the traditional Internet of Things gateway has realized the basic data processing link, it still does not have flexible collection instruction configuration and message analysis configuration methods, and it is not convenient to configure a large number of devices. Data communication is easily affected by faults, resulting in partial data loss, increasing labor costs and reducing user satisfaction.

[0004] Therefore, how to realize flexible configuration of data collection and analysis, avoid data loss and reduce labor costs is a technical problem to be solved at present. SUMMARY

[0005] The technical task of the application is to provide a data processing system based on edge gateway access to multiple sub-devices to solve the problem of how to realize flexible configuration of data collection and analysis, avoid data loss and reduce labor costs.

[0006] The technical task of the application is realized in the following manner. A data processing system based on edge gateway access to multiple sub-devices, the system comprises:

[0007] a web interface for providing a management interaction page, replacing the traditional input instruction mode, simplifying the direct input of register information and offset configuration data collection and data analysis;

[0008] a data collection module for automatically generating collection instructions according to user-configured register information, ultimately obtaining the reply message of the device, and using sqlite technology to realize data storage, supporting sub-device configuration import and export;

[0009] a data reporting module for automatically generating analysis rules according to user-configured register information, obtaining sensor data of the device and reporting json-type data to the cloud platform, integrating heartbeat monitoring and data retransmission, and maximizing the reliability and efficiency of data transmission.

[0010] As preferred, the WEB interface comprises a gateway device configuration sub-interface and a sub-device management sub-interface;

[0011] The gateway device configuration sub-interface is configured into the gateway device configuration according to the gateway device information generated by the cloud platform, and the gateway device configuration is used for reporting threads.

[0012] The sub-device management sub-interface comprises a sub-device configuration table, a sub-device data acquisition configuration table, and a sub-device data analysis configuration table. The sub-device management sub-interface configures the sub-device name, serial number, extension address, acquisition period, and reporting topic in the sub-device configuration table, and supports adding, deleting, and editing device information at will. Multiple sub-device configurations are supported.

[0013] More preferably, the gateway device configuration sub-interface comprises a cloud platform access configuration unit and a serial port configuration unit. The serial port configuration unit is used for customizing the communication mode and the device communication protocol.

[0014] More preferably, the sub-device data acquisition configuration table replaces the traditional input instruction mode, and the instruction configuration is dropped to the register level. If it is a Modbus protocol, the user only needs to input the extension address, function code, register start address, register number, and byte sequence according to the sensor protocol. The instruction generation is completed by the data acquisition thread.

[0015] The sub-device data analysis configuration table visualizes the data analysis information, and drops the message analysis to the register level. If it is a Modbus protocol, the user only needs to input the register number, the offset relative to the first byte of the message, the conversion coefficient, and the data type according to the sensor protocol. The acquisition thread can realize the interception and conversion of the message according to the user's configured acquisition rules. In addition, considering the huge device configuration workload under large-scale production, all device configurations of the WEB interface can be imported and exported as files, which greatly improves the configuration efficiency.

[0016] More preferably, the working process of the WEB interface is as follows:

[0017] (1) Import the database configuration;

[0018] (2) The user clicks to create a gateway device through the gateway device configuration sub-interface;

[0019] (3) The user defines the communication mode and the device communication protocol through the serial port configuration unit;

[0020] (4) The user configures the sub-device table and specifies the port and topic information;

[0021] (5) The user configures the sub-device data acquisition configuration table and sets the register information;

[0022] (6) The user configures the sub-device data analysis configuration table and sets the offset and conversion coefficient;

[0023] (7) Export the database configuration.

[0024] As preferred, the data collection module is completed by a data collection thread, specifically as follows:

[0025] ① Initialize the serial port;

[0026] ② Take out the database sub-device data collection parameters and encapsulate them;

[0027] ③ Send a collection instruction every collection period and listen to the serial port reply message;

[0028] ④ Add the message to the data linked list, and each linked list node represents all the messages replied by a sub-device;

[0029] If there are multiple sub-devices, steps ① to ④ are executed in a loop.

[0030] As preferred, the data reporting module completes data analysis and data reporting by a data reporting thread; specifically as follows:

[0031] (1) Obtain platform access configuration and sub-device model information from the database;

[0032] (2) Log in to the cloud platform;

[0033] (3) When analyzing data, traverse the linked list and extract the data linked list message;

[0034] (4) According to the sub-device analysis configuration table in the database, generate an analysis rule, offset and intercept the reply message of each command, and obtain the sensor data required by the user;

[0035] (5) When reporting data, call the cjson library to encapsulate the sensor data into a json format string;

[0036] (6) According to the reporting period set by the interface, report the json format data to the cloud platform.

[0037] More preferably, the data reporting module integrates a heartbeat monitoring function and a data retransmission function. When an MQTT connection exception is detected, the data will be temporarily stored in the database, the data size will be limited within the range set by the user, and the data will be continuously updated. When the connection is restored, the data will be uploaded to the cloud platform at the first time;

[0038] Among them, the reporting thread under the connection exception will continuously initiate an MQTT connection request until the connection is successful.

[0039] 5G communication technology: can provide full-mode connection capability of 5G, support 5G NSA&SA networking to provide data security, interface security, system authentication and security isolation, etc. While supporting data acquisition, data transmission, protocol analysis and other industry attributes, it has 5G LAN, slicing, low latency and other 5G native capabilities.

[0040] SQLite database technology: SQLite is an open source, lightweight, fast, independent and highly reliable SQL database engine written in C language, also has stable file format, cross-platform capability and backward compatible characteristics, is widely used in mobile phones, televisions, set-top boxes, game consoles, cameras, watches and other smart devices, is a very lightweight database solution.

[0041] MQTT technology: MQTT is a leading open source protocol for connecting Internet of Things in the field of intelligent devices - MQTT is a lightweight open message delivery protocol that provides a simple method for resource-constrained network clients to distribute telemetry information in low-bandwidth environments. The protocol uses a publish / subscribe (PUSH / SUBSCRIBE) communication mode for machine-to-machine (M2M) communication.

[0042] HTTP technology: HTTP is a protocol used to retrieve resources such as HTML documents. It is the basis for any data exchange on the Web, and is also a client-server protocol, that is, the request is initiated by the recipient, usually a web browser. HTTP is an application layer protocol that is sent over TCP or TLS (an encrypted TCP connection).

[0043] The data processing system based on the edge gateway accessing multiple sub-devices of the application has the following advantages:

[0044] (I) The application solves the problem that the traditional gateway cannot be flexibly configured for data acquisition and analysis, improves the flexibility and reliability of gateway configuration, improves the application efficiency of the gateway, and provides a new idea for data application of other types of Internet of Things gateways;

[0045] (II) The application considers that the device configuration workload is huge under large-scale production, and all device configurations of the WEB interface can be imported and exported as files, greatly improving the configuration efficiency;

[0046] (III) The application provides a flexible data processing method of configurable acquisition instructions and configurable analysis rules, which improves the convenience of data acquisition and analysis;

[0047] (IV) The application supports database configuration import and export, which greatly reduces the deployment cost in actual application;

[0048] (v) By introducing a heartbeat monitoring mechanism and a data disconnection retransmission mechanism, this invention effectively reduces the probability of abnormal data reporting and improves the situation of data loss during data interaction of industrial IoT gateways.

[0049] (vi) This invention supports configurable acquisition commands and configurable parsing rules in the interactive interface, supports importing and exporting database configurations in the interactive interface, and can be used to equip similar devices on a large scale. At the same time, it introduces a heartbeat monitoring mechanism and a data disconnection retransmission mechanism, which effectively reduces the probability of abnormal data reporting and improves the situation of data loss during the data interaction of industrial IoT gateways. Attached Figure Description

[0050] The invention will be further described below with reference to the accompanying drawings.

[0051] Appendix Figure 1 This is a schematic diagram of the structure of a data processing system that accesses multiple sub-devices based on an edge gateway;

[0052] Appendix Figure 2 This is an implementation block diagram of a data processing system based on edge gateway access to multiple sub-devices;

[0053] Appendix Figure 3 A flowchart illustrating the working process of a web interface;

[0054] Appendix Figure 4 A flowchart illustrating the working process of the data acquisition module;

[0055] Appendix Figure 5 This is a flowchart illustrating the workflow of the data reporting module. Detailed Implementation

[0056] The data processing system for accessing multiple sub-devices based on an edge gateway according to the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] Example:

[0058] As attached Figure 1 and 2 As shown, the data processing system of the present invention based on edge gateway access to multiple sub-devices includes:

[0059] A web interface is provided to offer a management interaction page, replacing the traditional input command method and simplifying it to directly input register information and offset configuration data acquisition and data parsing.

[0060] The data acquisition module is used to automatically generate acquisition commands based on the register information configured by the user, and finally obtain the response message from the device. It uses SQLite technology to realize data storage and supports the import and export of sub-device configurations.

[0061] Data reporting module, for automatically generating parsing rules according to the user configured register information, obtaining the sensing data of the device and reporting the json type data to the cloud platform, integrating heartbeat monitoring and data retransmission, and maximizing the reliability and efficiency of data transmission.

[0062] The WEB interface in the embodiment includes a gateway device configuration sub-interface and a sub-device management sub-interface.

[0063] The gateway device configuration sub-interface is configured into the gateway device configuration according to the gateway device information generated by the cloud platform, and the gateway device configuration is used for reporting threads.

[0064] The sub-device management sub-interface includes a sub-device configuration table, a sub-device data acquisition configuration table and a sub-device data parsing configuration table. The sub-device management sub-interface configures the sub-device name, serial number, extension address, acquisition period and reporting theme in the sub-device configuration table, and supports adding, deleting and editing device information at will, and supports multiple sub-device configurations.

[0065] The gateway device configuration sub-interface in the embodiment includes a cloud platform access configuration unit and a serial port configuration unit. The serial port configuration unit is used for customizing the communication mode and the device communication protocol.

[0066] The sub-device data acquisition configuration table in the embodiment replaces the traditional input instruction mode, and configures the instruction to the register level. If it is a Modbus protocol, the user only needs to input the extension address, function code, register start address, register number and byte sequence according to the sensor protocol, and the instruction generation is completed by the data acquisition thread.

[0067] The sub-device data parsing configuration table visualizes the data parsing information, and configures the message parsing to the register level. If it is a Modbus protocol, the user only needs to input the register number, offset relative to the first byte of the message, conversion coefficient and data type according to the sensor protocol, and the acquisition thread can realize the interception and conversion of the message according to the user configured acquisition rule. In addition, considering the huge device configuration workload under large-scale production, all device configurations of the WEB interface can be imported and exported as files, which greatly improves the configuration efficiency.

[0068] As shown in the accompanying Figure 3 The working process of the WEB interface in the embodiment is as follows:

[0069] (1) Import the database configuration;

[0070] (2) The user clicks to create a gateway device through the gateway device configuration sub-interface;

[0071] (3) The user defines the communication mode and the device communication protocol through the serial port configuration unit;

[0072] (4) The user configures the sub-device table and specifies the port and theme information;

[0073] (5) The user configures the sub-device data acquisition configuration table and sets the register information;

[0074] (6) The user configures the sub-device data analysis configuration table and sets the offset and conversion coefficient;

[0075] (7) Export the database configuration.

[0076] As shown in the accompanying Figure 4 , the data acquisition module in the embodiment is completed by a data acquisition thread, and specifically as follows:

[0077] ① Initialize the serial port;

[0078] ② Take out the database sub-device data acquisition parameters and encapsulate them;

[0079] ③ Send the acquisition instruction every acquisition cycle and listen to the serial port reply message;

[0080] ④ Add the message to the data linked list, and each linked list node represents all the messages replied by a sub-device;

[0081] If there are multiple sub-devices, steps ① to ④ are executed in a loop.

[0082] As shown in the accompanying Figure 5 , the data reporting module in the embodiment completes data analysis and data reporting by a data reporting thread; specifically as follows:

[0083] (1) Obtain the platform access configuration and sub-device model information from the database;

[0084] (2) Log in to the cloud platform;

[0085] (3) When data analysis, traverse the linked list and extract the data linked list message;

[0086] (4) According to the sub-device analysis configuration table in the database, generate the analysis rule, offset and intercept the reply message of each command, and obtain the sensor data required by the user;

[0087] (5) When data reporting, call the cjson library to encapsulate the sensor data into a json format string;

[0088] (6) According to the reporting period set by the interface, report the json format data to the cloud platform.

[0089] More preferably, the data reporting module integrates heartbeat monitoring function and data retransmission function. When the MQTT connection is monitored to be abnormal, the data is temporarily stored in the database, the data is limited in the size range set by the user, and is continuously updated, and is uploaded to the cloud platform at the first time after the connection is restored.

[0090] Among them, the reporting thread under the connection exception will continue to initiate the MQTT connection request until the connection is successful.

[0091] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing system based on edge gateway accessing multiple sub-devices, characterized in that, The system comprises: a web interface for providing a management interaction page, direct input of register information and offset configuration data collection and data analysis; a data collection module for automatically generating a collection instruction according to the register information configured by a user, finally obtaining a reply message of a device, and realizing data storage by using a sqlite technology, supporting import and export of a sub-device configuration; a data reporting module for automatically generating a parsing rule according to the register information configured by the user, obtaining sensor data of the device and reporting the json type data to a cloud platform, integrating heartbeat monitoring and data retransmission, and maximizing the reliability and efficiency of data transmission; wherein the web interface comprises a gateway device configuration sub-interface and a sub-device management sub-interface; wherein the gateway device configuration sub-interface is used for configuring gateway device information generated by the cloud platform into a gateway device configuration, and the gateway device configuration is used for reporting threads; the sub-device management sub-interface comprises a sub-device configuration table, a sub-device data collection configuration table and a sub-device data analysis configuration table; the sub-device management sub-interface configures a sub-device name, a serial number, an extension address, a collection period and a reporting topic or any increase, deletion or editing of device information in the sub-device configuration table, and supports multiple sub-device configurations; the sub-device data collection configuration table configures instructions to fall to a register level; if it is a Modbus protocol, the user only needs to input an extension address, a function code, a register start address, a register number and a byte sequence according to a sensor protocol, and the instruction generation is completed by a data collection thread; the sub-device data analysis configuration table visualizes data analysis information and configures a message analysis to fall to a register level; if it is a Modbus protocol, the user only needs to input a register number, an offset relative to a message first byte, a conversion coefficient and a data type according to a sensor protocol, and the collection thread can realize message interception and conversion according to the collection rule configured by the user.

2. The data processing system based on edge gateway accessing multiple sub-devices according to claim 1, wherein, The gateway device configuration sub-interface comprises a cloud platform access configuration unit and a serial port configuration unit; the serial port configuration unit is used for customizing a communication mode and a device communication protocol.

3. The data processing system based on edge gateway accessing multiple sub-devices according to claim 1 or 2, characterized in that, The working process of the web interface is as follows: (1) import a database configuration; (2) the user clicks to create a gateway device through the gateway device configuration sub-interface; (3) the user defines a communication mode and a device communication protocol through the serial port configuration unit; (4) the user configures a sub-device table and specifies port and topic information; (5) the user configures a sub-device data collection configuration table and sets register information; (6) the user configures a sub-device data analysis configuration table and sets an offset and a conversion coefficient; (7) export a database configuration.

4. The data processing system based on edge gateway accessing multiple sub-devices according to claim 1, wherein, The data collection module is completed by a data collection thread, and specifically as follows: ① initialize a serial port; ② take out database sub-device data collection parameters and encapsulate them; ③ send a collection instruction every collection period and listen to a serial port reply message; ④ add the message to a data linked list, and each linked list node represents all messages replied by a sub-device; wherein if there are multiple sub-devices, steps ① to ④ are executed in a loop.

5. The data processing system based on edge gateway accessing multiple sub-devices according to claim 1, characterized in that, The data reporting module completes data analysis and data reporting by a data reporting thread; specifically as follows: (I) Obtain platform access configuration and sub-device model information from the database; (II) Log in to the cloud platform; (III) When data is parsed, traverse the linked list and extract the data linked list message; (IV) Generate parsing rules according to the sub-device parsing configuration table in the database, and offset and intercept the reply message of each command to obtain the sensor data required by the user; (V) When data is reported, call the cjson library to encapsulate the sensor data into a json format string; (VI) According to the reporting period set by the interface, report the json format data to the cloud platform.

6. The data processing system based on edge gateway accessing multiple sub-devices according to claim 5, wherein, The data reporting module integrates heartbeat monitoring function and data retransmission function. When the MQTT connection is abnormal, the data will be temporarily stored in the database, the data will be limited within the size range set by the user, and the data will be continuously updated. When the connection is restored, the data will be uploaded to the cloud platform at the first time; Among them, the reporting thread under the connection exception will continuously initiate the MQTT connection request until the connection is successful.

Images