Data processing system for accessing multiple sub-devices based on edge gateway

Abstract

The invention discloses a data processing system for accessing multiple sub-devices based on an edge gateway, belongs to the technical field of Internet, and aims to solve the technical problem of how to realize flexible configuration of data acquisition and analysis, avoid data loss and reduce labor cost at the same time. Directly inputting register information and offset configuration data acquisition and data analysis; the data acquisition module is used for automatically generating an acquisition instruction according to register information configured by a user, finally obtaining a reply message of the equipment, realizing data storage by adopting an sqlite technology, and supporting configuration import and export of sub-equipment; and the data reporting module is used for automatically generating an analysis rule according to register information configured by a user, obtaining sensing data of the equipment and reporting json type data to the cloud platform, heartbeat monitoring and data retransmission are integrated, and the reliability and efficiency of data transmission are guaranteed to the maximum extent.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, in particular to a data processing system based on edge gateway access to multiple sub-devices. Background Art

[0002] With the rapid development of 5G technology, the Internet of Things technology has also developed rapidly in the industrial field. At the same time, how to process and apply data quickly, accurately and conveniently has become a hot topic, and a new device is needed to share some of the functions of the core network.

[0003] In recent years, the Industrial Internet of Things gateway has gradually taken center stage in the industrial application arena. It integrates a series of traditional data processing technologies such as data collection, data parsing, data analysis, and data reporting. With the support of 5G, it has been favored by more and more manufacturing companies. Although the traditional Internet of Things gateway has realized the basic data processing link, it does not have flexible collection instruction configuration and message parsing configuration methods, nor does it have the convenience of large-scale configuration of equipment. Data communication is easily affected by failures, resulting in partial data loss, which increases labor costs and reduces user satisfaction.

[0004] Therefore, how to achieve flexible configuration of data collection and analysis to avoid data loss and reduce labor costs is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The technical task of the present invention is to provide a data processing system based on edge gateway access to multiple sub-devices to solve the problem of how to achieve flexible configuration of data collection and analysis, avoid data loss and reduce labor costs.

[0006] The technical task of the present invention is achieved in the following manner: a data processing system based on edge gateway access to multiple sub-devices, the system comprising:

[0007] The web interface is used to provide management interaction pages, replacing the traditional way of inputting commands and simplifying data collection and data analysis by directly inputting register information and offset configuration;

[0008] The data acquisition module is used to automatically generate acquisition instructions based on the register information configured by the user, and finally obtain the reply message of the device. It also uses SQLite technology to realize data storage and supports the import and export of sub-device configurations.

[0009] The data reporting module is used to automatically generate parsing rules based on the register information configured by the user, obtain the sensor data of the device and report the JSON type data to the cloud platform, integrating heartbeat monitoring and data retransmission to maximize the reliability and efficiency of data transmission.

[0010] Preferably, the WEB interface includes a gateway device configuration sub-interface and a sub-device management sub-interface;

[0011] Among them, the gateway device configuration sub-interface is used to configure the gateway device configuration according to the gateway device information generated by the cloud platform, and the gateway device configuration is used to report the thread;

[0012] The sub-device management sub-interface includes the sub-device configuration table, sub-device data collection configuration table and sub-device data analysis configuration table; the sub-device management sub-interface configures the sub-device name, serial port number, extension address, collection cycle and reporting subject in the sub-device configuration table or arbitrarily adds, deletes and edits device information, and supports multiple sub-device configurations.

[0013] Preferably, the gateway device configuration sub-interface includes a cloud platform access configuration unit and a serial port configuration unit; the serial port configuration unit is used to customize the communication method and device communication protocol.

[0014] Better yet, the sub-device data acquisition configuration table replaces the traditional input command mode and drops the command configuration to the register level; if it is the Modbus protocol, the user only needs to enter the extension address, function code, register start address, number of registers and byte order according to the sensor protocol, and the command generation is completed by the data acquisition thread;

[0015] The sub-device data analysis configuration table visualizes the data analysis information and analyzes the message down to the register level. If it is the Modbus protocol, the user only needs to input the number of registers, the offset relative to the first byte of the message, the conversion coefficient and the data type according to the sensor protocol, and the acquisition thread can intercept and convert the message according to the user-configured acquisition rules. In addition, considering the huge workload of device configuration under large-scale mass production, all device configurations on the WEB interface can be imported and exported as files, which greatly improves the configuration efficiency.

[0016] Preferably, the working process of the WEB interface is as follows:

[0017] (1) Import database configuration;

[0018] (2) The user clicks Create Gateway Device in the Gateway Device Configuration sub-interface;

[0019] (3) The user defines the communication mode and device communication protocol through the serial port configuration unit;

[0020] (4) The user configures the sub-device table and specifies the port and subject information;

[0021] (5) The user configures the sub-device data collection configuration table and sets the register information;

[0022] (6) The user configures the sub-device data parsing configuration table and sets the offset and conversion coefficient;

[0023] (7) Export database configuration.

[0024] Preferably, the data acquisition module is completed by a data acquisition thread, as follows:

[0025] ①Initialize the serial port;

[0026] ② Take out the data collection parameters of the database sub-device and encapsulate them;

[0027] ③Send collection instructions in each collection cycle and monitor the serial port reply message;

[0028] ④Add the message to the data linked list, where each linked list node represents all messages replied by a sub-device;

[0029] If there are multiple sub-devices, steps ① to ④ are executed in a loop.

[0030] Preferably, the data reporting module completes data parsing and data reporting by a data reporting thread; specifically as follows:

[0031] (1) Obtaining platform access configuration and sub-device model information from the database;

[0032] (2) Log in to the cloud platform;

[0033] (3) When parsing data, traverse the linked list and extract the data linked list message;

[0034] (iv) generating parsing rules according to the sub-device parsing configuration table in the database, offset-intercepting the reply message of each command, and obtaining the sensor data required by the user;

[0035] (5) When reporting data, the cjson library is called to encapsulate the sensor data into a json format string;

[0036] (VI) Report the data in JSON format to the cloud platform according to the reporting cycle set in the interface.

[0037] Preferably, the data reporting module integrates heartbeat monitoring function and data retransmission function. When an abnormal MQTT connection is detected, the data will be temporarily stored in the database, the data will be limited to the size range set by the user, and will be continuously updated, and uploaded to the cloud platform as soon as the connection is restored;

[0038] Among them, the reporting thread under connection exception will continue to initiate MQTT connection requests until the connection is successful.

[0039] 5G communication technology: It can provide 5G full-standard connection capabilities, support 5G NSA&SA networking to provide data security, interface security, system authentication and security isolation, etc. While supporting industry attributes such as data acquisition and transmission, protocol analysis, etc., it has 5G native capabilities such as 5G LAN, slicing, and low latency.

[0040] SQLite database technology: SQLite is an open source, lightweight, fast, independent and highly reliable SQL database engine written in C language. It also has the characteristics of stable file format, cross-platform capabilities and backward compatibility. It is widely used in smart devices such as mobile phones, TVs, set-top boxes, game consoles, cameras, watches, etc. It is a very lightweight database solution.

[0041] MQTT technology: MQTT is a lightweight, open messaging protocol that connects the Internet of Things to smart devices. It provides a simple way for resource-constrained network clients to distribute telemetry information in low-bandwidth environments. The protocol uses a publish / subscribe (PUSH / SUBSCRIBE) communication model for machine-to-machine (M2M) communication.

[0042] HTTP technology: HTTP is a protocol used to obtain resources such as HTML documents. It is the basis for any data exchange on the Web and is also a client-server protocol, that is, the request is initiated by the recipient, usually a Web browser. HTTP is an application layer protocol and is sent over TCP or TLS (an encrypted TCP connection).

[0043] The data processing system based on edge gateway access to multiple sub-devices of the present invention has the following advantages:

[0044] (1) The present invention solves the problem that traditional gateways cannot flexibly configure data collection and analysis, improves the flexibility and reliability of gateway configuration, improves gateway application efficiency, and provides a new idea for data application of other types of IoT gateways;

[0045] (ii) The present invention takes into account the huge workload of equipment configuration under large-scale mass production, and all equipment configurations of the WEB interface can be imported and exported as files, which greatly improves the configuration efficiency;

[0046] (III) The present invention provides a flexible data processing method with configurable collection instructions and configurable analysis rules, which improves the convenience of data collection and analysis;

[0047] (iv) The present invention supports importing and exporting database configurations, which greatly reduces deployment costs in practical applications;

[0048] (V) The present invention effectively reduces the probability of abnormal data reporting and improves the situation of data loss during data interaction of the industrial Internet of Things gateway by introducing a heartbeat monitoring mechanism and a data disconnection retransmission mechanism;

[0049] (VI) The present invention supports configurable collection instructions and configurable parsing rules for the interactive interface, supports import and export of interactive interface database configuration, and can be used to equip similar devices on a large scale. At the same time, it introduces a heartbeat monitoring mechanism and a data disconnection retransmission mechanism, which effectively reduces the probability of abnormal data reporting and improves the data loss situation during the data interaction process of the industrial Internet of Things gateway. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The present invention is further described below in conjunction with the accompanying drawings.

[0051] Attached Figure 1 It is a structural diagram of a data processing system based on edge gateway access to multiple sub-devices;

[0052] Attached Figure 2 It is an implementation block diagram of a data processing system based on edge gateway access to multiple sub-devices;

[0053] Attached Figure 3 It is a flowchart of the working process of the WEB interface;

[0054] Attached Figure 4 It is a flowchart of the working process of the data acquisition module;

[0055] Attached Figure 5 This is a flowchart of the working process of the data reporting module. DETAILED DESCRIPTION

[0056] The data processing system based on edge gateway access to multiple sub-devices of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments of the specification.

[0057] Example:

[0058] As attached Figure 1 and 2 As shown, the data processing system based on edge gateway access to multiple sub-devices of the present invention includes:

[0059] The web interface is used to provide management interaction pages, replacing the traditional way of inputting commands and simplifying data collection and data analysis by directly inputting register information and offset configuration;

[0060] The data acquisition module is used to automatically generate acquisition instructions based on the register information configured by the user, and finally obtain the reply message of the device. It also uses SQLite technology to realize data storage and supports the import and export of sub-device configurations.

[0061] The data reporting module is used to automatically generate parsing rules based on the register information configured by the user, obtain the sensor data of the device and report the JSON type data to the cloud platform, integrating heartbeat monitoring and data retransmission to maximize the reliability and efficiency of data transmission.

[0062] The WEB interface in this embodiment includes a gateway device configuration sub-interface and a sub-device management sub-interface;

[0063] Among them, the gateway device configuration sub-interface is used to configure the gateway device configuration according to the gateway device information generated by the cloud platform, and the gateway device configuration is used to report the thread;

[0064] The sub-device management sub-interface includes the sub-device configuration table, sub-device data collection configuration table and sub-device data analysis configuration table; the sub-device management sub-interface configures the sub-device name, serial port number, extension address, collection cycle and reporting subject in the sub-device configuration table or arbitrarily adds, deletes and edits device information, and supports multiple sub-device configurations.

[0065] The gateway device configuration sub-interface in this embodiment includes a cloud platform access configuration unit and a serial port configuration unit; the serial port configuration unit is used to customize the communication mode and device communication protocol.

[0066] The sub-device data acquisition configuration table in this embodiment replaces the traditional input command mode and drops the command configuration to the register level; if it is a Modbus protocol, the user only needs to enter the extension address, function code, register start address, register number and byte order according to the sensor protocol, and the command generation is completed by the data acquisition thread;

[0067] The sub-device data analysis configuration table visualizes the data analysis information and analyzes the message down to the register level. If it is the Modbus protocol, the user only needs to input the number of registers, the offset relative to the first byte of the message, the conversion coefficient and the data type according to the sensor protocol, and the acquisition thread can intercept and convert the message according to the user-configured acquisition rules. In addition, considering the huge workload of device configuration under large-scale mass production, all device configurations on the WEB interface can be imported and exported as files, which greatly improves the configuration efficiency.

[0068] As attached Figure 3 As shown, the working process of the WEB interface in this embodiment is as follows:

[0069] (1) Import database configuration;

[0070] (2) The user clicks Create Gateway Device in the Gateway Device Configuration sub-interface;

[0071] (3) The user defines the communication mode and device communication protocol through the serial port configuration unit;

[0072] (4) The user configures the sub-device table and specifies the port and subject information;

[0073] (5) The user configures the sub-device data collection configuration table and sets the register information;

[0074] (6) The user configures the sub-device data parsing configuration table and sets the offset and conversion coefficient;

[0075] (7) Export database configuration.

[0076] As attached Figure 4 As shown, the data acquisition module in this embodiment is completed by the data acquisition thread, which is as follows:

[0077] ①Initialize the serial port;

[0078] ② Take out the data collection parameters of the database sub-device and encapsulate them;

[0079] ③Send collection instructions in each collection cycle and monitor the serial port reply message;

[0080] ④Add the message to the data linked list, where each linked list node represents all messages replied by a sub-device;

[0081] If there are multiple sub-devices, steps ① to ④ are executed in a loop.

[0082] As attached Figure 5 As shown, the data reporting module in this embodiment completes data parsing and data reporting by the data reporting thread; the details are as follows:

[0083] (1) Obtaining platform access configuration and sub-device model information from the database;

[0084] (2) Log in to the cloud platform;

[0085] (3) When parsing data, traverse the linked list and extract the data linked list message;

[0086] (iv) generating parsing rules according to the sub-device parsing configuration table in the database, offset-intercepting the reply message of each command, and obtaining the sensor data required by the user;

[0087] (5) When reporting data, the cjson library is called to encapsulate the sensor data into a json format string;

[0088] (VI) Report the data in JSON format to the cloud platform according to the reporting cycle set in the interface.

[0089] Preferably, the data reporting module integrates heartbeat monitoring function and data retransmission function. When an abnormal MQTT connection is detected, the data will be temporarily stored in the database, the data will be limited to the size range set by the user, and will be continuously updated, and uploaded to the cloud platform as soon as the connection is restored;

[0090] Among them, the reporting thread under connection exception will continue to initiate MQTT connection requests until the connection is successful.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data processing system based on edge gateway access to multiple sub-devices, characterized in that: The system includes: Web interface, used to provide management interaction page, directly input register information and offset configuration data collection and data parsing; The data acquisition module is used to automatically generate acquisition instructions based on the register information configured by the user, and finally obtain the reply message of the device. It also uses SQLite technology to realize data storage and supports the import and export of sub-device configurations. The data reporting module is used to automatically generate parsing rules based on the register information configured by the user, obtain the sensor data of the device and report the JSON type data to the cloud platform, integrating heartbeat monitoring and data retransmission to maximize the reliability and efficiency of data transmission.

2. The data processing system based on edge gateway access to multiple sub-devices according to claim 1 is characterized in that: The WEB interface includes a gateway device configuration sub-interface and a sub-device management sub-interface; Among them, the gateway device configuration sub-interface is used to configure the gateway device configuration according to the gateway device information generated by the cloud platform, and the gateway device configuration is used to report the thread; The sub-device management sub-interface includes the sub-device configuration table, sub-device data collection configuration table and sub-device data analysis configuration table; the sub-device management sub-interface configures the sub-device name, serial port number, extension address, collection cycle and reporting subject in the sub-device configuration table or arbitrarily adds, deletes and edits device information, and supports multiple sub-device configurations.

3. The data processing system based on edge gateway access to multiple sub-devices according to claim 2 is characterized in that: The gateway device configuration sub-interface includes a cloud platform access configuration unit and a serial port configuration unit; the serial port configuration unit is used to customize the communication method and device communication protocol.

4. The data processing system based on edge gateway access to multiple sub-devices according to claim 2 is characterized in that: The sub-device data acquisition configuration table drops the command configuration to the register level; if it is the Modbus protocol, the user only needs to enter the extension address, function code, register start address, number of registers and byte order according to the sensor protocol, and the command generation is completed by the data acquisition thread; The sub-device data analysis configuration table visualizes the data analysis information and analyzes the message down to the register level. If it is the Modbus protocol, the user only needs to input the number of registers, the offset relative to the first byte of the message, the conversion coefficient and the data type according to the sensor protocol. The acquisition thread can then intercept and convert the message according to the acquisition rules configured by the user.

5. The data processing system based on edge gateway access to multiple sub-devices according to claim 2, characterized in that: The working process of the WEB interface is as follows: (1) Import database configuration; (2) The user clicks Create Gateway Device in the Gateway Device Configuration sub-interface; (3) The user defines the communication mode and device communication protocol through the serial port configuration unit; (4) The user configures the sub-device table and specifies the port and subject information; (5) The user configures the sub-device data collection configuration table and sets the register information; (6) The user configures the sub-device data parsing configuration table and sets the offset and conversion coefficient; (7) Export database configuration.

6. The data processing system based on edge gateway access to multiple sub-devices according to claim 1, characterized in that: The data acquisition module is completed by the data acquisition thread, as follows: ①Initialize the serial port; ② Take out the data collection parameters of the database sub-device and encapsulate them; ③Send collection instructions in each collection cycle and monitor the serial port reply message; ④Add the message to the data linked list, where each linked list node represents all messages replied by a sub-device; If there are multiple sub-devices, steps ① to ④ are executed in a loop.

7. The data processing system based on edge gateway access to multiple sub-devices according to claim 1, characterized in that: The data reporting module completes data analysis and data reporting by the data reporting thread; the details are as follows: (1) Obtaining platform access configuration and sub-device model information from the database; (2) Log in to the cloud platform; (3) When parsing data, traverse the linked list and extract the data linked list message; (iv) generating parsing rules according to the sub-device parsing configuration table in the database, offset-intercepting the reply message of each command, and obtaining the sensor data required by the user; (5) When reporting data, the cjson library is called to encapsulate the sensor data into a json format string; (VI) Report the data in JSON format to the cloud platform according to the reporting cycle set in the interface.

8. The data processing system based on edge gateway access to multiple sub-devices according to claim 7 is characterized in that: The data reporting module integrates heartbeat monitoring and data retransmission functions. When an abnormal MQTT connection is detected, the data will be temporarily stored in the database. The data is limited to the size range set by the user and continuously updated. After the connection is restored, it will be uploaded to the cloud platform as soon as possible. Among them, the reporting thread under connection exception will continue to initiate MQTT connection requests until the connection is successful.

Images