A data transmission and decompression method and device based on measurement point table

Abstract

The present application relates to the field of Internet of Things (IoT) data decompression technology, and discloses a data transmission and decompression method and device based on a measurement point table. The method comprises configuring a measurement point table; downloading the configured measurement point table to a gateway or data transmission unit IoT device of a target platform; obtaining and temporarily storing the measurement point data of the monitored device based on the measurement point table, and determining whether a preset upload data interval has been reached; creating a binary measurement point data stream when the upload data interval has been reached; encoding the binary measurement point data stream to obtain an encoding value; using the encoding value as the value of the measurement point data stream and combining it with a preset upload data type to create a compressed data packet; creating a protocol frame based on a preset data transmission protocol type and using the compressed data packet as a protocol payload; and uploading the protocol frame to the target platform. The present application has the effect of improving the transmission efficiency of IoT data and enhancing the applicability of the compression algorithm.

Description

Technical Field

[0001] The present application relates to the technical field of Internet of Things data decompression, and in particular to a data transmission and decompression method and device based on a measurement point table. Background Art

[0002] Currently, the JSON data format is widely used for IoT data transmission using the MQTT protocol due to its lightweight nature, ease of reading and writing, and ease of machine parsing and generation. To facilitate readability and machine parsing, the standard JSON data format uses recurring identifiers, such as key-value pair separators and key-value separators.

[0003] In the prior art, the CJSON compression algorithm is used to compress JSON data, and the unordered set of key-value pairs is organized into the form of a template and a value; or the HPack compression algorithm is used to extract the key and value from the unordered set of key-value pairs. The first value in the resulting array is the template of the HPack compression algorithm, and the subsequent values ​​are in order. However, during data transmission, the repeated identifiers in the JSON data will affect the data transmission efficiency and waste network resources. In particular, the number of devices connected to the Internet of Things platform is in the tens of thousands, and the amount of data that needs to be uploaded is very large. The data transmission efficiency is low, and the occupation of network resources is further aggravated. Furthermore, due to the inconsistent key-value pair formats of the JSON data based on the measurement point table, the CJSON compression algorithm and the HPack compression algorithm are not suitable for JSON data compression based on the measurement point table.

[0004] In view of the above-mentioned related technologies, the inventors found that the existing Internet of Things data transmission methods have the problems of low data transmission efficiency, large network resource occupation and poor applicability. Summary of the Invention

[0005] In order to improve the transmission efficiency of IoT data, reduce the use of network resources, and enhance the applicability of compression algorithms, the present application provides a data transmission and decompression method and device based on a measurement point table.

[0006] In a first aspect, the present application provides a data transmission and decompression method based on a measurement point table.

[0007] This application is achieved through the following technical solutions:

[0008] A data transmission and decompression method based on a measuring point table includes the following steps:

[0009] Configure a measurement point table, which includes the gateway or data transmission unit IoT device attributes, other IoT device attributes, monitored device attributes, measurement point attributes, and a measurement point table check code;

[0010] Download the configured measurement point table to the target platform’s gateway or data transmission unit IoT device;

[0011] Based on the measuring point table, obtaining and temporarily storing the measuring point data of the monitored equipment, and determining whether a preset data upload interval has been reached;

[0012] When the data upload interval is reached, the measurement point data is converted into a binary measurement point data stream in the form of measurement point sequence numbers and measurement point values;

[0013] Performing data encoding on the binary measurement point data stream to obtain an encoding value;

[0014] Using the coded value as the value of the measurement point data stream and combining it with a preset upload data type, a compressed data packet is created;

[0015] Creating a protocol frame using the compressed data packet as a protocol payload according to a preset data transmission protocol type;

[0016] The protocol frame is uploaded to the target platform.

[0017] In a preferred example, the present application can be further configured as follows: the protocol frame includes the device unique identifier of the gateway or data transmission unit of the target platform, the current timestamp of the uploaded data, the byte length value of the measurement point sequence number, the coding value including the measurement point sequence number and the measurement point value, the total number of uploaded measurement point data and the measurement point table check code.

[0018] In a preferred example, the present application can be further configured as follows:

[0019] Acquire a device unique identifier of the gateway or the data transmission unit of the target platform from the protocol frame;

[0020] Matching a measurement point table pre-stored on the target platform according to the unique identifier of the device;

[0021] If the measuring point table exists, determining whether a measuring point table check code in the measuring point table is consistent with a measuring point table check code in the protocol frame;

[0022] If the measurement point table check code in the measurement point table is consistent with the measurement point table check code in the protocol frame, the protocol frame is converted into a corresponding binary measurement point data stream, and the measurement point data of the monitored device is decompressed in the form of the measurement point sequence number and the measurement point value, combined with the configured upload data type.

[0023] In a preferred example, the present application may be further configured as follows: when the protocol frame exists, after the step of determining whether the measurement point table check code in the measurement point table is consistent with the measurement point table check code in the protocol frame, the step further includes:

[0024] If the measurement point table check code in the measurement point table is inconsistent with the measurement point table check code in the protocol frame, the protocol frame is discarded and the application layer is notified of a data frame error.

[0025] In a preferred example, the present application may be further configured as follows: after the step of matching the measurement point table pre-stored in the target platform according to the unique identifier of the device, the method further includes:

[0026] If there is no matching result in the measurement point table, the protocol frame is discarded and the application layer is notified of the data frame error.

[0027] In a second aspect, the present application provides a data transmission and decompression device based on a measurement point table.

[0028] This application is achieved through the following technical solutions:

[0029] A data transmission and decompression device based on a measuring point table, comprising:

[0030] A configuration module is used to configure a measurement point table, wherein the measurement point table includes the attributes of the gateway or data transmission unit IoT device, the attributes of other IoT devices, the attributes of the monitored device, the attributes of the measurement point, and the measurement point table check code;

[0031] Initialization module, used to download the configured measurement point table to the target platform’s gateway or data transmission unit IoT device;

[0032] A temporary storage module is used to obtain and temporarily store the measurement point data of the monitored equipment based on the measurement point table, and to determine whether a preset data upload interval has been reached;

[0033] A binary module is used to create a binary measurement point data stream according to the measurement point sequence number and measurement point value of the measurement point data when the data upload interval is reached;

[0034] An encoding module, configured to perform data encoding on the binary measurement point data stream to obtain an encoding value;

[0035] A compression module, configured to create a compressed data packet using the coded value as the value of the measurement point data stream in combination with a preset upload data type;

[0036] A frame module, configured to create a protocol frame using the compressed data packet as a protocol payload according to a preset data transmission protocol type;

[0037] The uploading module is used to upload the protocol frame to the target platform.

[0038] In a preferred example, the present application can be further configured as follows:

[0039] an identification module, configured to obtain a unique device identifier of the gateway or the data transmission unit of the target platform from the protocol frame;

[0040] A matching module, configured to match a measurement point table pre-stored on a target platform according to the unique identifier of the device;

[0041] a verification module, configured to determine, when the measuring point table exists, whether a measuring point table verification code in the measuring point table is consistent with a measuring point table verification code in the protocol frame;

[0042] a decompression module configured to convert the protocol frame into a corresponding binary measurement point data stream when the measurement point table check code in the measurement point table is consistent with the measurement point table check code in the protocol frame, and decompress the measurement point data of the monitored device in the form of a measurement point sequence number and a measurement point value in combination with the configured upload data type.

[0043] In a third aspect, the present application provides a computer device.

[0044] This application is achieved through the following technical solutions:

[0045] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the processor implements the steps of any one of the above-mentioned data transmission and decompression methods based on a measurement point table.

[0046] In a fourth aspect, the present application provides a computer-readable storage medium.

[0047] This application is achieved through the following technical solutions:

[0048] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements the steps of any one of the above-mentioned data transmission and decompression methods based on a measurement point table.

[0049] In a fifth aspect, the present application provides a computer program product.

[0050] This application is achieved through the following technical solutions:

[0051] A computer program product includes a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of any one of the above-mentioned data transmission and decompression methods based on a measurement point table.

[0052] In summary, compared with the prior art, the technical solution provided by this application has at least the following beneficial effects:

[0053] Configure the measuring point table so that the gateway or data transmission unit can automatically collect the measuring point data of the IoT device itself and the monitored device according to the measuring point information of the configured measuring point table; download the configured measuring point table to the gateway or data transmission unit IoT device of the target platform to complete the preparation work; based on the measuring point table, obtain and temporarily store the measuring point data of the monitored device to realize the automatic collection of measuring point data, and at the same time judge whether the preset upload data interval is reached to avoid the IoT data from being continuously uploaded and occupying the network bandwidth; when the upload data interval is reached, create a binary measuring point data stream in the form of measuring point sequence number and measuring point value, use the measuring point sequence number to distinguish each measuring point and its upload order, reduce the key definition and measuring point value separator, thereby saving the transmission amount of measuring point data, improving data transmission efficiency, and also For subsequent data encoding, it can also flexibly and disorderly combine the uploaded measurement point data; encode the binary measurement point data stream to obtain the encoding value to realize the conversion from binary to character, further reducing the data volume; use the encoding value as the value of the measurement point data stream, combined with the preset upload data type, to create a compressed data packet, further compressing the data volume of the measurement point data; according to the preset data transmission protocol type, use the compressed data packet as the protocol payload to create a protocol frame, and upload it to the target platform to complete the transmission of the Internet of Things measurement point data; and then a data transmission and decompression method based on the measurement point table can improve the transmission efficiency of the Internet of Things data, save transmission bandwidth, reduce the use of network resources, reduce operating costs, especially significantly reduce the cost of cellular mobile networks, and enhance the applicability of the compression algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A basic architecture diagram of an Internet of Things system providing a data transmission and decompression method based on a measurement point table, provided as an exemplary embodiment of the present application.

[0055] Figure 2 A transmission flow diagram of a data transmission and decompression method based on a measurement point table is provided as another exemplary embodiment of the present application.

[0056] Figure 3 A schematic diagram of a decompression process of a data transmission and decompression method based on a measurement point table is provided as another exemplary embodiment of the present application.

[0057] Figure 4 A structural block diagram of a data transmission and decompression device based on a measurement point table provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0058] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed, but as long as they are within the scope of the claims of the present application, they are protected by the patent law.

[0059] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0060] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document, unless otherwise specified, generally indicates an "or" relationship between the related objects.

[0061] JSON is a lightweight data interchange format. Based on a subset of ECMAScript, it uses a text format that is completely independent of programming languages ​​to store and represent data. It is easy for humans to read and write, as well as easy for machines to parse and generate, and effectively improves network transmission efficiency. JSON consists of two data structures: objects and arrays. Objects are unordered collections of key-value pairs, where the key is a string and the value can be of any type. Objects are surrounded by a pair of curly braces "{}", with the key and value separated by a colon ":". Key-value pairs are separated by a comma ","; arrays are ordered collections of values, each of which can be of any type. Arrays are surrounded by a pair of square brackets "[]", with values ​​separated by a comma ",". The simplicity and clear hierarchical structure make JSON an ideal data interchange language. JSON is a text-based, lightweight data interchange format that does not support the direct transmission of binary data.

[0062] The MQTT protocol is a "lightweight" communication protocol based on the publish / subscribe model, which is built on the TCP / IP protocol. The MQTT protocol is designed for communication between a large number of remote sensors and control devices with limited computing power and working on low-bandwidth, unreliable networks. Features of the MQTT protocol: (1) Using the publish / subscribe messaging model, it provides one-to-many message publishing and decouples applications; (2) Message transmission that shields payload content; (3) Using TCP / IP to provide network connectivity; (4) There are three message publishing service qualities: (5) Small transmission, low overhead, and minimized protocol exchanges to reduce network traffic; (6) A mechanism that uses the Last Will and Testament features to notify relevant parties of client abnormal interruptions. The MQTT protocol is widely used in the field of the Internet of Things for communication between sensors and servers, information collection, etc. The JSON data format is widely used in the information transmission based on the MQTT protocol in the Internet of Things.

[0063] However, when using the JSON data format to transmit IoT data, repeated identifiers can affect data transmission efficiency and waste network resources. Furthermore, IoT platforms connect to tens of thousands of devices, requiring massive amounts of data to be uploaded, resulting in low data transmission efficiency. IoT network bandwidth is extremely limited, and the higher the bandwidth, the higher the cost, further increasing network resource utilization. In particular, using cellular networks for transmission results in even greater data waste, significantly increasing service charges and operational costs.

[0064] Although the CJSON compression algorithm and the HPack compression algorithm can significantly improve the transmission efficiency of IoT data and save network resources in scenarios with consistent key-value pair formats, for the special application scenario of JSON data format compression based on the measurement point table, the existing data compression algorithms are not suitable for JSON data compression based on the measurement point table due to the inconsistent key-value pair format of the JSON data based on the measurement point table.

[0065] In order to improve the transmission efficiency of measurement point data, save network resources, save cellular mobile network traffic and charges, and improve the applicability of compression algorithms, this application designs a JSON data compression method based on the measurement point table. On the basis of following the JSON data format, the standard JSON data is compressed at the expense of a certain degree of readability, thereby improving the transmission efficiency of IoT data, saving transmission bandwidth, and saving operating costs.

[0066] The embodiments of the present application are described in further detail below with reference to the accompanying drawings.

[0067] Reference Figure 1, an embodiment of the present application provides a data transmission and decompression method based on a measurement point table, and its Internet of Things system includes an Internet of Things platform, an Internet of Things device and a monitored device. The monitored devices include temperature sensors, water meters, electricity meters, air conditioners, transformers, etc. The monitored devices are devices monitored by the Internet of Things platform. Internet of Things devices include gateways, data transmission units DTU, LoRa terminals, etc. The Internet of Things devices collect data from the monitored devices and transmit it upward to the Internet of Things platform. The Internet of Things platform is an integrated platform that integrates functions such as device access, device management, data security communication, message subscription, message forwarding and data services. The Internet of Things platform supports connecting to a large number of monitored devices downward and collecting data from the monitored devices and uploading it to the cloud.

[0068] The main steps of a data transmission and decompression method based on a measurement point table are described as follows.

[0069] Configure a measurement point table, which includes the gateway or data transmission unit IoT device attributes, other IoT device attributes, monitored device attributes, measurement point attributes, and a measurement point table check code;

[0070] Download the configured measurement point table to the target platform’s gateway or data transmission unit IoT device;

[0071] Based on the measuring point table, obtaining and temporarily storing the measuring point data of the monitored equipment, and determining whether a preset data upload interval has been reached;

[0072] When the data upload interval is reached, the measurement point data is converted into a binary measurement point data stream in the form of measurement point sequence numbers and measurement point values;

[0073] Performing data encoding on the binary measurement point data stream to obtain an encoding value;

[0074] Using the coded value as the value of the measurement point data stream and combining it with a preset upload data type, a compressed data packet is created;

[0075] Creating a protocol frame using the compressed data packet as a protocol payload according to a preset data transmission protocol type;

[0076] The protocol frame is uploaded to the target platform.

[0077] Specifically, a measurement point table is configured on the IoT platform. The measurement point table includes gateway or data transmission unit (DTU) IoT device attributes, other IoT device attributes, monitored device attributes, measurement point attributes, and a measurement point table checksum. Gateway or data transmission unit (DTU) IoT device attributes include the gateway or DTU device identifier, communication address, device type, and device ID. Other IoT device attributes include other IoT device identifiers, communication address, device type, and device ID. Monitored device attributes include the monitored device identifier, communication address, device type, parent IoT device ID, communication interface parameters, and device protocol type. Measurement point attributes include the device to which it belongs, data type collected, measurement point address collected, collection command word, upload data type, and upload interval. The measurement point table checksum can be a CRC16 checksum to ensure the accuracy and uniqueness of the measurement point table.

[0078] In this embodiment, the gateway or data transmission unit IoT device is the IoT root device, directly communicating with the platform. Other IoT devices are wireless networking sub-devices centered around the gateway, such as LoRa nodes or Zigbee nodes. Together with the LoRa gateway module or Zigbee gateway module in the gateway, they form a wireless network. Data collected by monitored devices connected to the nodes is transmitted over the wireless network to the gateway, which then creates protocol frames for upload to the IoT platform. In the measurement point table, the gateway or data transmission unit IoT device and other IoT devices are used to describe the attributes associated with the root device.

[0079] By expressing the different attributes of different devices using digital language, different devices can be identified by identifying different attributes to monitor device information. When configuring the measurement point table, set the measurement point attributes based on the measurement point information to be monitored. In the measurement point table, follow the module rules, fill in the template of the target measurement point, enter the measurement point data attributes and the measurement point table check code.

[0080] The measurement point table is a configuration file used to configure the properties of the measurement points collected by the gateway or DTU IoT device and the monitored device, so that the gateway or DTU can automatically collect measurement point data based on the configured measurement point information.

[0081] Next, download the configured measurement point table to the target platform's gateway or data transmission unit IoT device. The IoT platform creates configuration or control commands, compresses them in a standard JSON data format, and sends them to the gateway or DTU IoT device using the MQTT protocol. The gateway or DTU IoT device then compresses the results of the configuration or control commands in a standard JSON data format and uploads them to the IoT platform using the MQTT protocol.

[0082] Reference Figure 2 ,According to the measuring point table, the gateway or data transmission unit ,IoT device uses the pre-configured collection port, protocol type and other ,configuration information to continuously and cyclically collect the ,measurement point data of all the measuring points defined in the ,measurement point table, and temporarily stores the ,measurement point data (sampling values) of the monitored ,device into RAM.

[0083] Determine whether the preset data upload interval has been reached.

[0084] When the upload interval for a measurement point meets the requirements, the data upload interval is determined to have been reached. The temporarily stored measurement point data is then converted into a binary measurement point data stream in the form of a measurement point sequence number and a measurement point value. The measurement point sequence number identifies each measurement point and its upload order. The measurement point sequence number represents the order in which the measurement point is arranged in the measurement point table. If the number of acquisition points defined in the measurement point table is less than 256, the measurement point sequence number is 1 byte; if the number of acquisition points is greater than or equal to 256, the measurement point sequence number is 2 bytes. All uploaded measurement point values ​​are treated as a single value, and the "," separator between measurement point values ​​is omitted. This reduces the number of measurement point value separators and saves significant data transmission. Each uploaded measurement point value is presented as a combination of "measurement point sequence number + measurement point value," allowing for flexible and unordered combination of uploaded measurement point values.

[0085] Perform data encoding on the binary measurement point data stream, such as using base64 encoding, to obtain the encoded value. Base64 is one of the most common encoding methods used to transmit 8-bit bytecode on the Internet. Base64 represents binary data based on 64 printable characters. Base64 includes a 64-character character set: lowercase letters a-z, uppercase letters A-Z, digits 0-9, and the symbols "+" and " / ". Any symbol can be converted to a character in this character set. This conversion process is called base64 encoding. Base64 encoding converts binary data into characters and can be used to transmit longer identifiers in HTTP environments. After base64 encoding, data must be decoded before it can be read. The original JSON data format is essentially a text format and can only be transmitted in character format, not binary format. Converting 1 byte of binary data to ASCII character data requires 2 bytes; converting 3 bytes of binary data to ASCII character data requires 6 bytes; however, converting 3 bytes of binary data to base64 encoding requires only 4 bytes. Therefore, using base64 encoding for binary data can save 33% of data compared to using ASCII encoding.

[0086] Use the above encoding value as the value of the measurement point data stream, combine it with the preset upload data type, create a compressed data packet, and create the collected measurement point data into a compressed JSON data format.

[0087] Based on the preset data transmission protocol type, a compressed data packet is used as the protocol payload to create a protocol frame, and the protocol frame is uploaded to the gateway or data transmission unit of the target platform. If the MQTT protocol is used to upload measurement point data, a compressed JSON data packet is used as the MQTT protocol payload to create an MQTT protocol frame. The measurement point MQTT protocol frame is then uploaded to the IoT platform via Ethernet / cellular mobile network / Wi-Fi.

[0088] In one embodiment, the protocol frame includes the device unique identifier of the gateway or data transmission unit of the target platform, the current timestamp of the uploaded data, the byte length value of the measurement point sequence number, the coding value including the measurement point sequence number and the measurement point value, the total number of uploaded measurement point data and the measurement point table check code.

[0089] For example, {"g":"device id","t":time stamp,"l":point serial number length,"v":"base64 code data","c":points number,"j":crc16}

[0090] The instructions are as follows:

[0091] ① "g": "device id": The unique identifier of the gateway or DTU device that uploads data to the platform. The device unique identifier is a numeric string. The platform must save the measurement point table downloaded to each gateway or DTU device, and the measurement point table saved by the platform must be consistent with the measurement point table downloaded to the gateway or DTU device to ensure consistency between the platform and the gateway or DTU device.

[0092] ②"t":time stamp: the timestamp when uploading data, a decimal number.

[0093] ③"l": point serial number length: The length of the point serial number in bytes. In order for the platform to correctly parse the uploaded point information, it is a decimal number with a value of 1 or 2. If the total number of points defined in the point table is less than 256, the value is 1; if the total number of points defined in the point table is greater than or equal to 256, the value is 2.

[0094] ④"v":"base64 code data": upload base64-encoded measurement point value data stream. Each measurement point consists of: measurement point sequence number + measurement point value.

[0095] ⑤"c":points number: the total number of measurement points uploaded at that time, a decimal number.

[0096] ⑥"j":crc16: CRC16 check value of the measurement point table, which is a decimal number. It is used by the platform to determine whether the measurement point table of the uploaded data is consistent with the measurement point table saved by the platform. If they are inconsistent, the uploaded measurement point data will not be parsed. If they are consistent, the uploaded measurement point data will be parsed.

[0097] By setting the compression format of the protocol frame, the measurement point sequence number is used to distinguish each measurement point and its upload order. All uploaded measurement points are defined as a single key. Compared to defining a key for each measurement point, this eliminates the need for multiple key definitions. All uploaded measurement point values ​​are treated as a single value, eliminating the "," separator between measurement point values ​​and significantly reducing transmission data. If the number of acquisition points defined in the measurement point table is less than 256, the measurement point sequence number is 1 byte; if the number of acquisition points is greater than or equal to 256, the measurement point sequence number is 2 bytes. Using base64 to encode binary measurement point data saves even more data than using ASCII encoding. The IoT platform parses the measurement point values ​​based on the measurement point table and verifies the measurement point data based on the table's CRC16 checksum.

[0098] Assuming that 100 measurement point data are reported, the standard JSON format of the MQTT protocol frame upload format is as follows:

[0099] {"g":"2007777777","t":1615511830,"m":[{"i":"FF0001","v":xxxxxxxx},…,{"i":"yyyy","v":xxxxxxxx}]}

[0100] Each measurement point requires 28 bytes, and 100 measurement points require 37+28*100=2837 bytes. 37 represents fixed payload information, namely "{"g":"2007777777","t":1615511830,"m":[", and "]}".

[0101] The format for uploading MQTT protocol frames in compressed JSON format is as follows:

[0102] {"g":"2007777777","t":1679033290,"l":1,"v":"DQAZAQAAAAHTAgGYCAAFA0NcAAAEPxrhSAoAAQsAAAwAAA==","c":10,"j":63202}

[0103] Each measurement point requires 8 bytes, and 100 measurement points require 54+8*100=854 bytes. 54 represents fixed payload information, namely “{"g":"2007777777","t":1679033290,"l":1,"v":"", and “","c":10,"j":63202}”.

[0104] By comparing the traffic saved by compressed JSON format and standard JSON format, taking 100 measurement point data as an example, the compressed JSON format data volume / standard JSON format data volume = 854 / 2837 = 30%, that is, the compressed JSON format data volume is 70% of the standard JSON format data volume, saving 70% of the traffic.

[0105] In one embodiment, a data transmission and decompression method based on a measurement point table further includes the following steps:

[0106] Acquire a unique device identifier of the gateway or the data transmission unit of the target platform from the protocol frame;

[0107] Matching a measurement point table pre-stored on the target platform according to the unique identifier of the device;

[0108] If the measuring point table exists, determining whether a measuring point table check code in the measuring point table is consistent with a measuring point table check code in the protocol frame;

[0109] If the measurement point table check code in the measurement point table is consistent with the measurement point table check code in the protocol frame, the protocol frame is converted into a corresponding binary measurement point data stream, and the measurement point data of the monitored device is decompressed in the form of the measurement point sequence number and the measurement point value, combined with the configured upload data type.

[0110] Reference Figure 3 Specifically, when the IoT platform receives the measurement point MQTT protocol frame uploaded by the gateway or data transmission unit, it parses the protocol frame to obtain the device unique identifier (device ID) of the gateway or data transmission unit of the target platform, and matches the measurement point table pre-stored in the target platform according to the device unique identifier to find the corresponding measurement point table pre-stored in the target platform.

[0111] If there is no matching result in the measurement point table, the protocol frame is discarded and the application layer is notified of the data frame error.

[0112] If there is a corresponding measurement point table, it is determined whether the CRC16 check value in the measurement point table is consistent with the CRC16 check value in the protocol frame.

[0113] When the CRC16 check value in the measuring point table is consistent with the CRC16 check value in the protocol frame, the base64-encoded measuring point data value of the protocol frame is converted into a binary data stream. Then, in the form of "measuring point sequence number + measuring point value", combined with the predefined upload data type of the measuring point table, the measuring point data is parsed and saved to the database.

[0114] When the CRC16 check value in the measurement point table is inconsistent with the CRC16 check value in the protocol frame, the protocol frame is discarded and the application layer is notified of the data frame error.

[0115] In summary, a data transmission and decompression method based on a measuring point table configures a measuring point table on an IoT platform, controls a gateway or a data transmission unit IoT device to continuously and cyclically collect the measuring point data of the monitored device. If the data upload interval of the measuring point data meets the requirements, a base64-encoded data stream of the uploaded measuring point is created according to the compression algorithm, and a compressed JSON data packet is created with the base64-encoded data as the JSON data value. An MQTT protocol frame is created with the compressed JSON data packet as the payload and uploaded to the IoT platform. This method can improve the transmission efficiency of IoT data, save transmission bandwidth, reduce the use of network resources, reduce operating costs, especially significantly reduce the cost of cellular mobile networks, improve the problem of very limited bandwidth of the IoT, and enhance the applicability of the compression algorithm.

[0116] An efficient data transmission mechanism based on the data transmission and decompression method of the measurement point table is suitable for the Internet of Things platform in different application scenarios.

[0117] When a gateway or DTU IoT device uses a cellular mobile network to upload data, for example, a customer who uploads data in standard JSON format requires a 300MB / month data card; when using compressed JSON format, only a 100MB / month data card is required. A 300MB / month data card costs 18 yuan / year, while a 100MB / month data card costs 1 yuan / year. If this customer installs 10,000 gateways, the annual data fee savings can be (18-1)*10,000=170,000 yuan, significantly reducing SIM card traffic and operating costs.

[0118] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0119] Reference Figure 4 The embodiment of the present application further provides a data transmission and decompression device based on a measuring point table, which corresponds one-to-one to a data transmission and decompression method based on a measuring point table in the above embodiment. The data transmission and decompression device based on a measuring point table includes:

[0120] A configuration module is used to configure a measurement point table, wherein the measurement point table includes the attributes of the gateway or data transmission unit IoT device, the attributes of other IoT devices, the attributes of the monitored device, the attributes of the measurement point, and the measurement point table check code;

[0121] Initialization module, used to download the configured measurement point table to the target platform’s gateway or data transmission unit IoT device;

[0122] A temporary storage module is used to obtain and temporarily store the measurement point data of the monitored equipment based on the measurement point table, and to determine whether a preset data upload interval has been reached;

[0123] A binary module is used to create a binary measurement point data stream according to the measurement point sequence number and measurement point value of the measurement point data when the data upload interval is reached;

[0124] An encoding module, configured to perform data encoding on the binary measurement point data stream to obtain an encoding value;

[0125] A compression module, configured to create a compressed data packet using the coded value as the value of the measurement point data stream in combination with a preset upload data type;

[0126] A frame module, configured to create a protocol frame using the compressed data packet as a protocol payload according to a preset data transmission protocol type;

[0127] The uploading module is used to upload the protocol frame to the target platform.

[0128] A data transmission and decompression device based on a measuring point table further includes:

[0129] an identification module, configured to obtain a unique device identifier of the gateway or the data transmission unit of the target platform from the protocol frame;

[0130] A matching module, configured to match a measurement point table pre-stored on a target platform according to the unique identifier of the device;

[0131] a verification module, configured to determine, when the measuring point table exists, whether a measuring point table verification code in the measuring point table is consistent with a measuring point table verification code in the protocol frame;

[0132] a decompression module configured to convert the protocol frame into a corresponding binary measurement point data stream when the measurement point table check code in the measurement point table is consistent with the measurement point table check code in the protocol frame, and decompress the measurement point data of the monitored device in the form of a measurement point sequence number and a measurement point value in combination with the configured upload data type.

[0133] A data transmission and decompression device based on a measuring point table further includes:

[0134] The alarm module is used to discard the protocol frame and notify the application layer of the data frame error when there is no matching result in the measuring point table, or to discard the protocol frame and notify the application layer of the data frame error when the measuring point table check code in the measuring point table is inconsistent with the measuring point table check code in the protocol frame.

[0135] For specific definitions of a data transmission and decompression device based on a measurement point table, please refer to the above definitions of a data transmission and decompression method based on a measurement point table, which will not be repeated here.

[0136] Each module in the aforementioned data transmission and decompression device based on a measurement point table can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0137] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When executed by the processor, the computer program implements any of the above-mentioned data transmission and decompression methods based on the measurement point table.

[0138] In one embodiment, a computer-readable storage medium is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any one of the above-mentioned data transmission and decompression methods based on the measurement point table is implemented.

[0139] In one embodiment, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the computer program implements any one of the above-mentioned data transmission and decompression methods based on the measurement point table.

[0140] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. When the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0141] Those skilled in the art will clearly understand that for the sake of convenience and brevity in description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A data transmission and decompression method based on a measuring point table, characterized in that: The following steps are included: Configure a measurement point table, which includes the gateway or data transmission unit IoT device attributes, other IoT device attributes, monitored device attributes, measurement point attributes, and a measurement point table check code; Download the configured measurement point table to the target platform’s gateway or data transmission unit IoT device; Based on the measuring point table, obtaining and temporarily storing the measuring point data of the monitored equipment, and determining whether a preset data upload interval has been reached; When the data upload interval is reached, the measurement point data is converted into a binary measurement point data stream in the form of measurement point sequence numbers and measurement point values; Performing data encoding on the binary measurement point data stream to obtain an encoding value; Using the coded value as the value of the measurement point data stream and combining it with a preset upload data type, a compressed data packet is created; Creating a protocol frame using the compressed data packet as a protocol payload according to a preset data transmission protocol type; The protocol frame is uploaded to the target platform.

2. The data transmission and decompression method based on the measuring point table according to claim 1 is characterized in that: The protocol frame includes the unique device identifier of the target platform's gateway or data transmission unit, the current timestamp of the uploaded data, the byte length value of the measurement point sequence number, the coding value including the measurement point sequence number and the measurement point value, the total number of uploaded measurement point data and the measurement point table check code.

3. The data transmission and decompression method based on the measuring point table according to claim 1 or 2, characterized in that: The following steps are also included: Acquire a device unique identifier of the gateway or the data transmission unit of the target platform from the protocol frame; Matching a measurement point table pre-stored on the target platform according to the unique identifier of the device; If the measuring point table exists, determining whether a measuring point table check code in the measuring point table is consistent with a measuring point table check code in the protocol frame; If the measurement point table check code in the measurement point table is consistent with the measurement point table check code in the protocol frame, the protocol frame is converted into a corresponding binary measurement point data stream, and the measurement point data of the monitored device is decompressed in the form of the measurement point sequence number and the measurement point value, combined with the configured upload data type.

4. The data transmission and decompression method based on the measuring point table according to claim 3 is characterized in that: When the protocol frame exists, after the step of determining whether the measurement point table check code in the measurement point table is consistent with the measurement point table check code in the protocol frame, the method further includes: If the measurement point table check code in the measurement point table is inconsistent with the measurement point table check code in the protocol frame, the protocol frame is discarded and the application layer is notified of a data frame error.

5. The data transmission and decompression method based on the measuring point table according to claim 3 is characterized in that: After the step of matching the measurement point table pre-stored in the target platform according to the unique identifier of the device, the method further includes: If there is no matching result in the measurement point table, the protocol frame is discarded and the application layer is notified of the data frame error.

6. A data transmission and decompression device based on a measuring point table, characterized in that: include, A configuration module is used to configure a measurement point table, wherein the measurement point table includes the attributes of the gateway or data transmission unit IoT device, the attributes of other IoT devices, the attributes of the monitored device, the attributes of the measurement point, and the measurement point table check code; Initialization module, used to download the configured measurement point table to the target platform’s gateway or data transmission unit IoT device; A temporary storage module is used to obtain and temporarily store the measurement point data of the monitored equipment based on the measurement point table, and to determine whether a preset data upload interval has been reached; A binary module is used to create a binary measurement point data stream according to the measurement point sequence number and measurement point value of the measurement point data when the data upload interval is reached; An encoding module, configured to perform data encoding on the binary measurement point data stream to obtain an encoding value; A compression module, configured to create a compressed data packet using the coded value as the value of the measurement point data stream in combination with a preset upload data type; A frame module, configured to create a protocol frame using the compressed data packet as a protocol payload according to a preset data transmission protocol type; The uploading module is used to upload the protocol frame to the target platform.

7. The data transmission and decompression device based on the measuring point table according to claim 6, characterized in that: Also includes, an identification module, configured to obtain a unique device identifier of the gateway or the data transmission unit of the target platform from the protocol frame; A matching module, configured to match a measurement point table pre-stored on a target platform according to the unique identifier of the device; a verification module, configured to determine, when the measuring point table exists, whether a measuring point table verification code in the measuring point table is consistent with a measuring point table verification code in the protocol frame; a decompression module configured to convert the protocol frame into a corresponding binary measurement point data stream when the measurement point table check code in the measurement point table is consistent with the measurement point table check code in the protocol frame, and decompress the measurement point data of the monitored device in the form of a measurement point sequence number and a measurement point value in combination with the configured upload data type.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the method according to any one of claims 1 to 5 when the computer program is executed by a processor.

Images