Data transmission and decompression method and device based on measuring point table

Abstract

The invention relates to the technical field of data decompression of the Internet of Things, and discloses a data transmission and decompression method and device based on a measuring point table, and the method comprises the steps: configuring the measuring point table; downloading the configured measuring point table to the gateway of the target platform or the Internet of Things equipment of the data transmission unit; based on the measuring point table, acquiring and temporarily storing measuring point data of the monitored equipment, and judging whether a preset data uploading interval is reached or not; when a data uploading interval is reached, creating a binary measuring point data stream; performing data coding on the binary measuring point data stream to obtain a coded value; creating a compressed data packet by taking the coded value as a value of the measuring point data stream and combining a preset uploading data type; creating a protocol frame by taking the compressed data packet as a protocol load according to a preset data transmission protocol type; and uploading the protocol frame to a target platform. The method has the effects of improving the transmission efficiency of the Internet of Things data and enhancing the applicability of the compression algorithm.

Description

Technical Field

[0001] The present application relates to the technical field of data decompression in the Internet of Things, and in particular to a data transmission and decompression method and device based on a measuring point table. Background Art

[0002] At present, due to the characteristics of JSON data format being lightweight, easy to read and write, and easy to machine parse and generate, JSON data format is widely used in IoT data transmission of MQTT protocol. In order to facilitate reading and machine parsing, the standard JSON data format uses repeated identifiers when expressing, such as key-value pair separators, key and value separators, etc.

[0003] In the prior art, the CJSON compression algorithm is used to compress JSON data, and the unordered set of key-value pairs is organized into the form of a template and a value; or, the HPack compression algorithm is used to extract the key and the value from the unordered set of key-value pairs, and the first value in the obtained array is the template of the HPack compression algorithm, and the following values ​​are in order. However, during the data transmission process, the repeated identifiers of the JSON data will affect the data transmission efficiency and waste network resources. In particular, the number of devices connected to the Internet of Things platform is in the tens of thousands, and the amount of data that needs to be uploaded is very large. The data transmission efficiency is low, and the occupation of network resources is further aggravated. Furthermore, due to the inconsistent key-value pair format of the JSON data based on the measurement point table, the CJSON compression algorithm and the HPack compression algorithm are not suitable for JSON data compression based on the measurement point table.

[0004] With respect to the above-mentioned related technologies, the inventors found that the existing IoT data transmission methods have the problems of low data transmission efficiency, occupying more network resources and poor applicability. Summary of the invention

[0005] In order to improve the transmission efficiency of IoT data, reduce the use of network resources, and enhance the applicability of compression algorithms, the present application provides a data transmission and decompression method and device based on a measurement point table.

[0006] In a first aspect, the present application provides a data transmission and decompression method based on a measurement point table.

[0007] This application is achieved through the following technical solutions:

[0008] A data transmission and decompression method based on a measuring point table comprises the following steps:

[0009] Configure a measuring point table, the measuring point table including gateway or data transmission unit IoT device attributes, other IoT device attributes, monitored device attributes, measuring point attributes and a measuring point table check code;

[0010] Download the configured measurement point table to the target platform’s gateway or data transmission unit IoT device;

[0011] Based on the measuring point table, obtaining and temporarily storing the measuring point data of the monitored equipment, and determining whether a preset data uploading interval has been reached;

[0012] When the data upload interval is reached, the measuring point data is used to create a binary measuring point data stream in the form of measuring point sequence numbers and measuring point values;

[0013] Performing data encoding on the binary measurement point data stream to obtain an encoding value;

[0014] Using the coded value as the value of the measuring point data stream and combining it with a preset upload data type, a compressed data packet is created;

[0015] According to a preset data transmission protocol type, a protocol frame is created using the compressed data packet as a protocol payload;

[0016] The protocol frame is uploaded to the target platform.

[0017] In a preferred example, the present application can be further configured as follows: the protocol frame includes the device unique identifier of the gateway or data transmission unit of the target platform, the current timestamp of the uploaded data, the byte length value of the measuring point sequence number, the coding value including the measuring point sequence number and the measuring point value, the total number of uploaded measuring point data and the measuring point table check code.

[0018] In a preferred example, the present application can be further configured as follows: it also includes the following steps:

[0019] Acquire a unique device identifier of the gateway or the data transmission unit of the target platform from the protocol frame;

[0020] According to the unique identifier of the device, matching the measurement point table pre-stored in the target platform;

[0021] If the measuring point table exists, determining whether the measuring point table check code in the measuring point table is consistent with the measuring point table check code in the protocol frame;

[0022] If the measuring point table check code in the measuring point table is consistent with the measuring point table check code in the protocol frame, the protocol frame is converted into a corresponding binary measuring point data stream, and the measuring point data of the monitored device is decompressed in the form of measuring point sequence number and measuring point value in combination with the configured upload data type.

[0023] In a preferred example, the present application may be further configured as follows: when the protocol frame exists, after the step of determining whether the measurement point table check code in the measurement point table is consistent with the measurement point table check code in the protocol frame, the step further includes:

[0024] If the measuring point table check code in the measuring point table is inconsistent with the measuring point table check code in the protocol frame, the protocol frame is discarded and the application layer is notified of a data frame error.

[0025] In a preferred example, the present application may be further configured as follows: after the step of matching the measurement point table pre-stored in the target platform according to the unique identification of the device, the method further includes:

[0026] If there is no matching result in the measurement point table, the protocol frame is discarded and the application layer is notified of the data frame error.

[0027] In a second aspect, the present application provides a data transmission and decompression device based on a measurement point table.

[0028] This application is achieved through the following technical solutions:

[0029] A data transmission and decompression device based on a measuring point table, comprising:

[0030] A configuration module, used to configure a measuring point table, wherein the measuring point table includes the gateway or data transmission unit IoT device attributes, other IoT device attributes, monitored device attributes, measuring point attributes and a measuring point table check code;

[0031] Initialization module, used to download the configured measurement point table to the gateway or data transmission unit IoT device of the target platform;

[0032] A temporary storage module, used to obtain and temporarily store the measurement point data of the monitored equipment based on the measurement point table, and to determine whether a preset data upload interval has been reached;

[0033] A binary module, used for creating a binary measuring point data stream according to the measuring point sequence number and the measuring point value of the measuring point data when the data upload interval is reached;

[0034] An encoding module, used for performing data encoding on the binary measurement point data stream to obtain an encoding value;

[0035] A compression module, used to create a compressed data packet by using the coded value as the value of the measuring point data stream and combining it with a preset upload data type;

[0036] A frame module, used to create a protocol frame according to a preset data transmission protocol type using the compressed data packet as a protocol payload;

[0037] The uploading module is used to upload the protocol frame to the target platform.

[0038] In a preferred example, the present application can be further configured as follows: further comprising:

[0039] An identification module, used for obtaining a unique device identification of the gateway or the data transmission unit of the target platform from the protocol frame;

[0040] A matching module, used for matching a measurement point table pre-stored in a target platform according to the unique identifier of the device;

[0041] A verification module, used for determining whether a measurement point table verification code in the measurement point table is consistent with a measurement point table verification code in the protocol frame when the measurement point table exists;

[0042] A decompression module is used to convert the protocol frame into a corresponding binary measuring point data stream when the measuring point table check code in the measuring point table is consistent with the measuring point table check code in the protocol frame, and decompress the measuring point data of the monitored device in the form of measuring point sequence number and measuring point value in combination with the configured upload data type.

[0043] In a third aspect, the present application provides a computer device.

[0044] This application is achieved through the following technical solutions:

[0045] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned data transmission and decompression methods based on a measuring point table are implemented.

[0046] In a fourth aspect, the present application provides a computer-readable storage medium.

[0047] This application is achieved through the following technical solutions:

[0048] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements the steps of any one of the above-mentioned data transmission and decompression methods based on a measuring point table.

[0049] In a fifth aspect, the present application provides a computer program product.

[0050] This application is achieved through the following technical solutions:

[0051] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of any one of the above-mentioned data transmission and decompression methods based on a measuring point table.

[0052] In summary, compared with the prior art, the technical solution provided by this application has at least the following beneficial effects:

[0053] Configure the measuring point table so that the gateway or data transmission unit can automatically collect the measuring point data of the IoT device itself and the monitored device according to the measuring point information of the configured measuring point table; download the configured measuring point table to the gateway or data transmission unit IoT device of the target platform to complete the preparation work; based on the measuring point table, obtain and temporarily store the measuring point data of the monitored device to realize the automatic collection of measuring point data, and at the same time determine whether the preset upload data interval has been reached to avoid the continuous upload of IoT data and occupying the network bandwidth; when the upload data interval is reached, create a binary measuring point data stream in the form of measuring point sequence number and measuring point value, use the measuring point sequence number to distinguish each measuring point and its upload order, reduce the key definition and measuring point value separator, thereby saving the transmission amount of measuring point data, improving data transmission efficiency, and also facilitating For subsequent data encoding, it can also flexibly and disorderly combine the uploaded measuring point data; encode the binary measuring point data stream to obtain the encoding value to realize the conversion from binary to character, and further reduce the data volume; use the encoding value as the value of the measuring point data stream, combined with the preset upload data type, to create a compressed data packet, and further compress the data volume of the measuring point data; according to the preset data transmission protocol type, use the compressed data packet as the protocol payload, create a protocol frame, and upload it to the target platform to complete the transmission of the Internet of Things measuring point data; and then a data transmission and decompression method based on the measuring point table can improve the transmission efficiency of the Internet of Things data, save transmission bandwidth, reduce the use of network resources, reduce operating costs, especially significantly reduce the cost of cellular mobile networks, and enhance the applicability of compression algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A basic architecture diagram of an Internet of Things system providing a data transmission and decompression method based on a measurement point table as an exemplary embodiment of the present application.

[0055] Figure 2 A transmission flow diagram of a data transmission and decompression method based on a measurement point table is provided as another exemplary embodiment of the present application.

[0056] Figure 3 A schematic diagram of a decompression process of a data transmission and decompression method based on a measurement point table is provided as another exemplary embodiment of the present application.

[0057] Figure 4 A structural block diagram of a data transmission and decompression device based on a measurement point table provided for an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0058] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make modifications to the present embodiment without any creative contribution as needed, but such modifications are protected by the patent law as long as they are within the scope of the claims of the present application.

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0061] JSON is a lightweight data exchange format. It is based on a subset of ECMAScript and uses a text format that is completely independent of programming languages ​​to store and represent data. It is easy for people to read and write, and it is also easy for machines to parse and generate, and it effectively improves network transmission efficiency. JSON consists of two data structures: objects and arrays; objects are unordered collections of key-value pairs, where the key is a string and the value can be of any type. Objects are surrounded by a pair of curly braces "{}", and the key and value are separated by a colon ":". Key-value pairs are separated by a comma ","; arrays are ordered collections of values, each of which can be of any type. Arrays are wrapped by a pair of square brackets "[]", and values ​​are separated by a comma ",". The simplicity and clear hierarchical structure make JSON an ideal data exchange language. JSON is a text-based lightweight data exchange format that does not support direct transmission of binary data.

[0062] The MQTT protocol is a "lightweight" communication protocol based on the publish / subscribe model. The protocol is built on the TCP / IP protocol. The MQTT protocol is designed for communication between a large number of remote sensors and control devices with limited computing power and working on low-bandwidth, unreliable networks. Features of the MQTT protocol: (1) Using the publish / subscribe messaging model, it provides one-to-many message publishing and decouples applications; (2) Message transmission with payload content shielding; (3) Using TCP / IP to provide network connection; (4) There are three message publishing service qualities: (5) Small transmission, low overhead, and minimized protocol exchanges to reduce network traffic; (6) A mechanism that uses the Last Will and Testament features to notify relevant parties of abnormal client interruptions. The MQTT protocol is widely used in the field of the Internet of Things for communication between sensors and servers, information collection, etc. The JSON data format is widely used in the information transmission based on the MQTT protocol in the Internet of Things.

[0063] However, when using the JSON data format to transmit IoT data, repeated identifiers will affect data transmission efficiency and waste network resources. Furthermore, the number of devices connected to the IoT platform is in the tens of thousands, and the amount of data that needs to be uploaded is very large, and the data transmission efficiency is low. The network bandwidth of the IoT is very limited. The wider the bandwidth, the higher the cost, and the occupation of network resources is further aggravated. In particular, when using cellular mobile networks for transmission, more traffic will be wasted, resulting in a significant increase in tariffs and higher operating costs.

[0064] Although the CJSON compression algorithm and the HPack compression algorithm can greatly improve the transmission efficiency of IoT data and save network resources for scenarios with consistent key-value pair formats, for special application scenarios of JSON data format compression based on the measurement point table, the existing data compression algorithms are not suitable for JSON data compression based on the measurement point table due to the inconsistent key-value pair format of the JSON data based on the measurement point table.

[0065] In order to improve the transmission efficiency of measurement point data, save network resources, save cellular mobile network traffic and charges, and improve the applicability of compression algorithms, this application designs a JSON data compression method based on the measurement point table. On the basis of following the JSON data format, the standard JSON data is compressed by sacrificing a certain degree of readability, thereby improving the transmission efficiency of IoT data, saving transmission bandwidth, and saving operating costs.

[0066] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0067] Reference Figure 1, an embodiment of the present application provides a data transmission and decompression method based on a measuring point table, and its Internet of Things system includes an Internet of Things platform, an Internet of Things device and a monitored device. The monitored devices include temperature sensors, water meters, electricity meters, air conditioners, transformers, etc. The monitored device is a device monitored by the Internet of Things platform. Internet of Things devices include gateways, data transmission units DTU, LoRa terminals, etc. The Internet of Things devices collect data from the monitored devices and transmit it upward to the Internet of Things platform. The Internet of Things platform is an integrated platform that integrates functions such as device access, device management, data security communication, message subscription, message forwarding, and data services. The Internet of Things platform supports connecting to a large number of monitored devices downward, and collects data from the monitored devices and uploads it to the cloud.

[0068] The main steps of a data transmission and decompression method based on a measurement point table are described as follows.

[0069] Configure a measuring point table, the measuring point table including gateway or data transmission unit IoT device attributes, other IoT device attributes, monitored device attributes, measuring point attributes and a measuring point table check code;

[0070] Download the configured measurement point table to the target platform’s gateway or data transmission unit IoT device;

[0071] Based on the measuring point table, obtaining and temporarily storing the measuring point data of the monitored equipment, and determining whether a preset data uploading interval has been reached;

[0072] When the data upload interval is reached, the measuring point data is used to create a binary measuring point data stream in the form of measuring point sequence numbers and measuring point values;

[0073] Performing data encoding on the binary measurement point data stream to obtain an encoding value;

[0074] Using the coded value as the value of the measuring point data stream and combining it with a preset upload data type, a compressed data packet is created;

[0075] According to a preset data transmission protocol type, a protocol frame is created using the compressed data packet as a protocol payload;

[0076] The protocol frame is uploaded to the target platform.

[0077] Specifically, a measuring point table is configured on the IoT platform, and the measuring point table includes the IoT device attributes of the gateway or data transmission unit DTU, other IoT device attributes, monitored device attributes, measuring point attributes, and measuring point table check code. Among them, the IoT device attributes of the gateway or data transmission unit DTU include the gateway or DTU device identification, communication address, device type, device ID, etc. Other IoT device attributes include other IoT device identification, communication address, device type, device ID, etc. The monitored device attributes include the monitored device identification, communication address, device type, parent IoT device ID, communication interface parameters, device protocol type, etc. The measuring point attributes include the device to which it belongs, the data type of collection, the address of the collection measuring point, the collection command word, the data type of upload, the time interval of upload, etc. The measuring point table check code can be the measuring point table CRC16 check to ensure the correctness and uniqueness of the measuring point table.

[0078] In this embodiment, the gateway or data transmission unit IoT device is the IoT root device, which directly communicates with the platform. Other IoT devices are wireless networking sub-devices centered on the gateway, such as LoRa nodes or zigbee nodes and other IoT devices. They form a wireless network with the LoRa gateway module or zigbee gateway module in the gateway, and the monitored devices connected to the nodes collect data, which is transmitted to the gateway through the wireless network, and then the gateway creates a protocol frame and uploads it to the IoT platform. In the measurement point table, the gateway or data transmission unit IoT device and other IoT devices are used to describe the attributes associated with the sub-device and the root device.

[0079] By expressing the different properties of different devices in digital language, different devices can be identified by identifying different properties to monitor device information. When configuring the measuring point table, set the measuring point properties according to the measuring point information to be monitored. In the measuring point table, follow the module rules, fill in the template of the target measuring point, enter the measuring point data properties and the measuring point table check code.

[0080] The measurement point table is a configuration file used to configure the gateway or DTU IoT device to collect the properties of the measurement points of the IoT device itself and the monitored device, so that the gateway or DTU can automatically collect the measurement point data based on the configured measurement point information.

[0081] Next, download the configured measurement point table to the target platform's gateway or data transmission unit IoT device. The IoT platform creates configuration or control commands, creates compressed data packets based on the standard JSON data format, and sends them to the gateway or DTU IoT device using the MQTT protocol. The gateway or DTU IoT device creates compressed data packets based on the standard JSON data format with the execution results of the configuration or control commands, and uploads them to the IoT platform using the MQTT protocol.

[0082] Reference Figure 2 ,According to the measuring point table, the gateway or data transmission unit IoT device,uses the pre-configured collection port, protocol type and other configuration information,,continuously and cyclically collects the measuring point data of all the measuring points defined in the,measuring point table, and temporarily stores the measuring point data (sampling values) of the,monitored device in RAM.

[0083] Determine whether the preset data upload interval has been reached.

[0084] When the upload interval of the measuring point meets the conditions, it is determined that the upload data interval has been reached, and the temporarily stored measuring point data is created into a binary measuring point data stream in the form of measuring point sequence number and measuring point value. The measuring point sequence number is used to distinguish each measuring point and its upload order. The measuring point sequence number represents the arrangement order of the measuring point in the measuring point table. If the number of collected measuring points defined in the measuring point table is less than 256, the measuring point sequence number is 1 byte. If the number of collected measuring points is greater than or equal to 256, the measuring point sequence number is 2 bytes. All uploaded measuring point values ​​are regarded as one value, and the "," separator between the measuring point values ​​is omitted, which reduces the measuring point value separator and saves a lot of transmission data. Each uploaded measuring point value is presented in the combination of "measuring point sequence number + measuring point value", which can flexibly and unorderly combine the uploaded measuring point values.

[0085] Encode the binary measurement point data stream, such as base64 encoding, to obtain the encoding value. Base64 is one of the most common encoding methods for transmitting 8-bit bytecodes on the Internet. Base64 is a method of representing binary data based on 64 printable characters. Base64 includes a character set of 64 characters, including lowercase letters az, uppercase letters AZ, numbers 0-9, symbols "+", and " / ". Any symbol can be converted into a character in this character set. This conversion process is called base64 encoding. Base64 encoding is a process from binary to character, which can be used to transmit longer identification information in the HTTP environment. After base64 encoding, it needs to be decoded before it can be read. The original JSON data format is essentially a text format, and data can only be transmitted in character format, not in binary format. Converting 1-byte binary data to ASCII character data requires 2 bytes; converting 3-byte binary data to ASCII character data requires 6 bytes; but converting 3-byte binary data to base64 encoded data only requires 4 bytes. Therefore, using base64 encoding for binary data can save 33% of the data volume compared to using ASCII encoding.

[0086] The above-mentioned encoding value is used as the value of the measuring point data stream, combined with the preset upload data type, to create a compressed data packet, and the collected measuring point data is created in a compressed JSON data format.

[0087] According to the preset data transmission protocol type, the compressed data packet is used as the protocol payload to create a protocol frame, and the protocol frame is uploaded to the gateway or data transmission unit of the target platform. If the MQTT protocol is used to upload the measurement point data, the compressed JSON data packet is used as the MQTT protocol payload to create an MQTT protocol frame, and then the measurement point MQTT protocol frame is uploaded to the IoT platform via Ethernet / cellular mobile network / WiFi.

[0088] In one embodiment, the protocol frame includes the device unique identifier of the gateway or data transmission unit of the target platform, the current timestamp of the uploaded data, the byte length value of the measuring point sequence number, the coding value including the measuring point sequence number and the measuring point value, the total number of uploaded measuring point data and the measuring point table check code.

[0089] For example, {"g":"device id","t":time stamp,"l":point serial number length,"v":"base64 code data","c":points number,"j":crc16}

[0090] The instructions are as follows:

[0091] ①"g":"device id": unique identifier of the gateway or DTU device that uploads data to the platform. The unique identifier of the device is a numeric string. The platform must save the measurement point table downloaded to each gateway or DTU device, and the measurement point table saved by the platform must be consistent with the measurement point table downloaded to the gateway or DTU device to ensure the consistency of the measurement point table of the platform and the gateway or DTU device.

[0092] ②"t":time stamp: The timestamp when uploading data, a decimal number.

[0093] ③”l”:point serial number length: the length of the point serial number in bytes. In order for the platform to correctly parse the uploaded point information, it is a decimal number with a value of 1 or 2. If the total number of points defined in the point table is less than 256, it takes 1; if the total number of points defined in the point table is greater than or equal to 256, it takes 2.

[0094] ④"v":"base64 code data": upload base64-encoded measurement point value data stream. Each measurement point consists of: measurement point sequence number + measurement point value.

[0095] ⑤"c":points number: the total number of measurement points uploaded this time, a decimal number.

[0096] ⑥"j":crc16: CRC16 check value of the measuring point table, which is a decimal number. It is used by the platform to determine whether the measuring point table of the uploaded data is consistent with the measuring point table saved by the platform; if they are inconsistent, the uploaded measuring point data will not be parsed; if they are consistent, the uploaded measuring point data will be parsed.

[0097] Through the compression format setting of the above protocol frame, the measurement point sequence number is used to distinguish each measurement point and its upload order. All uploaded measurement points are defined as 1 key. Compared with defining 1 key for each measurement point, a large number of key definitions can be omitted. All uploaded measurement point values ​​are regarded as one value, saving the "," separator between measurement point values, saving a lot of transmission data. If the number of acquisition measurement points defined in the measurement point table is less than 256, the measurement point sequence number is 1 byte; if the number of acquisition measurement points is greater than or equal to 256, the measurement point sequence number is 2 bytes. Using base64 to encode binary measurement point data saves more data than using ASCII encoding. The Internet of Things platform parses the measurement point value in conjunction with the measurement point table and verifies the measurement point data according to the CRC16 check value of the measurement point table.

[0098] Assuming that 100 measurement point data are reported, the standard JSON format of the MQTT protocol frame upload format is as follows:

[0099] {"g":"2007777777","t":1615511830,"m":[{"i":"FF0001","v":xxxxxxxx},…,{"i":"yyyy","v":xxxxxxxx}]}

[0100] Each measuring point requires 28 bytes, and 100 measuring points require 37+28*100=2837 bytes, where 37 represents fixed payload information, namely “{"g":"2007777777","t":1615511830,"m":[", and "]}".

[0101] The format of MQTT protocol frame upload in compressed JSON format is as follows:

[0102] {"g":"2007777777","t":1679033290,"l":1,"v":"DQAZAQAAAAHTAgGYCAAFA0NcAAAEPxrhSAoAAQsAAAwAAA==","c":10,"j":63202}

[0103] Among them, each measuring point requires 8 bytes, and 100 measuring points require 54+8*100=854 bytes, where 54 represents fixed payload information, namely “{"g":"2007777777","t":1679033290,"l":1,"v":"", and “","c":10,"j":63202}”.

[0104] By comparing the traffic saved by compressed JSON format and standard JSON format, taking 100 measurement point data as an example, the compressed JSON format data volume / standard JSON format data volume = 854 / 2837 = 30%, that is, the compressed JSON format data volume is 70% of the standard JSON format data volume, saving 70% of the traffic.

[0105] In one embodiment, a data transmission and decompression method based on a measuring point table further includes the following steps:

[0106] Acquire a unique device identifier of the gateway or the data transmission unit of the target platform from the protocol frame;

[0107] According to the unique identifier of the device, matching the measurement point table pre-stored in the target platform;

[0108] If the measuring point table exists, determining whether the measuring point table check code in the measuring point table is consistent with the measuring point table check code in the protocol frame;

[0109] If the measuring point table check code in the measuring point table is consistent with the measuring point table check code in the protocol frame, the protocol frame is converted into a corresponding binary measuring point data stream, and the measuring point data of the monitored device is decompressed in the form of measuring point sequence number and measuring point value in combination with the configured upload data type.

[0110] Reference Figure 3 Specifically, when the IoT platform receives the measurement point MQTT protocol frame uploaded by the gateway or data transmission unit, it parses the protocol frame to obtain the device unique identifier (device ID) of the gateway or data transmission unit of the target platform, and matches the measurement point table pre-stored in the target platform according to the device unique identifier to find the corresponding measurement point table pre-stored in the target platform.

[0111] If there is no matching result in the measurement point table, the protocol frame is discarded and the application layer is notified of the data frame error.

[0112] If there is a corresponding measuring point table, it is determined whether the CRC16 check value in the measuring point table is consistent with the CRC16 check value in the protocol frame.

[0113] When the CRC16 check value in the measuring point table is consistent with the CRC16 check value in the protocol frame, the base64 encoded measuring point data value of the protocol frame is converted into a binary data stream. Then, in the form of "measuring point sequence number + measuring point value", combined with the predefined upload data type in the measuring point table, the measuring point data is parsed and saved to the database.

[0114] When the CRC16 check value in the measurement point table is inconsistent with the CRC16 check value in the protocol frame, the protocol frame is discarded and the application layer is notified of the data frame error.

[0115] In summary, a data transmission and decompression method based on a measuring point table configures a measuring point table on an Internet of Things platform, controls a gateway or a data transmission unit Internet of Things device to continuously and cyclically collect the measuring point data of the monitored device. If the data upload interval of the measuring point data meets the requirements, a base64 encoded data stream of the uploaded measuring point is created according to the compression algorithm, and a compressed JSON data packet is created with the base64 encoded data as the JSON data value. The compressed JSON data packet is used as the payload to create an MQTT protocol frame and upload it to the Internet of Things platform. This can improve the transmission efficiency of Internet of Things data, save transmission bandwidth, reduce the use of network resources, reduce operating costs, especially significantly reduce the cost of cellular mobile networks, improve the problem of very limited bandwidth of the Internet of Things, and enhance the applicability of the compression algorithm.

[0116] The invention discloses an efficient data transmission mechanism based on a data transmission and decompression method of a measuring point table, which is suitable for an Internet of Things platform in different application scenarios.

[0117] When a gateway or DTU IoT device uses a cellular mobile network to upload data, for example, a customer who uploads data in standard JSON format needs to use a 300MB / month traffic card; but after using compressed JSON format, only a 100MB / month traffic card is needed; a 300MB / month traffic card is 18 yuan / year, and a 100MB / month traffic card is 1 yuan / year; if this customer installs 10,000 gateways, the annual traffic fee can be saved by (18-1)*10,000=170,000 yuan=170,000 yuan, the SIM card traffic is greatly reduced, and the operating costs are also greatly reduced.

[0118] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0119] Reference Figure 4 The embodiment of the present application also provides a data transmission and decompression device based on a measuring point table, and the data transmission and decompression device based on a measuring point table corresponds one to one with a data transmission and decompression method based on a measuring point table in the above embodiment. The data transmission and decompression device based on a measuring point table includes:

[0120] A configuration module, used to configure a measuring point table, wherein the measuring point table includes the gateway or data transmission unit IoT device attributes, other IoT device attributes, monitored device attributes, measuring point attributes and a measuring point table check code;

[0121] Initialization module, used to download the configured measurement point table to the gateway or data transmission unit IoT device of the target platform;

[0122] A temporary storage module, used to obtain and temporarily store the measurement point data of the monitored equipment based on the measurement point table, and to determine whether a preset data upload interval has been reached;

[0123] A binary module, used for creating a binary measuring point data stream according to the measuring point sequence number and the measuring point value of the measuring point data when the data upload interval is reached;

[0124] An encoding module, used for performing data encoding on the binary measurement point data stream to obtain an encoding value;

[0125] A compression module, used to create a compressed data packet by using the coded value as the value of the measuring point data stream and combining it with a preset upload data type;

[0126] A frame module, used to create a protocol frame according to a preset data transmission protocol type using the compressed data packet as a protocol payload;

[0127] The uploading module is used to upload the protocol frame to the target platform.

[0128] A data transmission and decompression device based on a measuring point table also includes:

[0129] An identification module, used for obtaining a unique device identification of the gateway or the data transmission unit of the target platform from the protocol frame;

[0130] A matching module, used for matching a measurement point table pre-stored in a target platform according to the unique identifier of the device;

[0131] A verification module, used for determining whether a measurement point table verification code in the measurement point table is consistent with a measurement point table verification code in the protocol frame when the measurement point table exists;

[0132] A decompression module is used to convert the protocol frame into a corresponding binary measuring point data stream when the measuring point table check code in the measuring point table is consistent with the measuring point table check code in the protocol frame, and decompress the measuring point data of the monitored device in the form of measuring point sequence number and measuring point value in combination with the configured upload data type.

[0133] A data transmission and decompression device based on a measuring point table also includes:

[0134] The alarm module is used to discard the protocol frame and notify the application layer of the data frame error when there is no matching result of the measuring point table, or discard the protocol frame and notify the application layer of the data frame error when the measuring point table check code in the measuring point table is inconsistent with the measuring point table check code in the protocol frame.

[0135] The specific definition of a data transmission and decompression device based on a measuring point table can be found in the above definition of a data transmission and decompression method based on a measuring point table, which will not be repeated here.

[0136] Each module in the above-mentioned data transmission and decompression device based on the measuring point table can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0137] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, any of the above-mentioned data transmission and decompression methods based on the measuring point table is implemented.

[0138] In one embodiment, a computer-readable storage medium is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the above-mentioned data transmission and decompression methods based on the measuring point table is implemented.

[0139] In one embodiment, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, any one of the above-mentioned data transmission and decompression methods based on the measuring point table is implemented.

[0140] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. When the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0141] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A data transmission and decompression method based on a measuring point table, characterized in that: The following steps are included: Configure a measuring point table, the measuring point table including gateway or data transmission unit IoT device attributes, other IoT device attributes, monitored device attributes, measuring point attributes and a measuring point table check code; Download the configured measurement point table to the target platform’s gateway or data transmission unit IoT device; Based on the measuring point table, obtaining and temporarily storing the measuring point data of the monitored equipment, and determining whether a preset data uploading interval has been reached; When the data upload interval is reached, the measuring point data is used to create a binary measuring point data stream in the form of measuring point sequence numbers and measuring point values; Performing data encoding on the binary measurement point data stream to obtain an encoding value; Using the coded value as the value of the measuring point data stream and combining it with a preset upload data type, a compressed data packet is created; According to a preset data transmission protocol type, a protocol frame is created using the compressed data packet as a protocol payload; The protocol frame is uploaded to the target platform.

2. The data transmission and decompression method based on the measuring point table according to claim 1 is characterized in that: The protocol frame includes the unique device identifier of the target platform's gateway or data transmission unit, the current timestamp of the uploaded data, the byte length value of the measurement point sequence number, the coding value including the measurement point sequence number and the measurement point value, the total number of uploaded measurement point data and the measurement point table check code.

3. The data transmission and decompression method based on the measuring point table according to claim 1 or 2, characterized in that: The following steps are also included: Acquire a unique device identifier of the gateway or the data transmission unit of the target platform from the protocol frame; According to the unique identifier of the device, matching the measurement point table pre-stored in the target platform; If the measuring point table exists, determining whether the measuring point table check code in the measuring point table is consistent with the measuring point table check code in the protocol frame; If the measuring point table check code in the measuring point table is consistent with the measuring point table check code in the protocol frame, the protocol frame is converted into a corresponding binary measuring point data stream, and the measuring point data of the monitored device is decompressed in the form of measuring point sequence number and measuring point value in combination with the configured upload data type.

4. The data transmission and decompression method based on the measuring point table according to claim 3 is characterized in that: When the protocol frame exists, after the step of determining whether the measurement point table check code in the measurement point table is consistent with the measurement point table check code in the protocol frame, the method further includes: If the measuring point table check code in the measuring point table is inconsistent with the measuring point table check code in the protocol frame, the protocol frame is discarded and the application layer is notified of a data frame error.

5. The data transmission and decompression method based on the measuring point table according to claim 3 is characterized in that: After the step of matching the measuring point table pre-stored in the target platform according to the unique identification of the device, the method further includes: If there is no matching result in the measurement point table, the protocol frame is discarded and the application layer is notified of the data frame error.

6. A data transmission and decompression device based on a measuring point table, characterized in that: include, A configuration module, used to configure a measuring point table, wherein the measuring point table includes the gateway or data transmission unit IoT device attributes, other IoT device attributes, monitored device attributes, measuring point attributes and a measuring point table check code; Initialization module, used to download the configured measurement point table to the gateway or data transmission unit IoT device of the target platform; A temporary storage module, used to obtain and temporarily store the measurement point data of the monitored equipment based on the measurement point table, and to determine whether a preset data upload interval has been reached; A binary module, used for creating a binary measuring point data stream according to the measuring point sequence number and the measuring point value of the measuring point data when the data upload interval is reached; An encoding module, used for performing data encoding on the binary measurement point data stream to obtain an encoding value; A compression module, used to create a compressed data packet by using the coded value as the value of the measuring point data stream and combining it with a preset upload data type; A frame module, used to create a protocol frame according to a preset data transmission protocol type using the compressed data packet as a protocol payload; The uploading module is used to upload the protocol frame to the target platform.

7. The data transmission and decompression device based on the measuring point table according to claim 6 is characterized in that: Also includes, An identification module, used for obtaining a unique device identification of the gateway or the data transmission unit of the target platform from the protocol frame; A matching module, used for matching a measurement point table pre-stored in a target platform according to the unique identifier of the device; A verification module, used for determining whether a measurement point table verification code in the measurement point table is consistent with a measurement point table verification code in the protocol frame when the measurement point table exists; A decompression module is used to convert the protocol frame into a corresponding binary measuring point data stream when the measuring point table check code in the measuring point table is consistent with the measuring point table check code in the protocol frame, and decompress the measuring point data of the monitored device in the form of measuring point sequence number and measuring point value in combination with the configured upload data type.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the method according to any one of claims 1 to 5 when being executed by a processor.

Images