Data collection method and system for industrial Internet of Things gateway

By adopting multi-level data compression technology and real-time monitoring of transmission status in industrial IoT gateways, the resource occupation problem in real-time transmission of industrial IoT processing results is solved, and efficient and reliable data transmission and storage are achieved.

CN118540353BActive Publication Date: 2025-02-07ZHEJIANG LIANYUN ZHIDING INFORMATION TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411016151.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-27
Publication Date
2025-02-07
Estimated Expiration
2044-07-27

AI Technical Summary

Technical Problem

When real-time transmission of industrial Internet of Things results, it occupies a large amount of data transmission resources and network resources, and requires a lot of space to store and manage the results.

Method used

By implementing multi-level data compression technologies such as data acquisition, preprocessing, wavelet transformation, PCA dimensionality reduction, prediction coding and adaptive entropy coding in industrial IoT gateways, it significantly reduces data transmission and storage requirements, and transmits through real-time monitoring and recording state, ensuring the reliability and integrity of data transmission.

Benefits of technology

It realizes efficient data compression, significantly reduces resource usage for data transmission and storage, improves the speed and accuracy of data processing, and ensures the reliability and integrity of data transmission.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118540353B_ABST
    Figure CN118540353B_ABST
Patent Text Reader

Abstract

The present invention discloses a data collection method and system for an industrial Internet of Things gateway, and relates to the technical field of Internet of Things gateways. When the system is running, through the coordinated work of a data collection module, a data preprocessing module, a data compression module and a data transmission module, an edge gateway collects the internal and external operating status information of a device in real time to ensure the comprehensiveness and real-time nature of the data. The preprocessing module performs abnormal value detection, denoising, data smoothing and data correction on the collected data group, provides a high-quality data group to be compressed, and improves the accuracy and consistency of the data. The data compression module extracts data features through wavelet transform, and performs PCA dimensionality reduction, predictive coding and adaptive entropy coding, thereby realizing efficient data compression, reducing data transmission and storage requirements, optimizing data compression and transmission processes, reducing a large amount of data transmission and storage resource occupation, and increasing the reliability and integrity of data transmission.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things gateways, and in particular to a data collection method and system for an industrial Internet of Things gateway. Background Art

[0002] The patent document with Chinese patent application number CN202310008426.0 discloses a data acquisition system based on edge computing. The system includes a PLC that connects and controls industrial equipment, an edge gateway that connects industrial equipment or PLC, a cloud platform that connects the edge gateway, and a client that interacts with the cloud platform, wherein the cloud platform is used to generate data acquisition tasks and send the generated data acquisition tasks to the edge gateway; the edge gateway is used to collect data through the industrial equipment or PLC connected to it according to the received data acquisition tasks, and pre-process the collected data and forward it to the cloud platform. In the data acquisition system based on edge computing provided by this application, the cloud platform centrally manages the edge gateway, realizes the dynamic issuance of data acquisition tasks, reduces the workload of setting up data acquisition for the equipment, and improves the efficiency of equipment data acquisition.

[0003] It can be seen that after data edgeization, the final results still need to be transmitted to the cloud or processed. When the industrial Internet of Things processing results under a large base are transmitted in real time, it will still occupy a large amount of data transmission resources and network resources, and the storage and management of the results also require a lot of space for operation and execution. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a data collection method and system for an industrial Internet of Things gateway, which solves the problems mentioned in the background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a data acquisition system for an industrial Internet of Things gateway, including a data acquisition module, a data preprocessing module, a data compression module and a data transmission module;

[0006] The data acquisition module is connected to and controls the industrial equipment or PLC through the edge gateway, and collects the internal and external operating status information of the equipment in real time to form a collection data group;

[0007] The preprocessing module performs preprocessing on the collected data set, including outlier detection, denoising, data smoothing and data correction, to form a processed data set to be compressed. ;

[0008] The data compression module treats the compressed data group Perform wavelet transform to extract the data group to be compressed Compression features to obtain wavelet coefficients , synchronization of wavelet coefficients Perform PCA dimensionality reduction to obtain the data group to be compressed after dimensionality reduction , for the data set to be compressed after dimension reduction Perform predictive coding to obtain the data group to be compressed after predictive coding , for the data group to be compressed after predictive coding Perform adaptive entropy coding to obtain the compressed data set of the data group to be compressed ;

[0009] The data transmission module treats the compressed data set of the compressed data set as Carry out transmission, monitor the data transmission status in real time, detect and record the relevant data status that occurs during the transmission process.

[0010] Preferably, the data acquisition module includes a connection unit and a collection unit;

[0011] The connection unit connects multiple sensors to industrial equipment or PLC through an edge gateway, establishes a communication channel between the sensors and the industrial equipment or PLC, and performs information communication and transmission of the industrial equipment;

[0012] The acquisition unit collects status information of the internal and external operating status of the device through dynamically issued data acquisition tasks, and extracts unique tags from the acquisition devices, and combines them with the collected status information to form a collection data group;

[0013] The collected data group includes a device unique identifier, a sensor unique identifier, a timestamp, a device operating status, device operating environment information, and device operating parameters.

[0014] Preferably, the preprocessing module comprises a preprocessing unit and an integration unit;

[0015] The preprocessing unit preprocesses the collected data group, including using statistical methods and machine learning algorithms to calculate the mean and standard deviation related information of the data, and simultaneously identifying abnormal values ​​according to a preset threshold range; using low-pass filters and median filters to eliminate noise in the data; using sliding average and exponential smoothing methods to smooth the data and adjust data fluctuations; obtaining the calibration curve or known deviation of the equipment through the equipment factory operation manual and equipment user manual, and adjusting and correcting the data;

[0016] The integration unit integrates the pre-processed collected data group to form a processed data group to be compressed ,include{ , , ,..., }, t represents the acquisition time point, and i represents the data subscript in the data group.

[0017] Preferably, the data compression module includes an extraction and dimension reduction unit and an encoding and compression unit;

[0018] The extraction and dimension reduction unit extracts the preprocessed data group to be compressed Perform wavelet transform, extract the main features of the data, and obtain wavelet coefficients , synchronization of wavelet coefficients Perform PCA dimensionality reduction to obtain the data group to be compressed after dimensionality reduction ;

[0019] The encoding and compression unit performs compression on the reduced-dimensional data set. Perform predictive coding to obtain the data group to be compressed after predictive coding , for the data group to be compressed after predictive coding Perform adaptive entropy coding to obtain the compressed data set of the data group to be compressed .

[0020] Preferably, the wavelet coefficients Calculated by the following formula:

[0021] ;

[0022] In the formula, represents the data after preprocessing of the i-th type collected at time point t, represents the wavelet coefficients of the i-th type of data obtained by wavelet transform at time point t, Represents the wavelet transform function, which is used to transform the time domain data Convert data into frequency domain .

[0023] Preferably, the reduced-dimensional data set to be compressed Obtained through the following calculation formula:

[0024] ;

[0025] In the formula, represents the i-th type of data after dimensionality reduction at time point t, represents the wavelet coefficient of the i-th type of data obtained by wavelet transform at time point t, W represents the dimensionality reduction matrix, which is a matrix composed of main eigenvectors obtained by PCA principal component analysis.

[0026] Preferably, the predictively encoded data group to be compressed Obtained through the following calculation formula:

[0027] ;

[0028] In the formula, Represents the error value of the i-th type of data at time point t, represents the predicted value of the i-th type of data at time point t, Represents the i-th type of data after dimensionality reduction at time point t;

[0029] Among them, the predicted value of the i-th type of data at time point t is according to get, represents the prediction weight value, represents the historical data at time point tk, n represents the total number of historical data collection, and k represents the kth historical data collection.

[0030] Preferably, the compressed data set of the data set to be compressed Obtained through the following calculation formula:

[0031] ;

[0032] In the formula, represents a compressed data set of the i-th type of data at time point t, It represents the error value of the i-th type of data at time point t, and EntropyEncode represents the entropy coding function used to calculate the error value of the i-th type of data at time point t. to encode.

[0033] Preferably, the data transmission module includes a transmission detection unit and a transmission recording unit;

[0034] The transmission detection unit treats the compressed data set of the compressed data set as During transmission, real-time monitoring of data transmission status, including network status monitoring and transmission integrity check;

[0035] The transmission recording unit records relevant status information during the data transmission process, including log records and transmission performance records, provides traceability of the transmission status by recording the status information during the transmission process in detail, and optimizes the transmission strategy by analyzing the transmission records, including adjusting the compression method and parameters of the data compression module.

[0036] A data collection method for an industrial Internet of Things gateway comprises the following steps:

[0037] Step 1: The data acquisition module connects to and controls industrial equipment or PLC through the edge gateway, and collects the internal and external operating status information of the equipment in real time to form a collection data group;

[0038] Step 2: The preprocessing module preprocesses the collected data set, including outlier detection, denoising, data smoothing and data correction, to form the processed data set to be compressed. ;

[0039] Step 3: The data compression module treats the compressed data group Perform wavelet transform to extract the data group to be compressed Compression features to obtain wavelet coefficients , synchronization of wavelet coefficients Perform PCA dimensionality reduction to obtain the data group to be compressed after dimensionality reduction , for the data set to be compressed after dimension reduction Perform predictive coding to obtain the data group to be compressed after predictive coding , for the data group to be compressed after predictive coding Perform adaptive entropy coding to obtain the compressed data set of the data group to be compressed ;

[0040] Step 4: Data transmission module treats the compressed data set as the compressed data set Carry out transmission, monitor the data transmission status in real time, detect and record the relevant data status that occurs during the transmission process.

[0041] The present invention provides a data collection method and system for an industrial Internet of Things gateway, which has the following beneficial effects:

[0042] (1) When the system is running, the data acquisition module, data preprocessing module, data compression module and data transmission module work together to collect the internal and external operating status information of the equipment in real time through the edge gateway to ensure the comprehensiveness and real-time nature of the data. The preprocessing module performs outlier detection, denoising, data smoothing and data correction on the collected data group to provide high-quality data groups to be compressed. , improve the accuracy and consistency of data. The data compression module extracts data features through wavelet transform, and performs PCA dimensionality reduction, predictive coding and adaptive entropy coding to achieve efficient data compression and significantly reduce data transmission and storage requirements. The data transmission module is responsible for transmitting the compressed data set and monitoring the data transmission status in real time to ensure that the data is not lost or damaged during transmission, and records the relevant data status. By optimizing the data compression and transmission process, a large amount of data transmission and storage resource occupation is reduced, and the problem of occupying a large amount of data transmission resources and storage space when the industrial Internet of Things processing results are transmitted in real time is solved, ensuring the reliability and integrity of data transmission.

[0043] (2) By treating the compressed data group Perform wavelet transform to extract the data group to be compressed Compression features to obtain wavelet coefficients , synchronization of wavelet coefficients Perform PCA dimensionality reduction to make the data concise without losing important information, and obtain the data set to be compressed after dimensionality reduction , for the data set to be compressed after dimension reduction Perform predictive coding to obtain the data group to be compressed after predictive coding , for the data group to be compressed after predictive coding Perform adaptive entropy coding to obtain the compressed data set of the data group to be compressed This multi-level data compression technology not only greatly reduces the data transmission and storage occupancy, but also effectively improves the speed and accuracy of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a schematic diagram of a data acquisition system block diagram of an industrial Internet of Things gateway of the present invention;

[0045] Figure 2 The present invention is a schematic diagram of the steps of a data collection method for an industrial Internet of Things gateway. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0047] The present invention provides a data acquisition system for an industrial Internet of Things gateway, see Figure 1 , including a data acquisition module, a data preprocessing module, a data compression module and a data transmission module;

[0048] The data acquisition module is connected to and controls the industrial equipment or PLC through the edge gateway, and collects the internal and external operating status information of the equipment in real time to form a collection data group;

[0049] The preprocessing module performs preprocessing on the collected data set, including outlier detection, denoising, data smoothing and data correction, to form a processed data set to be compressed. ;

[0050] The data compression module treats the compressed data group Perform wavelet transform to extract the data group to be compressed Compression features to obtain wavelet coefficients , synchronization of wavelet coefficients Perform PCA dimensionality reduction to obtain the data group to be compressed after dimensionality reduction , for the data set to be compressed after dimension reduction Perform predictive coding to obtain the data group to be compressed after predictive coding , for the data group to be compressed after predictive coding Perform adaptive entropy coding to obtain the compressed data set of the data group to be compressed ;

[0051] The data transmission module treats the compressed data set of the compressed data set as Carry out transmission, monitor the data transmission status in real time, detect and record the relevant data status that occurs during the transmission process.

[0052] In this embodiment, through the coordinated work of the data acquisition module, the data preprocessing module, the data compression module and the data transmission module, the edge gateway collects the internal and external operating status information of the device in real time to ensure the comprehensiveness and real-time nature of the data. The preprocessing module performs outlier detection, denoising, data smoothing and data correction on the collected data group to provide a high-quality data group to be compressed. , improve the accuracy and consistency of data. The data compression module extracts data features through wavelet transform, and performs PCA dimensionality reduction, predictive coding and adaptive entropy coding to achieve efficient data compression and significantly reduce data transmission and storage requirements. The data transmission module is responsible for transmitting the compressed data set and monitoring the data transmission status in real time to ensure that the data is not lost or damaged during transmission, and records the relevant data status. By optimizing the data compression and transmission process, a large amount of data transmission and storage resource occupation is reduced, and the problem of occupying a large amount of data transmission resources and storage space when the industrial Internet of Things processing results are transmitted in real time is solved, ensuring the reliability and integrity of data transmission.

[0053] This embodiment is explained in Example 1, please refer to Figure 1 ,Specifically: the data acquisition module includes a connection unit and a collection unit;

[0054] The connection unit connects multiple sensors to industrial equipment or PLC through an edge gateway, establishes a communication channel between the sensors and the industrial equipment or PLC, and performs information communication and transmission of the industrial equipment;

[0055] The acquisition unit collects status information of the internal and external operating status of the device through dynamically issued data acquisition tasks, and extracts unique tags from the acquisition devices, and combines the unique tags with the collected status information to form the collected data;

[0056] The collected data group includes a device unique identifier, a sensor unique identifier, a timestamp, a device operating status, device operating environment information, and device operating parameters.

[0057] The preprocessing module includes a preprocessing unit and an integration unit;

[0058] The preprocessing unit preprocesses the collected data group, including using statistical methods and machine learning algorithms to calculate the mean and standard deviation related information of the data, and simultaneously identifying abnormal values ​​according to a preset threshold range; using low-pass filters and median filters to eliminate noise in the data; using sliding average and exponential smoothing methods to smooth the data and adjust data fluctuations; obtaining the calibration curve or known deviation of the equipment through the equipment factory operation manual and equipment user manual, and adjusting and correcting the data;

[0059] The integration unit integrates the pre-processed collected data group to form a processed data group to be compressed ,include{ , , ,..., }, t represents the acquisition time point, i represents the data subscript in the data group, and specifically represents the data after preprocessing.

[0060] This embodiment is explained in Example 1, please refer to Figure 1 ,Specifically: the data compression module includes an extraction and dimension reduction unit and an encoding and compression unit;

[0061] The extraction and dimension reduction unit extracts the preprocessed data group to be compressed Perform wavelet transform, extract the main features of the data, and obtain wavelet coefficients , synchronization of wavelet coefficients Perform PCA dimensionality reduction to obtain the data group to be compressed after dimensionality reduction ;

[0062] The encoding and compression unit performs compression on the reduced-dimensional data set. Perform predictive coding to obtain the data group to be compressed after predictive coding , for the data group to be compressed after predictive coding Perform adaptive entropy coding to obtain the compressed data set of the data group to be compressed .

[0063] The wavelet coefficients Calculated by the following formula:

[0064] ;

[0065] In the formula, represents the data after preprocessing of the i-th type collected at time point t, represents the wavelet coefficients of the i-th type of data obtained by wavelet transform at time point t, Represents the wavelet transform function, which transforms the time domain data Convert data into frequency domain ;

[0066] Among them, the wavelet transform function The specific conversion is performed by discrete wavelet transform DWT, including the following steps:

[0067] Step 1: Scattered wavelet transform decomposes the signal into approximate components and detail components by scaling and shifting the mother wavelet function:

[0068] ;

[0069] In the formula, represents the wavelet function at scale s and displacement u, Mother wavelet function, s represents the scale parameter, specifically used to control the width of the wavelet function, a larger s corresponds to a wider wavelet function (low frequency), a smaller s corresponds to a narrower wavelet function (high frequency), u represents the displacement parameter, specifically used to control the position of the wavelet function on the time axis, and t represents the time point variable;

[0070] Step 2: Continuous wavelet transform CWT, by scaling and translating the mother wavelet function, integrate the signal to obtain a representation in the time-frequency domain:

[0071] ;

[0072] In the formula, represents the continuous wavelet transform coefficient at scale s and displacement u, represents the original signal data, represents the complex conjugate of the wavelet function, Indicates integration over the entire time domain;

[0073] Step 3: Discrete wavelet transform DWT, through multi-scale analysis, decomposes the signal into multi-level approximate components and detail components:

[0074] ;

[0075] In the formula, represents the discrete wavelet transform coefficient at scale j and displacement k, represents a discrete signal, represents the discrete wavelet function, It means summing discrete signals;

[0076] Step 4: Discrete wavelet function, by scaling and translating the mother wavelet function, a set of complete basis functions is generated:

[0077] ;

[0078] In the formula, represents the discrete wavelet function at scale j and displacement k, Mother wavelet function, j represents the scale parameter, specifically used to control the width of the wavelet function, a larger j corresponds to a wider wavelet function (low frequency), a smaller j corresponds to a narrower wavelet function (high frequency), k represents the displacement parameter, specifically used to control the position of the wavelet function on the time axis, and n represents a discrete time variable.

[0079] The data group to be compressed after dimension reduction Obtained through the following calculation formula:

[0080] ;

[0081] In the formula, represents the i-th type of data after dimensionality reduction at time point t, represents the wavelet coefficient of the i-th type of data obtained by wavelet transform at time point t, W represents the dimensionality reduction matrix, and the matrix composed of the main eigenvectors obtained by PCA principal component analysis.

[0082] The predictively encoded data group to be compressed Obtained through the following calculation formula:

[0083] ;

[0084] In the formula, Represents the error value of the i-th type of data at time point t, represents the predicted value of the i-th type of data at time point t, Represents the i-th type of data after dimensionality reduction at time point t;

[0085] Among them, the predicted value of the i-th type of data at time point t is pass Calculate historical data acquisition, represents the prediction weight value, represents the historical data at time point tk, n represents the total number of historical data collection, and k represents the kth historical data collection.

[0086] The compressed data set of the data set to be compressed Obtained through the following calculation formula:

[0087] ;

[0088] In the formula, represents a compressed data set of the i-th type of data at time point t, Represents the error value of the i-th type of data at time point t. EntropyEncode represents the entropy coding function, including Huffman coding and arithmetic coding. The error value of the i-th type of data at time point t Encoding;

[0089] The entropy coding function EntropyEncode performs compression through the following Huffman coding steps:

[0090] Step 1: Count the symbol frequency and calculate the frequency of occurrence of each symbol in the data;

[0091] Step 2: Construct a Huffman tree based on the symbol frequency;

[0092] Step 3: Generate a coding table and generate the code for each symbol based on the Huffman tree;

[0093] Step 4: Encode the data and use the coding table to encode the data into Huffman code.

[0094] In this embodiment, by treating the compressed data group Perform wavelet transform to extract the data group to be compressed Compression features to obtain wavelet coefficients , synchronization of wavelet coefficients Perform PCA dimensionality reduction to make the data concise without losing important information, and obtain the data set to be compressed after dimensionality reduction , for the data set to be compressed after dimension reduction Perform predictive coding to obtain the data group to be compressed after predictive coding , for the data group to be compressed after predictive coding Perform adaptive entropy coding to obtain the compressed data set of the data group to be compressed This multi-level data compression technology not only greatly reduces the data transmission and storage occupancy, but also effectively improves the speed and accuracy of data processing.

[0095] This embodiment is explained in Example 1, please refer to Figure 1 ,Specifically: the data transmission module includes a transmission detection unit and a transmission recording unit;

[0096] The transmission detection unit treats the compressed data set of the compressed data set as During transmission, real-time monitoring of data transmission status, including network status monitoring and transmission integrity check;

[0097] The transmission recording unit records relevant status information during the data transmission process, including log records and transmission performance records. By recording the status information during the transmission process in detail, the traceability of the transmission status is provided. By analyzing the transmission records, the transmission strategy is optimized, including adjusting the compression method and parameters of the data compression module.

[0098] In this embodiment, the transmission detection unit monitors the network status in real time during the data transmission process and performs a transmission integrity check to ensure that the data is not lost or damaged during the transmission process. The transmission recording unit records the logs and performance information during the transmission process in detail. Through these detailed records, the system can trace the transmission status, identify potential problems, and optimize the transmission strategy through data analysis. In particular, by adjusting the compression method and parameters of the data compression module, the data transmission efficiency and reliability are further improved. This comprehensive monitoring and optimization strategy not only ensures the stability and integrity of data transmission, but also improves the overall performance and response speed of the system.

[0099] A data collection method for industrial Internet of Things gateway, please refer to Figure 2 , specifically: including the following steps:

[0100] Step 1: The data acquisition module connects to and controls industrial equipment or PLC through the edge gateway, and collects the internal and external operating status information of the equipment in real time to form a collection data group;

[0101] Step 2: The preprocessing module preprocesses the collected data set, including outlier detection, denoising, data smoothing and data correction, to form the processed data set to be compressed. ;

[0102] Step 3: The data compression module treats the compressed data group Perform wavelet transform to extract the data group to be compressed Compression features to obtain wavelet coefficients , synchronization of wavelet coefficients Perform PCA dimensionality reduction to obtain the data group to be compressed after dimensionality reduction , for the data set to be compressed after dimension reduction Perform predictive coding to obtain the data group to be compressed after predictive coding , for the data group to be compressed after predictive coding Perform adaptive entropy coding to obtain the compressed data set of the data group to be compressed ;

[0103] Step 4: Data transmission module treats the compressed data set as the compressed data set Carry out transmission, monitor the data transmission status in real time, detect and record the relevant data status that occurs during the transmission process.

[0104] In this embodiment, through steps one to four, efficient management of data from collection to transmission is achieved, and the edge gateway collects equipment operating status information in real time to ensure the diversity and comprehensiveness of data sources, and then removes outliers, eliminates noise, smoothes data and performs corrections, laying a solid foundation for subsequent data processing. Data features are extracted and simplified through wavelet transform and PCA dimensionality reduction, and predictive coding and adaptive entropy coding technology are used to greatly compress the data volume, thereby improving the efficiency of data transmission and storage. The data transmission module is not only responsible for the transmission of compressed data, but also monitors the transmission status in real time, detects and records any problems that may arise during the transmission process, and integrates multi-layer optimization of data collection, preprocessing, compression and transmission, which significantly improves the overall performance and reliability of the system and meets the efficient and stable data management needs of the industrial Internet of Things.

[0105] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A data acquisition system for an industrial Internet of Things gateway, characterized in that: It includes data acquisition module, data preprocessing module, data compression module and data transmission module; The data acquisition module is connected to and controls the industrial equipment or PLC through the edge gateway, and collects the internal and external operating status information of the equipment in real time to form a collection data group; The preprocessing module performs preprocessing on the collected data set, including outlier detection, denoising, data smoothing and data correction, to form a processed data set to be compressed. ; The data compression module treats the compressed data group Perform wavelet transform to extract the data group to be compressed Compression features to obtain wavelet coefficients , synchronization of wavelet coefficients Perform PCA dimensionality reduction to obtain the data group to be compressed after dimensionality reduction , for the data set to be compressed after dimension reduction Perform predictive coding to obtain the data group to be compressed after predictive coding , for the data group to be compressed after predictive coding Perform adaptive entropy coding to obtain the compressed data set of the data group to be compressed ; wherein the data group to be compressed after the predictive coding Obtained through the following calculation formula: ; In the formula, Represents the error value of the i-th type of data at time point t, represents the predicted value of the i-th type of data at time point t, represents the i-th type of data after dimensionality reduction at time point t; where the predicted value of the i-th type of data at time point t is according to get, represents the prediction weight value, represents the historical data at time point tk, n represents the total number of historical data collection, and k represents the kth historical data collection; the compressed data set of the data group to be compressed Obtained through the following calculation formula: ; In the formula, represents a compressed data set of the i-th type of data at time point t, It represents the error value of the i-th type of data at time point t, and EntropyEncode represents the entropy coding function used to calculate the error value of the i-th type of data at time point t. Encoding; the reduced dimension data group to be compressed Obtained through the following calculation formula: ; represents the i-th type of data after dimensionality reduction at time point t, represents the wavelet coefficient of the i-th type of data obtained by wavelet transform at time point t, W represents the dimension reduction matrix, which is a matrix composed of the main eigenvectors obtained by PCA principal component analysis; The data transmission module treats the compressed data set of the compressed data set as Carry out transmission, monitor the data transmission status in real time, detect and record the relevant data status that occurs during the transmission process.

2. The data acquisition system of an industrial Internet of Things gateway according to claim 1 is characterized in that: The data acquisition module includes a connection unit and a collection unit; The connection unit connects multiple sensors to industrial equipment or PLC through an edge gateway, establishes a communication channel between the sensors and the industrial equipment or PLC, and performs information communication and transmission of the industrial equipment; The acquisition unit collects status information of the internal and external operating status of the device through dynamically issued data acquisition tasks, and extracts unique tags from the acquisition devices, and combines the unique tags with the collected status information to form the acquisition data group; The collected data group includes a device unique identifier, a sensor unique identifier, a timestamp, a device operating status, device operating environment information, and device operating parameters.

3. The data acquisition system of an industrial Internet of Things gateway according to claim 1 is characterized in that: The data preprocessing module includes a preprocessing unit and an integration unit; The preprocessing unit preprocesses the collected data group, including using statistical methods and machine learning algorithms to calculate the mean and standard deviation related information of the data, and simultaneously identifying abnormal values ​​according to a preset threshold range; using low-pass filters and median filters to eliminate noise in the data; using sliding average and exponential smoothing methods to smooth the data and adjust data fluctuations; obtaining the calibration curve or known deviation of the equipment through the equipment factory operation manual and equipment user manual, and adjusting and correcting the data; The integration unit integrates the pre-processed collected data group to form a processed data group to be compressed ,include{ , , , …, }, t represents the acquisition time point, and i represents the data subscript in the data group.

4. The data acquisition system of an industrial Internet of Things gateway according to claim 1 is characterized in that: The wavelet coefficients Calculated by the following formula: ; In the formula, represents the data after preprocessing of the i-th type collected at time point t, represents the wavelet coefficients of the i-th type of data obtained by wavelet transform at time point t, Represents the wavelet transform function, which is used to transform the time domain data Convert data into frequency domain .

5. The data acquisition system of an industrial Internet of Things gateway according to claim 1 is characterized in that: The data transmission module includes a transmission detection unit and a transmission recording unit; The transmission detection unit treats the compressed data set of the compressed data set as During transmission, real-time monitoring of data transmission status, including network status monitoring and transmission integrity check; The transmission recording unit records relevant status information during the data transmission process, including log records and transmission performance records, provides traceability of the transmission status by recording the status information during the transmission process in detail, and optimizes the transmission strategy by analyzing the transmission records, including adjusting the compression method and parameters of the data compression module.

6. A data collection method for an industrial Internet of Things gateway, using a data collection system for an industrial Internet of Things gateway according to any one of claims 1 to 5, characterized in that: The following steps are involved: Step 1: The data acquisition module connects to and controls industrial equipment or PLC through the edge gateway, and collects the internal and external operating status information of the equipment in real time to form a collection data group; Step 2: The preprocessing module preprocesses the collected data set, including outlier detection, denoising, data smoothing and data correction, to form the processed data set to be compressed. ; Step 3: The data compression module treats the compressed data group Perform wavelet transform to extract the data group to be compressed Compression features to obtain wavelet coefficients , synchronization of wavelet coefficients Perform PCA dimensionality reduction to obtain the data group to be compressed after dimensionality reduction , for the data set to be compressed after dimension reduction Perform predictive coding to obtain the data group to be compressed after predictive coding , for the data group to be compressed after predictive coding Perform adaptive entropy coding to obtain the compressed data set of the data group to be compressed ; including: the data group to be compressed after the predictive coding described in Obtained through the following calculation formula: ; In the formula, Represents the error value of the i-th type of data at time point t, represents the predicted value of the i-th type of data at time point t, represents the i-th type of data after dimensionality reduction at time point t; where the predicted value of the i-th type of data at time point t is according to get, represents the prediction weight value, represents the historical data at time point tk, n represents the total number of historical data collection, and k represents the kth historical data collection; the compressed data set of the data group to be compressed Obtained through the following calculation formula: ; In the formula, represents a compressed data set of the i-th type of data at time point t, It represents the error value of the i-th type of data at time point t, and EntropyEncode represents the entropy coding function used to calculate the error value of the i-th type of data at time point t. Encoding; the reduced dimension data group to be compressed Obtained through the following calculation formula: ; represents the i-th type of data after dimensionality reduction at time point t, represents the wavelet coefficient of the i-th type of data obtained by wavelet transform at time point t, W represents the dimension reduction matrix, which is a matrix composed of the main eigenvectors obtained by PCA principal component analysis; Step 4: Data transmission module treats the compressed data set as the compressed data set Carry out transmission, monitor the data transmission status in real time, detect and record the relevant data status that occurs during the transmission process.

Citation Information

Patent Citations

  • Data acquisition system based on edge computing

    CN116055525B

  • Edge computing system based on industrial Internet of Things platform

    CN117097727A