Internet of Things data transmission and recovery system and method based on adaptive compressed sensing

Abstract

The invention relates to the technical field of Internet of Things, in particular to an Internet of Things data transmission and recovery system and method based on adaptive compressed sensing, and the system comprises a sensor node which is used for collecting data and transmitting the data to an adaptive compressed sensing module; the self-adaptive compressed sensing module is used for dynamically adjusting a sampling matrix and a compression ratio according to the sparsity and correlation of the sensor data; the data transmission module is used for transmitting the compressed data to the cloud; the cloud reconstruction module is used for reconstructing the compressed data by adopting a greedy algorithm and recovering an approximate value of the original data; and the data recovery module optimizes the reconstructed data by using a deep learning technology, and improves the accuracy and robustness of data recovery, and according to the Internet of Things data transmission and recovery system and method based on adaptive compressed sensing, the data transmission efficiency can be improved, the data quality can be ensured, the energy consumption can be reduced, and the system reliability can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of the Internet of Things, and specifically to an Internet of Things data transmission and recovery system and method based on adaptive compressive sensing. Background Art

[0002] In an Internet of Things (IoT) system, sensor nodes need to transmit the collected data to the cloud for further processing. However, due to bandwidth and energy consumption limitations, the data transmission efficiency is low, and a large amount of data causes resource waste during transmission. In the prior art, compression algorithms are usually used to compress sensor data to reduce the amount of data transmitted.

[0003] However, these traditional compression algorithms often cannot be adaptively adjusted according to the characteristics of the data, resulting in low data recovery quality. Specifically, traditional compression algorithms cannot fully consider the sparsity and correlation of sensor data, etc., so a large amount of key information is lost during the compression process. Therefore, in view of the above situation, there is an urgent need to develop an Internet of Things data transmission and recovery system and method based on adaptive compressive sensing to overcome the deficiencies in current practical applications. Summary of the Invention

[0004] The purpose of the present invention is to provide an Internet of Things data transmission and recovery system and method based on adaptive compressive sensing to solve the problems raised in the above background art.

[0005] To achieve the above purpose, the present invention provides the following technical solutions:

[0006] An Internet of Things data transmission and recovery system based on adaptive compressive sensing, comprising:

[0007] Sensor nodes, used for collecting data and transmitting the data to the adaptive compressive sensing module;

[0008] An adaptive compressive sensing module, used for dynamically adjusting the sampling matrix and the compression ratio according to the sparsity and correlation of the sensor data, wherein the sampling matrix Φ satisfies the following conditions:

[0009] The number of rows of Φ is less than the dimension of the original data to achieve compression;

[0010] The number of columns of Φ is the same as the dimension of the original data to ensure data integrity;

[0011] Each column of Φ is an orthogonal vector to ensure data sparsity;

[0012] A data transmission module, used for transmitting the compressed data to the cloud;

[0013] A cloud reconstruction module, which uses a greedy algorithm to reconstruct the compressed data and recover an approximation of the original data;

[0014] A data recovery module that uses deep learning technology to optimize the reconstructed data and improve the accuracy and robustness of data recovery.

[0015] As a further solution of the present invention: The adaptive compressive sensing module further includes a dictionary learning algorithm for constructing a sparse dictionary suitable for different sensor types and representing the sensor data as a sparse signal.

[0016] As a further solution of the present invention: The greedy algorithm is an orthogonal matching pursuit algorithm, which specifically includes the following steps:

[0017] Initialize the residual and the index set;

[0018] Iteratively select the column in the dictionary that has the highest correlation with the current residual;

[0019] Update the index set and the temporary coefficient vector;

[0020] Calculate the final coefficient vector and reconstruct the data.

[0021] As a further solution of the present invention: The deep learning technology adopted by the data recovery module includes a convolutional neural network or a recurrent neural network, and the generalization ability of the model is improved through data augmentation technology.

[0022] As a further solution of the present invention: The data transmission module adopts a wireless transmission method, including Wi-Fi, LoRa or NB-IoT, and uses encryption technology to protect data security during transmission.

[0023] An Internet of Things data transmission and recovery method based on adaptive compressive sensing, including the following steps:

[0024] The sensor node collects data and transmits the data to the adaptive compressive sensing module;

[0025] The adaptive compressive sensing module dynamically adjusts the sampling matrix and the compression ratio according to the sparsity and correlation of the sensor data;

[0026] The data transmission module transmits the compressed data to the cloud;

[0027] The cloud reconstruction module uses a greedy algorithm to reconstruct the compressed data;

[0028] The data recovery module uses deep learning technology to optimize the reconstructed data.

[0029] As a further solution of the present invention: The dynamic adjustment of the compression ratio includes the following steps:

[0030] Calculate the sparsity of the sensor data;

[0031] Dynamically select the number of rows of the sampling matrix Φ according to the comparison result between the sparsity and the preset threshold, so that the data with higher sparsity corresponds to fewer rows.

[0032] As a further solution of the present invention: The greedy algorithm is the OMP algorithm, which specifically includes the following steps:

[0033] Initialize the residual and the index set;

[0034] Iteratively select the column in the dictionary that has the greatest correlation with the current residual;

[0035] Update the index set and the temporary coefficient vector;

[0036] Calculate the final coefficient vector and reconstruct the data.

[0037] As a further solution of the present invention: The data recovery module learns the feature representation and distribution law of the data by training a neural network model to optimize the reconstructed data.

[0038] As a further solution of the present invention: The data transmission module adopts a multi-path transmission strategy and encryption technology during the transmission process to improve the reliability and security of data transmission.

[0039] Compared with the prior art, the beneficial effects of the present invention are:

[0040] 1. Improve data transmission efficiency: Significantly reduce the amount of transmitted data through the adaptive compression algorithm and reduce bandwidth consumption;

[0041] 2. Ensure data quality: Utilize adaptive compression and deep learning technologies to ensure the accuracy and robustness of data recovery;

[0042] 3. Reduce energy consumption: Reducing the amount of data transmission means reducing the energy consumption of sensor nodes;

[0043] 4. Improve system reliability: The multi-path transmission strategy and encryption technology enhance the reliability and security of data transmission;

[0044] 5. Support intelligent applications: Provide strong support for the intelligent applications of the Internet of Things system. Description of the Drawings

[0045] Figure 1 It is the architecture diagram of the Internet of Things data transmission and recovery system based on adaptive compressive sensing in the embodiment of the present invention.

[0046] Figure 2 It is the flowchart of the adaptive compressive sensing data transmission and recovery method in the embodiment of the present invention. Detailed Embodiments

[0047] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0048] The following describes the specific implementation of the present invention in detail in conjunction with specific embodiments.

[0049] Please refer to Figure 1 and Figure 2 , the Internet of Things data transmission and recovery system based on adaptive compressive sensing provided by the embodiments of the present invention includes:

[0050] Sensor nodes, which are used to collect data and transmit the data to the adaptive compressive sensing module;

[0051] The adaptive compressive sensing module is used to dynamically adjust the sampling matrix and the compression ratio according to the sparsity and correlation of the sensor data. Among them, the sampling matrix Φ satisfies the following conditions:

[0052] The number of rows of Φ is less than the dimension of the original data to achieve compression;

[0053] The number of columns of Φ is the same as the dimension of the original data to ensure data integrity;

[0054] Each column of Φ is an orthogonal vector to ensure data sparsity;

[0055] The data transmission module is used to transmit the compressed data to the cloud;

[0056] The cloud reconstruction module uses a greedy algorithm to reconstruct the compressed data and restore an approximation of the original data;

[0057] The data recovery module uses deep learning technology to optimize the reconstructed data and improve the accuracy and robustness of data recovery.

[0058] The adaptive compressive sensing module further includes a dictionary learning algorithm, which is used to construct a sparse dictionary suitable for different sensor types and represent the sensor data as a sparse signal.

[0059] The greedy algorithm is the orthogonal matching pursuit (OMP) algorithm, which specifically includes the following steps:

[0060] Initialize the residual and the index set;

[0061] Iteratively select the column in the dictionary that has the largest correlation with the current residual;

[0062] Update the index set and the temporary coefficient vector;

[0063] Calculate the final coefficient vector and reconstruct the data.

[0064] The deep learning technology adopted by the data recovery module includes Convolutional Neural Network (CNN) or Recurrent Neural Network (RNN), and the generalization ability of the model is improved through data augmentation technology.

[0065] The data transmission module adopts wireless transmission methods, including Wi-Fi, LoRa or NB-IoT, and uses encryption technology to protect data security during transmission.

[0066] The Internet of Things data transmission and recovery method based on adaptive compressive sensing includes the following steps:

[0067] The sensor node collects data and transmits the data to the adaptive compressive sensing module;

[0068] The adaptive compressive sensing module dynamically adjusts the sampling matrix and compression ratio according to the sparsity and correlation of the sensor data;

[0069] The data transmission module transmits the compressed data to the cloud;

[0070] The cloud reconstruction module reconstructs the compressed data using the greedy algorithm;

[0071] The data recovery module optimizes the reconstructed data using deep learning technology.

[0072] The specific steps for dynamically adjusting the compression ratio include:

[0073] For data with a higher sparsity, use a higher compression ratio;

[0074] For data with a lower sparsity, use a lower compression ratio.

[0075] The greedy algorithm is the OMP algorithm, which specifically includes the following steps:

[0076] Initialize the residual and index set;

[0077] Iteratively select the column in the dictionary that has the highest correlation with the current residual;

[0078] Update the index set and the temporary coefficient vector;

[0079] Calculate the final coefficient vector and reconstruct the data.

[0080] The data recovery module learns the feature representation and distribution law of the data by training a neural network model to realize the optimization of the reconstructed data.

[0081] The data transmission module adopts a multi-path transmission strategy and encryption technology during transmission to improve the reliability and security of data transmission.

[0082] Example 1: The Internet of Things data transmission and recovery system based on adaptive compressive sensing of the present invention includes a sensor node, an adaptive compressive sensing module, a data transmission module, a cloud reconstruction module, and a data recovery module. The sensor node is responsible for collecting data. The adaptive compressive sensing module adaptively compresses according to the characteristics of the data. The data transmission module transmits the compressed data to the cloud. The cloud reconstruction module reconstructs the compressed data. The data recovery module uses deep learning technology to improve the accuracy and robustness of data recovery. The specific steps are as follows:

[0083] Step 1: The sensor node collects data and transmits the data to the adaptive compressive sensing module. The sensor node includes, but is not limited to, temperature sensors, humidity sensors, pressure sensors, etc. The collected data is an analog signal, which is converted into a digital signal by an analog-to-digital converter and then transmitted to the adaptive compressive sensing module.

[0084] Step 2: The adaptive compressive sensing module dynamically adjusts the sampling matrix and compression ratio according to the characteristics such as sparsity and correlation of the sensor data. For data with a higher sparsity, a higher compression ratio is used; for data with a lower sparsity, a lower compression ratio is used.

[0085] In addition, the adaptive compressive sensing module also uses a dictionary learning algorithm to construct a sparse dictionary suitable for different sensor types, represents the sensor data as a sparse signal, and further reduces the data volume. The dictionary learning algorithm learns an overcomplete dictionary from the sensor data through iterative optimization, so that the sensor data can be represented as a linear combination of a small number of atoms in the dictionary.

[0086] Among them, the dictionary learning algorithm is used to create a "dictionary" that contains elements that can represent the characteristics of different sensor data. Through this dictionary, the data collected by the sensors can be converted into a simpler and more easily processed format, that is, a sparse signal. A sparse signal means that most elements of the data are zero, and only a few are non-zero, which can reduce the data volume and facilitate transmission and storage. For example, for an image sensor, the dictionary learning algorithm may identify some basic image blocks (such as edges, textures, etc.), and then use a combination of these basic blocks to represent the original image. In this way, the originally complex image data can be represented by a combination of a few basic blocks, thereby achieving data compression.

[0087] Step 3: The data transmission module transmits the compressed data to the cloud. The data transmission module uses a wireless transmission method, such as Wi-Fi, LoRa, and NB-IoT, etc., to transmit the compressed data to the cloud server. During the transmission process, the data transmission module also uses encryption technology to protect the security of the data.

[0088] Among them, in order to protect the security of data during transmission, the data transmission module uses encryption technology. Encryption technology can convert the original data into a format that can only be interpreted by those who possess the correct key. For example, if the sensor node transmits temperature data, encryption technology can ensure that even if these data are intercepted during wireless transmission, they cannot be interpreted by a third party unless they have the correct decryption key. In this way, even if the data are intercepted during transmission, those without the key cannot obtain the content of the original data, thus protecting the security of the data.

[0089] Step 4: The cloud reconstruction module uses the greedy algorithm to reconstruct the compressed data and recover an approximation of the original data. The greedy algorithm iteratively selects the optimal combination of atoms to gradually approximate the original data and achieve data reconstruction. The cloud reconstruction module also utilizes sparse representation technology to further reduce the amount of data according to the sparsity of the reconstructed data.

[0090] Step 5: The data recovery module uses deep learning techniques (such as CNN, RNN) to further optimize the reconstructed data and improve the accuracy and robustness of data recovery. Deep learning techniques train neural network models to learn the feature representations and distribution laws of the data, and achieve the optimization and recovery of the reconstructed data. The data recovery module also adopts data augmentation techniques, such as noise addition, rotation, and scaling, to improve the generalization ability and robustness of the neural network model.

[0091] Step 6: The cloud processes and analyzes the recovered data to achieve intelligent applications of the Internet of Things system. The cloud server stores, processes, and analyzes the recovered data, extracts valuable information and knowledge, and provides support for the intelligent applications of the Internet of Things system. For example, in a smart home system, the cloud server can process and analyze the recovered temperature and humidity data, etc., to achieve automatic control and optimization of the smart home.

[0092] Step 7: System performance evaluation and optimization. Evaluate the performance of the system, including indicators such as data transmission efficiency, data recovery quality, and energy consumption. According to the evaluation results, optimize and improve the system to enhance the performance and stability of the system.

[0093] Embodiment 2: In Step 2, the adaptive compressive sensing algorithm uses the following formula for compression:

[0094] Y = ΦX;

[0095] where Y is the compressed data, Φ is the sampling matrix, and X is the original data.

[0096] The design of the sampling matrix Φ needs to meet the following conditions:

[0097] 1) The number of rows of Φ is much smaller than the number of columns, that is, the number of rows of Φ is less than the dimension of the original data, so as to achieve data compression;

[0098] 2) The number of columns of Φ is the same as the dimension of the original data to ensure data integrity;

[0099] 3) Each column of Φ is orthogonal to ensure data sparsity.

[0100] Among them, the sparsity calculation and compression ratio adjustment rules are as follows:

[0101] The sparsity is determined by calculating the L0 norm (number of non-zero elements) of the data or the energy concentration (such as the energy proportion of the first 10% coefficients after DCT transformation);

[0102] The preset threshold is set according to the sensor type (for example: the sparsity threshold S for temperature data = 0.3, and S for humidity data = 0.5);

[0103] When the sparsity > S, a high compression ratio is adopted (for example, the number of rows of the sampling matrix Φ is 10% of the dimension of the original data); when the sparsity ≤ S, a low compression ratio is adopted (such as 30% of the number of rows).

[0104] Example 3: In step 4, the specific implementation method is as follows:

[0105] 1) Initialization:

[0106] a. Let the original data matrix be (X), with a size of (m\times n).

[0107] b. Let the compressed data matrix be (Y), with a size of (k\times n), where (k < m).

[0108] c. Let the dictionary matrix be (D), with a size of (m\times k), where (D) is obtained by a certain method (such as principal component analysis, independent component analysis, etc.) and is used to project the original data (X) into a low-dimensional space.

[0109] 2) The greedy algorithm selects the optimal atom:

[0110] a. The Orthogonal Matching Pursuit (OMP) algorithm is used as an example of the greedy algorithm.

[0111] b. Initialize the residual (r_0 = Y), and the index set (I = \emptyset).

[0112] c. For (t = 1) to (m):

[0113] a) Find the column ($d_{i_t}$) in the dictionary ($D$) that has the highest correlation with the current residual ($r_{t - 1}$), i.e., ($i_t=\arg\max_{i}|d_i^T r_{t - 1}|$).

[0114] b) Update the index set ($I = I\cup\{i_t\}$).

[0115] c) Calculate the temporary coefficient vector ($h_t=\arg\min_{h}||Y - D_{I}h||_2^2$), where ($D_{I}$) is the matrix composed of the dictionary columns corresponding to the indices in ($I$).

[0116] d) Update the residual ($r_t = Y - D_{I}h_t$).

[0117] d. Obtain the final coefficient vector ($h = h_m$).

[0118] 3) Data reconstruction:

[0119] a. Reconstruct the data using the obtained coefficient vector ($h$) and the dictionary matrix ($D$): ($\hat{X}=D h$).

[0120] b. ($\hat{X}$) is an approximation of the original data ($X$).

[0121] 4) Sparse representation technique:

[0122] a. Further reduce the data volume using the sparse representation technique.

[0123] b. Apply a sparse coding algorithm (such as basis pursuit, LASSO, etc.) to the reconstructed data ($\hat{X}$) to find a sparse coefficient vector ($s$) that minimizes ($||\hat{X}-D s||_2^2$), while the number of non - zero elements in ($s$) is as small as possible.

[0124] c. Through the sparse coefficient vector ($s$) and a sparse dictionary ($D_s$) (which can be a subset of ($D$) or a completely different dictionary), a sparser representation can be obtained: ($\hat{X}_s = D_s s$).

[0125] 5) Output result:

[0126] Output the reconstructed data ($\hat{X}_s$), which is a sparser representation than the original data ($X$), thus achieving data compression and reconstruction.

[0127] It should be noted that in the present invention, it should be understood that although this specification is described according to the embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. An Internet of Things data transmission and recovery system based on adaptive compressive sensing, characterized in that, including: Sensor nodes, which are used to collect data and transmit the data to the adaptive compressive sensing module; The adaptive compressive sensing module, which is used to dynamically adjust the sampling matrix and the compression ratio according to the sparsity and correlation of the sensor data. Among them, the sampling matrix Φ satisfies the following conditions: The number of rows of Φ is less than the dimension of the original data to achieve compression; The number of columns of Φ is the same as the dimension of the original data to ensure data integrity; Each column of Φ is an orthogonal vector to ensure data sparsity; The data transmission module, which is used to transmit the compressed data to the cloud; The cloud reconstruction module, which uses a greedy algorithm to reconstruct the compressed data and recover an approximation of the original data; The data recovery module, which uses deep learning technology to optimize the reconstructed data and improve the accuracy and robustness of data recovery.

2. The Internet of Things data transmission and recovery system based on adaptive compressive sensing according to claim 1, wherein, The adaptive compressive sensing module further includes a dictionary learning algorithm, which is used to construct a sparse dictionary suitable for different sensor types and represent the sensor data as a sparse signal.

3. The Internet of Things data transmission and recovery system based on adaptive compressive sensing according to claim 1, characterized in that, The greedy algorithm is the orthogonal matching pursuit algorithm, which specifically includes the following steps: Initialize the residual and the index set; Iteratively select the column in the dictionary that has the highest correlation with the current residual; Update the index set and the temporary coefficient vector; Calculate the final coefficient vector and reconstruct the data.

4. The Internet of Things data transmission and recovery system based on adaptive compressive sensing according to claim 1, wherein The deep learning technology adopted by the data recovery module includes a convolutional neural network or a recurrent neural network, and the generalization ability of the model is improved through data augmentation technology.

5. The Internet of Things data transmission and recovery system based on adaptive compressive sensing according to claim 1, wherein The data transmission module adopts a wireless transmission method, including Wi-Fi, LoRa or NB-IoT, and uses encryption technology to protect data security during the transmission process.

6. An Internet of Things data transmission and recovery method based on adaptive compressive sensing, characterized in that, including the following steps: The sensor node collects data and transmits the data to the adaptive compressive sensing module; The adaptive compressive sensing module dynamically adjusts the sampling matrix and the compression ratio according to the sparsity and correlation of the sensor data; The data transmission module transmits the compressed data to the cloud; The cloud reconstruction module uses a greedy algorithm to reconstruct the compressed data; The data recovery module uses deep learning technology to optimize the reconstructed data.

7. The method for transmitting and recovering Internet of Things data based on adaptive compressive sensing according to claim 6, wherein, The dynamic adjustment of the compression ratio includes the following steps: Calculate the sparsity of the sensor data; According to the comparison result between the sparsity and the preset threshold, dynamically select the number of rows of the sampling matrix Φ, so that the data with higher sparsity corresponds to fewer rows.

8. The method for transmitting and recovering Internet of Things data based on adaptive compressive sensing according to claim 6, characterized in that, The greedy algorithm is the OMP algorithm, which specifically includes the following steps: Initialize the residual and the index set; Iteratively select the column in the dictionary that has the highest correlation with the current residual; Update the index set and the temporary coefficient vector; Calculate the final coefficient vector and reconstruct the data.

9. The method for transmitting and restoring Internet of Things data based on adaptive compressive sensing according to claim 6, wherein, The data recovery module learns the feature representation and distribution law of the data by training a neural network model to realize the optimization of the reconstructed data.

10. The method for transmitting and recovering Internet of Things data based on adaptive compressive sensing according to claim 6, wherein The data transmission module adopts a multi-path transmission strategy and encryption technology during the transmission process to improve the reliability and security of data transmission.

Images