Internet of Things data compression and reconstruction method and system based on deep compressed sensing

Abstract

The invention relates to the technical field of data processing, in particular to an Internet of Things data compression and reconstruction method and system based on deep compressed sensing, and the method comprises the steps: taking data collected by a plurality of sensors as original data, and carrying out the compressed sensing to obtain compressed data; decoding and reconstructing the compressed data; mapping the compressed data to an original space to obtain initialized data, and inputting the initialized data to a residual decoder to obtain decoded data; mapping decoding data of the residual decoder to a high-dimensional embedding space, and performing data reconstruction to obtain output data; and mapping the output data to the original space through the full connection layer to obtain reconstructed data which is the original data. And a learnable measurement matrix is adaptively generated by adopting the residual error sensing block, so that the data transmission quantity in the Internet of Things is reduced, and the data reconstruction precision is improved. The residual decoder and the data reconstruction decoder are combined, and the reconstruction of the Internet of Things data is completed by extracting key features in the compressed data.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for compressing and reconstructing Internet of Things data based on deep compressed sensing. Background Art

[0002] With the rapid development of Internet of Things technology, more and more sensor devices are connected to the network, resulting in a sharp increase in the amount of data in the Internet of Things, an increase in data transmission delay and packet loss rate, which affects the performance of sensor devices. The amount of data can be reduced by data compression, but the encoding and decoding in traditional data compression methods are relatively complex.

[0003] Compressed sensing is a signal acquisition and reconstruction method that separates encoding and decoding into independent measurement and reconstruction, and compresses and samples the original signal by transforming spatial projection, greatly reducing the amount of data acquisition. However, the data reconstruction performance of compressed sensing depends on the design of the measurement matrix, and the data reconstruction speed is slow. Combining deep learning and compressed sensing can not only achieve high-precision data reconstruction, but also reduce computational complexity and data storage requirements. Therefore, how to use deep compressed sensing to achieve efficient compression and accurate reconstruction of IoT data is a key issue. Summary of the invention

[0004] To solve the above problems, the present invention provides an Internet of Things data compression and reconstruction method based on deep compressed sensing, which is applied to an Internet of Things system. The Internet of Things system includes several sensors. The method includes: S1. Collect data from several sensors as raw data X , input to the compression module for compressed sensing to obtain compressed data; S2, inputting the compressed data into a reconstruction module for decoding and reconstruction; S2.1, mapping the compressed data to the original space to obtain the initialization data, and inputting it into the residual decoder of the reconstruction module to obtain the decoded data; S2.2, mapping the decoded data of the residual decoder to a high-dimensional embedding space, and reconstructing the data through a data reconstruction decoder to obtain output data; S2.3. Map the output data of the data reconstruction decoder to the original space through a fully connected layer to obtain reconstructed data, which is the original data.

[0005] The input in S1 is sent to the compression module for compressed sensing to obtain compressed data. The specific operation is: S1.1. Raw data X Input to the fully connected layer of the residual perception block in the compression module for redundant processing to obtain data x 1 ; S1.2. Data x 1 Perform layer normalization and obtain data 2 through ReLU activation function x 2 ; S1.3、Data 2 x 2 The data is obtained by linear transformation through the fully connected layer. x 3 , the data three x 3 With the original data X Perform residual connection to obtain data four x 4 ; S1.4, the residual perception block adaptively generates a learnable measurement matrix based on the data x 4 The learnable measurement matrix is ​​trained and optimized to obtain the optimal measurement matrix A ; S1.5, based on the optimal measurement matrix A , the compressed data obtained by compressing the original data y , y = AX .

[0006] In a specific embodiment, the residual decoder comprises three identical residual blocks.

[0007] The initialization data in S2 is input to the residual decoder of the reconstruction module to obtain the decoded data: , in, is the decoded data of the residual decoder; represents the residual block, Indicates The output data of the residual block.

[0008] The calculation and processing operations of the residual blocks on the input data include: a, batch normalize the input data, and then use the ReLU activation function to get the output data ; ; b, will output data one Repeat operation a for the second batch normalization and ReLU activation function to get output data 2 ; ; c, output data two Perform residual connection with the input data to obtain the output data of the residual block; , in, is the output data of the first residual block; Represents initialization data.

[0009] The data reconstruction decoder includes three identical window data reconstruction blocks.

[0010] The output data of the data reconstruction decoder in S2 is: , in, The output data of the decoder is reconstructed for data, Represents the window data reconstruction block, Indicates The output data of the block is reconstructed from the window data.

[0011] The calculation processing of the input data by each window data reconstruction block includes: A. Divide the high-dimensional embedding space into several local windows of size w×w, and calculate multi-head self-attention in each local window; B. Calculate the attention weight of each single-head self-attention and obtain the output data of the single-head self-attention; C. Merge the output data of all single-head self-attention in the local window and perform linear projection to obtain the merged data of the local window; D. Normalize the merged data first, and then perform residual connection with the original merged data to obtain the first output data; E. Pass the first output data through a multi-layer perceptron layer to obtain second output data; perform layer normalization on the first output data, and then perform residual connection with the second output data to obtain the output data of the window data reconstruction block.

[0012] The present invention also provides an Internet of Things data compression and reconstruction system based on deep compressed sensing, comprising: Data compression module, used to collect data from several sensors as raw data X , input to the compression module for compressed sensing to obtain compressed data; The decoding and reconstruction module is used to input the compressed data into the reconstruction module for decoding and reconstruction; the compressed data is mapped to the original space to obtain the initialization data, and the initialization data is input into the residual decoder of the reconstruction module to obtain the decoded data; Mapping the decoded data of the residual decoder to a high-dimensional embedding space, and reconstructing the data through a data reconstruction decoder to obtain output data; The output data of the data reconstruction decoder is mapped to the original space through a fully connected layer to obtain reconstructed data, which is the original data.

[0013] Beneficial effects: The present invention is a method and system for Internet of Things data compression and reconstruction based on deep compressed sensing, including a compression module and a reconstruction module. The compression module uses a residual sensing block to adaptively generate a learnable measurement matrix, reduce the amount of data transmission in the Internet of Things, and improve the accuracy of data reconstruction. The reconstruction module combines the residual decoder and the data reconstruction decoder to complete the reconstruction of the Internet of Things data by extracting key features from the compressed data. The present invention designs an end-to-end model, in which compression and reconstruction are jointly optimized, and all parameters are updated together, which can improve compression efficiency and reconstruction quality. DETAILED DESCRIPTION

[0014] Example This embodiment provides an IoT data compression and reconstruction method based on deep compressed sensing, which is applied to an IoT system, wherein the IoT system includes a plurality of sensors. By constructing an end-to-end deep compressed sensing model, the deep compressed sensing module includes a compression module and a reconstruction module.

[0015] Based on the compression module and the reconstruction module, the data of each sensor in the Internet of Things system is compressed and reconstructed respectively. The specific implementation steps of the Internet of Things data compression and reconstruction method are as follows: S1. Collect data from several sensors as raw data X , input to the compression module for compressed sensing to obtain compressed data; Each sensor in the Internet of Things collects IoT data at fixed time intervals. The data collected by N sensor devices n times is taken as the original data and expressed as ; The original data is input into a compression module for compressed sensing to obtain compressed data. The compression module includes a residual sensing block, and the specific operation is as follows: S1.1. Raw data X Input to the fully connected layer of the residual perception block in the compression module for redundant processing to obtain data x 1 ; IoT data is usually high-dimensional, and data from different sensors may be redundant. The fully connected layer is used to reduce redundancy so that IoT data can adapt to the network input. The redundancy processing is as follows: , in, x 1 Indicates data one, is the dimension transformation matrix.

[0016] S1.2. Data x 1 Perform layer normalization and obtain data 2 through ReLU activation function x 2 ; The second data is: , in, is the weight matrix, is the bias term; LayerNorm is layer normalization.

[0017] S1.3、Data 2 x 2 The data is obtained by linear transformation through the fully connected layer. x 3 , the data three x 3 With the original data X Perform residual connection to obtain data four x 4 ; , , in, and They represent weight matrices respectively.

[0018] S1.4, the residual perception block adaptively generates a learnable measurement matrix based on the data x 4 The learnable measurement matrix is ​​trained and optimized to obtain the optimal measurement matrix A ; The characteristics of IoT data vary greatly over time, and a fixed measurement matrix is ​​difficult to adapt to the dynamic characteristics of IoT data. Therefore, a learnable measurement matrix is ​​adaptively generated through the residual perception block, and the learnable measurement matrix is ​​trained and optimized, so that the residual perception block retains more effective information of the original data during compressed sensing, thereby improving the performance of compressed sensing and the quality of later reconstruction of compressed data.

[0019] Based on data four x 4 The optimal measurement matrix obtained by training and optimizing the learnable measurement matrix is: , in, represents the optimal measurement matrix, , To compress the dimension, is the residual perception block, To optimize the parameters.

[0020] S1.5, based on the optimal measurement matrix A , the compressed data obtained by compressing the original data y , y = AX .

[0021] S2, inputting the compressed data into a reconstruction module for decoding and reconstruction, wherein the reconstruction module includes a residual decoder and a data reconstruction decoder; S2.1, mapping the compressed data to the original space to obtain the initialization data, and inputting it into the residual decoder of the reconstruction module to obtain the decoded data; Compress the data y Mapped to the original space, the original space is the dimension of the original data of the Internet of Things, and the initialization data is generated : , in, is the dimension transformation matrix, is the bias term, is batch normalization, is the activation function.

[0022] In the network structure, if only fully connected layers or convolutional layers are used for transformation, important feature information of the data may be lost. The superposition of multiple residual blocks can enhance the nonlinear expression ability of the model, enable it to better adapt to complex data patterns, and improve reconstruction accuracy.

[0023] The initialization data is input to the residual decoder for decoding. The residual decoder includes three identical residual blocks. The residual decoder decodes the input initialization data to obtain the decoded data Z: , in, is the decoded data of the residual decoder; represents the residual block, Indicates The output data of the residual block.

[0024] The calculation and processing operations of the residual blocks on the input data include: a, batch normalize the input data, and then use the ReLU activation function to get the output data ; ; b, will output data one Repeat operation a for the second batch normalization and ReLU activation function to get output data 2 , further extract data features; ; c, output data two Perform residual connection with the input data to obtain the output data of the residual block; , in, is the output data of the first residual block.

[0025] The first residual block is initialized with data As input data, perform steps a~c to obtain the first residual block output data Z1; use the first residual block output data Z1 as the input data of the second residual block, and perform steps a~c to obtain the second residual block output data Z2; use the second residual block output data Z2 as the input data of the third residual block, and perform steps a~c to obtain the third residual block output data Z3.

[0026] S2.2, mapping the decoded data of the residual decoder to a high-dimensional embedding space, and reconstructing the data through a data reconstruction decoder to obtain output data.

[0027] Mapping the decoded output data of the residual decoder to the high-dimensional embedding space, we get: , in, is the embedding matrix, For high-order embedded spatial data, , is the sequence length, is the embedding dimension.

[0028] The data reconstruction decoder includes three identical window data reconstruction blocks, each of which includes window division, window self-attention mechanism, window merging, multi-layer perceptron layer and residual connection.

[0029] The high-dimensional embedded space data is input into the data reconstruction decoder for data reconstruction. The output data of the data reconstruction decoder is: , in, The output data of the decoder is reconstructed for data, Represents the window data reconstruction block, Indicates The output data of the window data reconstruction block; The calculation and processing operation of each window data reconstruction block on the input data is: A. Divide the high-dimensional embedding space into several local windows of size w×w, and calculate multi-head self-attention in each local window; For each data in each local window, calculate the query of each single-head self-attention Q ,key K and value V ; Among them, the query of the h-th single-head self-attention Q n ,key K h With value V h They are: , in, Indicates the features that the data in the current window focuses on when calculating the attention weight. Represents the feature representation of the data in the current window, Used to adjust output data according to attention weights; For the The weight matrix of a single-head self-attention, Display key Dimension.

[0030] B. Calculate the attention weight of each single-head self-attention and obtain the output data of the single-head self-attention; , in, is the normalization function, Attention h represents the attention weight of the h-th single-head self-attention.

[0031] C. Merge the output data of all single-head self-attention in the local window and perform linear projection to obtain the merged data of the local window ,By combining the features of different attention heads, the reconstruction and recovery capabilities of data details are improved; , in, is the linear projection matrix, The function connects the outputs of multiple attention heads together.

[0032] D. Merge data First perform layer normalization, then perform residual connection with the original merged data to obtain the first output data ; ; E. The first output data After the multi-layer perceptron layer, the second output data is obtained ; The first output data Perform layer normalization and then compare it with the second output data Perform residual connection to obtain the output data of the window data reconstruction block. The specific formula is: , in, , is the weight matrix, , is the bias term, is the activation function; .

[0033] The first window data reconstruction block is embedded in the high-dimensional spatial data As input data, perform steps A to F to obtain the first window data reconstruction block output data ; Reconstruct the first window data block output data As the input data of the second window data reconstruction block, perform steps A to F to obtain the output data of the second window data reconstruction block ; Reconstruct the second window data block output data As the input data of the third window data reconstruction block, perform steps A to F to obtain the output data of the first window data reconstruction block ; S2.3, mapping the output data of the data reconstruction decoder to the original space through a fully connected layer to obtain reconstructed data, where the reconstructed data is the original data; , in, is the dimension transformation matrix.

[0034] In addition, a data compression and reconstruction system for the Internet of Things based on deep compressed sensing is provided, including: Data compression module, used to collect data from several sensors as raw data X , input to the compression module for compressed sensing to obtain compressed data; The decoding and reconstruction module is used to input the compressed data into the reconstruction module for decoding and reconstruction; the compressed data is mapped to the original space to obtain the initialization data, and the initialization data is input into the residual decoder of the reconstruction module to obtain the decoded data; Mapping the decoded data of the residual decoder to a high-dimensional embedding space, and reconstructing the data through a data reconstruction decoder to obtain output data; The output data of the data reconstruction decoder is mapped to the original space through a fully connected layer to obtain reconstructed data, which is the original data.

Claims

1. A method for compressing and reconstructing Internet of Things data based on deep compressed sensing, applied to an Internet of Things system, wherein the Internet of Things system includes a plurality of sensors, characterized in that: include: S1. Collect data from several sensors as raw data X , input to the compression module for compressed sensing to obtain compressed data; S2, inputting the compressed data into a reconstruction module for decoding and reconstruction; S2.1, mapping the compressed data to the original space to obtain the initialization data, and inputting it into the residual decoder of the reconstruction module to obtain the decoded data; S2.2, mapping the decoded data of the residual decoder to a high-dimensional embedding space, and reconstructing the data through a data reconstruction decoder to obtain output data; S2.

3. Map the output data of the data reconstruction decoder to the original space through a fully connected layer to obtain reconstructed data, which is the original data.

2. The method for Internet of Things data compression and reconstruction based on deep compressed sensing according to claim 1 is characterized in that: The input in S1 is sent to the compression module for compressed sensing to obtain compressed data. The specific operation is: S1.

1. Raw data X Input to the fully connected layer of the residual perception block in the compression module for redundant processing to obtain data x 1 ; S1.

2. Data x 1 Perform layer normalization and obtain data 2 through ReLU activation function x 2 ; S1.3、Data 2 x 2 The data is obtained by linear transformation through the fully connected layer. x 3 , the data three x 3 With the original data X Perform residual connection to obtain data four x 4 ; S1.4, the residual perception block adaptively generates a learnable measurement matrix based on the data x 4 The learnable measurement matrix is ​​trained and optimized to obtain the optimal measurement matrix A ; S1.5, based on the optimal measurement matrix A , the compressed data obtained by compressing the original data y , y = AX .

3. The method for Internet of Things data compression and reconstruction based on deep compressed sensing according to claim 1 is characterized in that: The residual decoder includes three identical residual blocks.

4. The method for Internet of Things data compression and reconstruction based on deep compressed sensing according to claim 3 is characterized in that: The initialization data in S2 is input to the residual decoder of the reconstruction module to obtain the decoded data: ， in, is the decoded data of the residual decoder; represents the residual block, Indicates The output data of the residual block.

5. The method for Internet of Things data compression and reconstruction based on deep compressed sensing according to claim 4 is characterized in that: The calculation and processing operations of the residual blocks on the input data include: a, batch normalize the input data, and then use the ReLU activation function to get the output data ; ； b, will output data one Repeat operation a for the second batch normalization and ReLU activation function to get output data 2 ; ； c, output data two Perform residual connection with the input data to obtain the output data of the residual block; ， in, is the output data of the first residual block; Represents initialization data.

6. The method for Internet of Things data compression and reconstruction based on deep compressed sensing according to claim 1 is characterized in that: The data reconstruction decoder includes three identical window data reconstruction blocks.

7. The method for Internet of Things data compression and reconstruction based on deep compressed sensing according to claim 6 is characterized in that: The output data of the data reconstruction decoder in S2 is: ， in, The output data of the decoder is reconstructed for data, Represents the window data reconstruction block, Indicates The output data of the block is reconstructed from the window data.

8. The method for Internet of Things data compression and reconstruction based on deep compressed sensing according to claim 7 is characterized in that: The calculation processing of the input data by each window data reconstruction block includes: A. Divide the high-dimensional embedding space into several local windows of size w×w, and calculate multi-head self-attention in each local window; B. Calculate the attention weight of each single-head self-attention and obtain the output data of the single-head self-attention; C. Merge the output data of all single-head self-attention in the local window and perform linear projection to obtain the merged data of the local window; D. Normalize the merged data first, and then perform residual connection with the original merged data to obtain the first output data; E. Pass the first output data through a multi-layer perceptron layer to obtain second output data; perform layer normalization on the first output data, and then perform residual connection with the second output data to obtain the output data of the window data reconstruction block.

9. An IoT data compression and reconstruction system based on deep compressed sensing, characterized in that: include: Data compression module, used to collect data from several sensors as raw data X , input to the compression module for compressed sensing to obtain compressed data; The decoding and reconstruction module is used to input the compressed data into the reconstruction module for decoding and reconstruction; the compressed data is mapped to the original space to obtain the initialization data, and the initialization data is input into the residual decoder of the reconstruction module to obtain the decoded data; Mapping the decoded data of the residual decoder to a high-dimensional embedding space, and reconstructing the data through a data reconstruction decoder to obtain output data; The output data of the data reconstruction decoder is mapped to the original space through a fully connected layer to obtain reconstructed data, which is the original data.

Images