Internet of Things Data Compression and Reconstruction Method and System Based on Deep Compressed Sensing

Abstract

The present invention relates to the technical field of data processing, and specifically provides a method and system for compressing and reconstructing Internet of Things data based on deep compressive sensing. The collected data of several sensors is used as the original data, and compressive sensing is performed to obtain compressed data; the compressed data is decoded and reconstructed; the compressed data is mapped to the original space to obtain initialization data, which is input into the residual decoder to obtain decoded data; the decoded data of the residual decoder is mapped to a high-dimensional embedding space for data reconstruction to obtain output data; the output data is mapped to the original space through a fully connected layer to obtain reconstructed data, and the reconstructed data is the original data. A residual sensing block is used to adaptively generate a learnable measurement matrix, reducing the data transmission volume in the Internet of Things and improving the data reconstruction accuracy. The residual decoder and the data reconstruction decoder are combined to complete the reconstruction of Internet of Things data by extracting key features from the compressed data.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and specifically to an Internet of Things data compression and reconstruction method and system based on deep compressive sensing. Background Art

[0002] With the rapid development of Internet of Things technology, more and more sensing devices are connected to the network, resulting in a sharp increase in the amount of data in the Internet of Things, an increase in data transmission delay and packet loss rate, thus affecting the performance of sensing devices. By means of data compression, the amount of data can be reduced, but both encoding and decoding in traditional data compression methods are relatively complex.

[0003] Compressive sensing is a signal acquisition and reconstruction method that separates encoding and decoding into independent measurement and reconstruction, and compresses and samples the original signal through the method of transform space projection, greatly reducing the amount of data acquisition. However, the data reconstruction performance of compressive sensing depends on the design of the measurement matrix, and the data reconstruction speed is relatively slow. Combining deep learning and compressive sensing can not only achieve high-precision reconstruction of data, but also reduce the computational complexity and data storage requirements. Therefore, how to achieve efficient compression and accurate reconstruction of Internet of Things data by using deep compressive sensing is a key issue. Summary of the Invention

[0004] To solve the above problems, the present invention provides an Internet of Things data compression and reconstruction method based on deep compressive sensing, which is applied to an Internet of Things system. The Internet of Things system includes a plurality of sensors. The method includes:

[0005] S1. Taking the acquisition data of a plurality of sensors as original data X , and inputting it into a compression module for compressive sensing to obtain compressed data;

[0006] S2. Inputting the compressed data into a reconstruction module for decoding and reconstruction;

[0007] S2.1. Mapping the compressed data to the original space to obtain initialization data, and inputting it into the residual decoder of the reconstruction module to obtain decoded data;

[0008] S2.2. Mapping the decoded data of the residual decoder to a high-dimensional embedding space, and performing data reconstruction through a data reconstruction decoder to obtain output data;

[0009] S2.3. Mapping the output data of the data reconstruction decoder to the original space through a fully connected layer to obtain reconstructed data, and the reconstructed data is the original data.

[0010] The specific operation of inputting into the compression module for compressive sensing to obtain compressed data in S1 is:

[0011] S1.1. Input the original data X into the fully connected layer of the residual perception block in the compression module for redundancy processing to obtain Data 1 x 1 ;

[0012] S1.2. Normalize Data 1 x 1 through layer normalization and obtain Data 2 through the ReLU activation function x 2 ;

[0013] S1.3. Perform a linear transformation on Data 2 x 2 through the fully connected layer to obtain Data 3 x 3 , and perform a residual connection between Data 3 x 3 and the original data X to obtain Data 4 x 4 ;

[0014] S1.4. The residual perception block adaptively generates a learnable measurement matrix, and trains and optimizes the learnable measurement matrix based on Data 4 x 4 to obtain the optimal measurement matrix A ;

[0015] S1.5. Based on the optimal measurement matrix A , compress the original data through compressive sensing to obtain the compressed data y , y = AX .

[0016] In the specific implementation manner, the residual decoder includes three identical residual blocks.

[0017] The initialization data in S2 is input into the residual decoder of the reconstruction module, and the decoded data is as follows:

[0018] ,

[0019] wherein, is the decoded data of the residual decoder; represents the residual block, represents the output data of the

[0020] The calculation and processing operations of each residual block on the input data include:

[0021] a. Batch-normalize the input data and then obtain the output data 1 through the ReLU activation function ;

[0022] ;

[0023] b. Perform the first batch normalization and ReLU activation function on the output data one Repeat step a for the second time to obtain output data two through batch normalization and ReLU activation function ;

[0024] ;

[0025] c. Perform a residual connection between the output data two and the input data to obtain the output data of this residual block;

[0026] ,

[0027] wherein, is the output data of the first residual block; represents the initialization data.

[0028] The data reconstruction decoder includes three identical window data reconstruction blocks.

[0029] The output data of the data reconstruction decoder in S2 is:

[0030] ,

[0031] wherein, is the output data of the data reconstruction decoder, represents the window data reconstruction block, represents the output data of the

[0032] The computational processing of the input data by each window data reconstruction block includes:

[0033] A. Divide the high-dimensional embedding space into several local windows according to the window size w×w, and calculate the multi-head self-attention within each local window;

[0034] B. Calculate the attention weights of each single-head self-attention to obtain the output data of the single-head self-attention;

[0035] C. Combine the output data of all single-head self-attention within the local window and perform a linear projection to obtain the combined data of this local window;

[0036] D. First perform layer normalization on the combined data, and then perform a residual connection with the original combined data to obtain the first output data;

[0037] E. Obtain the second output data by passing the first output data through a multi-layer perceptron layer; perform layer normalization on the first output data, and then perform a residual connection with the second output data to obtain the output data of the window data reconstruction block.

[0038] The present invention also provides an Internet of Things data compression and reconstruction system based on deep compressive sensing, including:

[0039] A data compression module for taking the acquisition data of several sensors as the original data X , and inputting it into the compression module for compressive sensing to obtain compressed data;

[0040] A decoding and reconstruction module for inputting the compressed data into the reconstruction module for decoding and reconstruction; mapping the compressed data to the original space to obtain initialization data, and inputting it into the residual decoder of the reconstruction module to obtain decoded data;

[0041] Map the decoded data of the residual decoder to a high-dimensional embedding space, and perform data reconstruction through a data reconstruction decoder to obtain output data;

[0042] Map the output data of the data reconstruction decoder to the original space through a fully connected layer to obtain reconstructed data, and the reconstructed data is the original data.

[0043] Beneficial effects: The present invention is an Internet of Things data compression and reconstruction method and system based on deep compressive sensing, including a compression module and a reconstruction module. The compression module adaptively generates a learnable measurement matrix using a residual sensing block, reducing the data transmission volume in the Internet of Things and improving the data reconstruction accuracy. The reconstruction module combines a residual decoder and a data reconstruction decoder to complete the reconstruction of Internet of Things data by extracting key features from the compressed data. The present invention designs an end-to-end model, where compression and reconstruction are jointly optimized and all parameters are updated together, which can improve the compression efficiency and reconstruction quality. Detailed implementation manners

[0044] Embodiment

[0045] This embodiment provides an Internet of Things data compression and reconstruction method based on deep compressive sensing, which is applied to an Internet of Things system, and the Internet of Things system includes several sensors. By constructing an end-to-end deep compressive sensing model, the deep compressive sensing module includes a compression module and a reconstruction module.

[0046] Based on the compression module and the reconstruction module respectively compressing and reconstructing the data of each sensor in the Internet of Things system, the specific implementation steps of the Internet of Things data compression and reconstruction method are as follows:

[0047] S1. Take the acquisition data of several sensors as the original data X, input it into the compression module for compressive sensing to obtain compressed data;

[0048] In the Internet of Things, each sensor collects Internet of Things data at fixed time intervals. The data collected by N sensor devices n times is used as the original data, which is expressed as ;

[0049] Input the original data into the compression module for compressive sensing to obtain compressed data. The compression module includes a residual sensing block, and its specific operation is as follows:

[0050] S1.1. Input the original data X into the fully connected layer of the residual sensing block in the compression module for redundancy processing to obtain data one x 1 ;

[0051] Internet of Things data is usually high-dimensional, and there may be redundancy in the data of different sensors. Reducing redundancy through the fully connected layer enables the Internet of Things data to adapt to the input of the network. The redundancy processing is:

[0052] ,

[0053] where x 1 represents data one, is the dimensionality transformation matrix.

[0054] S1.2. Normalize data one x 1 and obtain data two through the ReLU activation function x 2 ;

[0055] The data two is:

[0056] ,

[0057] where is the weight matrix, is the bias term; LayerNorm is layer normalization.

[0058] S1.3. Perform a linear transformation on data two x 2 through the fully connected layer to obtain data three x 3 , and perform a residual connection between data three x 3 and the original data X to obtain data four x 4 ;

[0059] , ,

[0060] Among them, and respectively represent weight matrices.

[0061] S1.4. The residual-aware block adaptively generates a learnable measurement matrix, and based on data four x 4 the learnable measurement matrix is trained and optimized to obtain an optimal measurement matrix A ;

[0062] The characteristics of Internet of Things data vary greatly over time, and a fixed measurement matrix is difficult to adapt to the dynamic characteristics of Internet of Things data. Therefore, a learnable measurement matrix is adaptively generated through a residual-aware block, and the learnable measurement matrix is trained and optimized, so that the residual-aware block retains more effective information about the original data during compressive sensing, improves the performance of compressive sensing, and enhances the reconstruction quality of the compressed data in the later stage.

[0063] Based on data four x 4 the learnable measurement matrix is trained and optimized to obtain the optimal measurement matrix as:

[0064] ,

[0065] Among them, represents the optimal measurement matrix, , is the compression dimension, is the residual-aware block, is the optimization parameter.

[0066] S1.5. Based on the optimal measurement matrix A the original data is compressed and sensed to obtain compressed data y , y = AX .

[0067] S2. The compressed data is input into a reconstruction module for decoding and reconstruction, and the reconstruction module includes a residual decoder and a data reconstruction decoder;

[0068] S2.1. The compressed data is mapped to the original space to obtain initialization data, and the initialization data is input into the residual decoder of the reconstruction module to obtain decoded data;

[0069] The compressed data y is mapped to the original space, and the original space is the dimension of the original Internet of Things data, and the generated initialization data is:

[0070] ,

[0071] Among them, is the dimensional transformation matrix, is the bias term, is batch normalization, is the activation function.

[0072] In the network structure, if only relying on fully connected layers or convolutional layers for transformation, important feature information of the data may be lost. Stacking multiple residual blocks can enhance the non-linear expression ability of the model, enabling it to better adapt to complex data patterns and improve the reconstruction accuracy.

[0073] Input the initialization data into the residual decoder for decoding. The residual decoder includes three identical residual blocks. The decoded data Z obtained by the residual decoder for decoding the input initialization data is:

[0074] ,

[0075] Among them, is the decoded data of the residual decoder; represents the residual block, represents the output data of the

[0076] The calculation and processing operations of each residual block on the input data include:

[0077] a. Perform batch normalization on the input data, and then obtain the output data one through the ReLU activation function ;

[0078] ;

[0079] b. Repeat the operation of a for the output data one to perform the second batch normalization and ReLU activation function to obtain the output data two , further extracting the data features;

[0080] ;

[0081] c. Perform a residual connection on the output data two and the input data to obtain the output data of this residual block;

[0082] ,

[0083] Among them, is the output data of the first residual block.

[0084] The first residual block uses the initialization data As input data, perform steps a to c to obtain the output data Z1 of the first residual block; use the output data Z1 of the first residual block as the input data of the second residual block, and perform steps a to c to obtain the output data Z2 of the second residual block; use the output data Z2 of the second residual block as the input data of the third residual block, and perform steps a to c to obtain the output data Z3 of the third residual block.

[0085] S2.2. Map the decoded data of the residual decoder to a high-dimensional embedding space, and perform data reconstruction through the data reconstruction decoder to obtain the output data.

[0086] Map the decoded output data of the residual decoder to a high-dimensional embedding space to obtain:

[0087] ,

[0088] where, is the embedding matrix, is the data in the high-dimensional embedding space, , is the sequence length, is the embedding dimension.

[0089] The data reconstruction decoder includes three identical window data reconstruction blocks, and each window data reconstruction block includes window partitioning, window self-attention mechanism, window merging, multi-layer perceptron layer, and residual connection.

[0090] Input the data in the high-dimensional embedding space into the data reconstruction decoder for data reconstruction, and the output data of the data reconstruction decoder is:

[0091] ,

[0092] where, is the output data of the data reconstruction decoder, represents the window data reconstruction block, represents the th output data of the window data reconstruction block;

[0093] The calculation and processing operation of each window data reconstruction block on the input data is:

[0094] A. Divide the high-dimensional embedding space into several local windows according to the window size w×w, and calculate the multi-head self-attention within each local window;

[0095] For the data within each local window, calculate the query Q of each single-head self-attention, K key V and value Q n keyK h With values V h Are respectively:

[0096] ,

[0097] Among them, Represents the features that the data in the current window focuses on when calculating the attention weights, Represents the feature representation of the data in the current window, Used to adjust the output data according to the attention weights; Is the weight matrix of the th single-head self-attention, Represents the key Dimension of.

[0098] B. Calculate the attention weights of each single-head self-attention to obtain the output data of the single-head self-attention;

[0099] ,

[0100] Among them, Is the normalization function, Attention h Represents the attention weight of the

[0101] th single-head self-attention. C. Combine and linearly project the output data of all single-head self-attentions within the local window to obtain the combined data

[0102] of this local window, and improve the reconstruction and recovery ability of data details by combining the features of different attention heads;

[0103] Among them, Is the linear projection matrix, The function is to concatenate the outputs of the multi-head attention.

[0104] D. First perform layer normalization on the combined data , and then perform a residual connection with the original combined data to obtain the first output data ;

[0105] ;

[0106] E. Pass the first output data through the multi-layer perceptron layer to obtain the second output data ; Perform layer normalization on the first output data , and then perform a residual connection with the second output data to obtain the output data of the data reconstruction block of this window. The specific formula is:

[0107] ,

[0108] Among them, and are weight matrices, and are bias terms, is an activation function;

[0109] .

[0110] The first window data reconstruction block uses the high-dimensional embedded space data as input data, and performs steps A to E to obtain the output data of the first window data reconstruction block ; uses the output data of the first window data reconstruction block as the input data of the second window data reconstruction block, and performs steps A to E to obtain the output data of the second window data reconstruction block ; uses the output data of the second window data reconstruction block as the input data of the third window data reconstruction block, and performs steps A to E to obtain the output data of the window data reconstruction block ;

[0111] S2.3. Map the output data of the data reconstruction decoder to the original space through a fully connected layer to obtain the reconstructed data, and the reconstructed data is the original data;

[0112] ,

[0113] Among them, is a dimensional transformation matrix.

[0114] In addition, an Internet of Things data compression and reconstruction system based on deep compressive sensing is also provided, including:

[0115] A data compression module for using the acquisition data of several sensors as the original data X , and inputting it into the compression module for compressive sensing to obtain compressed data;

[0116] A decoding and reconstruction module for inputting the compressed data into the reconstruction module for decoding and reconstruction; mapping the compressed data to the original space to obtain initialization data, and inputting it into the residual decoder of the reconstruction module to obtain decoded data;

[0117] Mapping the decoded data of the residual decoder to the high-dimensional embedded space, and performing data reconstruction through the data reconstruction decoder to obtain output data;

[0118] Map the output data of the data reconstruction decoder to the original space through a fully connected layer to obtain the reconstructed data, and the reconstructed data is the original data.

Claims

1. A method for compressing and reconstructing Internet of Things data based on deep compressive sensing, which is applied to an Internet of Things system. The Internet of Things system includes several sensors, and is characterized in that, Including: S1. Use the acquisition data of several sensors as the original data X , and input it into the compression module for compressive sensing to obtain compressed data; In the Internet of Things, each sensor collects Internet of Things data at fixed time intervals. By taking the data collected by N sensor devices n times as the original data, it is expressed as ; S2. Input the compressed data into the reconstruction module for decoding and reconstruction; S2.1 Map the compressed data to the original space to obtain initialization data, and input it into the residual decoder of the reconstruction module to obtain decoded data; The residual decoder includes three identical residual blocks. The computational processing operations of each residual block on the input data include: a. Batch-normalize the input data and then obtain the first output data through the ReLU activation function ; ； b, output data one Repeat operation a for the second batch normalization and ReLU activation function to obtain output data two ; ； c, output data two Perform a residual connection with the input data to obtain the output data of this residual block; ， Among them, is the output data of the first residual block; represents the initialization data; The first residual block takes the initialized data as the input data, and performs steps a~c to obtain the output data Z1 of the first residual block; takes the output data Z1 of the first residual block as the input data of the second residual block, and performs steps a~c to obtain the output data Z2 of the second residual block; takes the output data Z2 of the second residual block as the input data of the third residual block, and performs steps a~c to obtain the output data Z3 of the third residual block; S2.2 Map the decoded data of the residual decoder to a high-dimensional embedding space, and perform data reconstruction through a data reconstruction decoder to obtain output data; the data reconstruction decoder includes three identical window data reconstruction blocks, and each window data reconstruction block includes window partitioning, window self-attention mechanism, window merging, multi-layer perceptron layer, and residual connection; The first window data reconstruction block takes the high-dimensional embedded space data as input data to obtain the output data of the first window data reconstruction block ; The output data of the first window data reconstruction block is used as the input data of the second window data reconstruction block to obtain the output data of the second window data reconstruction block ; The output data of the second window data reconstruction block is used as the input data of the third window data reconstruction block to obtain the output data of the window data reconstruction block ; S2.3 Map the output data of the data reconstruction decoder to the original space through a fully connected layer to obtain reconstructed data, and the reconstructed data is the original data.

2. The method for compressing and reconstructing Internet of Things data based on deep compressive sensing according to claim 1, wherein In S1, the specific operation of inputting into the compression module for compressive sensing to obtain compressed data is: S1.

1. Input the original data X into the fully connected layer of the residual perception block in the compression module for redundancy processing to obtain Data 1 x 1 ; S1.

2. Normalize Data 1 x 1 Perform layer normalization and obtain Data 2 through the ReLU activation function x 2 ; S1.

3. Linearly transform Data 2 x 2 through a fully connected layer to obtain Data 3 x 3 , and then x 3 perform a residual connection between Data 3 X and the original data to obtain Data 4 x 4 ; S1.

4. Adaptively generate a learnable measurement matrix based on the residual perception block, and based on the data four x 4 Train and optimize the learnable measurement matrix to obtain the optimal measurement matrix A ; S1.

5. Based on the optimal measurement matrix A , compress the original data through compressive sensing to obtain the compressed data y , y = AX .

3. The method for compressing and reconstructing Internet of Things data based on deep compressive sensing according to claim 1, characterized in that, The computational processing of each window data reconstruction block on the input data includes: A. Divide the high-dimensional embedding space into several local windows according to the window size w×w, and calculate multi-head self-attention within each local window; B. Calculate the attention weights of each single-head self-attention to obtain the output data of the single-head self-attention; C. Merge the output data of all single-head self-attentions within the local window and perform linear projection to obtain the merged data of this local window; D. First perform layer normalization on the merged data, and then perform residual connection with the original merged data to obtain the first output data; E. Pass the first output data through a multi-layer perceptron layer to obtain the second output data; perform layer normalization on the first output data, and then perform residual connection with the second output data to obtain the output data of this window data reconstruction block.

4. An Internet of Things data compression and reconstruction system based on deep compressive sensing, which is implemented by using the Internet of Things data compression and reconstruction method based on deep compressive sensing according to any one of claims 1-3, and is characterized in that, Including: A data compression module, which is used to take the acquisition data of several sensors as the original data X , and input it into the compression module for compressive sensing to obtain compressed data; A decoding and reconstruction module, which is used to input the compressed data into the reconstruction module for decoding and reconstruction; map the compressed data to the original space to obtain initialization data, and input it into the residual decoder of the reconstruction module to obtain decoded data; Map the decoded data of the residual decoder to a high-dimensional embedding space, and perform data reconstruction through a data reconstruction decoder to obtain output data; Map the output data of the data reconstruction decoder to the original space through a fully connected layer to obtain reconstructed data, and the reconstructed data is the original data.

Images