Adaptive compressive sensing imaging method based on error estimation

Abstract

The application relates to an adaptive compressive sensing imaging method based on error estimation. The specific steps of the method are as follows: first stage: (1) dividing an image I into blocks, performing compressive measurement on the image blocks by using a measurement matrix phi1 to obtain y1; (2) performing initial reconstruction on y1 to obtain (3) performing optimized reconstruction on to obtain second stage: (4) inputting the reconstruction result of the last stage, phi1 and y1 into an adaptive module ASM to output an adaptive mask A1; (5) performing compressive measurement on part of the image blocks by using phi2 according to A1 to obtain y2; (6) performing initial reconstruction on y2 and adding the initial reconstruction result of the last stage to obtain the initial reconstruction result of the current stage; (7) performing optimized reconstruction on the initial reconstruction result of the current stage to obtain the optimized reconstruction result of the current stage; and (8) repeating steps (4)-(7) until a target sampling rate is reached.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and particularly relates to the field of adaptive compressed sensing imaging. Background Technology

[0002] Compressive sensing is a novel sampling theory proposed in the 21st century. It can sample the original signal at a much lower sampling rate than the Nyquist rate, while accurately reconstructing the original signal, under the condition that the original signal satisfies sparsity. This effectively reduces the amount of data at the acquisition end and shifts the computational load to the back end. Its mathematical model is: y = φx, where x ∈ R. N Let be the original signal in N dimensions, φ∈R M×N Let y be an M×N dimensional measurement matrix, y∈R M Let M be the measurement result in M ​​dimensions, where M << N.

[0003] Typically, independent and identically distributed Gaussian random measurement matrices can serve as universal compressed sensing measurement matrices. However, if the original signal x has a high dimension, the measurement matrix becomes excessively large, leading to storage pressure. Therefore, in practice, input images are often segmented to reduce storage pressure, but this introduces noticeable blockiness. Since convolution can be expanded into traditional multiplication, with the development of deep learning in recent years, some researchers have attempted to replace traditional measurement matrices with linear fully convolutional neural networks. The advantage of using fully convolutional networks as measurement matrices is that they can process inputs of arbitrary scales without causing blockiness in the reconstruction results.

[0004] Adaptive compressed sensing imaging is one way to improve sampling efficiency, but its challenge lies in designing an adaptive sampling strategy when the original image is unavailable. Existing adaptive compressed sensing imaging methods divide the sampling and reconstruction process into multiple steps, with each sampling adaptively adjusted based on the existing reconstruction results. For example, it calculates the wavelet transform of the current low-resolution reconstruction result, and then samples only the regions with larger wavelet transform coefficients in the next iteration. However, the wavelet transform coefficients of the low-resolution reconstruction result cannot accurately reflect the texture distribution of the corresponding regions in the real high-resolution image, easily causing the adaptive sampling strategy to fail. Therefore, how to efficiently allocate sampling data based on the already acquired information is a crucial issue that can effectively improve sampling efficiency and reconstruction quality. Summary of the Invention

[0005] To address the problems of existing adaptive compressed sensing imaging methods, the present invention aims to propose an adaptive compressed sensing imaging method based on error estimation.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] An adaptive compressed sensing imaging method based on error estimation includes the following steps:

[0008] Level 1:

[0009] Step 1: Divide the original image I into blocks and use the first-level measurement matrix φ1 to perform compressed measurement on each image block to obtain the first-level measurement result y1;

[0010] Step 2: Use a 1×1 convolutional layer to perform dimensionality transformation on the first-level measurement result y1, and output the initial reconstruction result of the first level through data rearrangement operation.

[0011] Step 3: Use a fully convolutional neural network N1 to reconstruct the initial results of the first stage. Further optimization and reconstruction were performed to obtain the final reconstruction result of the first level.

[0012] Level 2:

[0013] Step 4: Input the final reconstruction result of the previous level, the measurement matrix φ1 of the first level, and the measurement result y1 of the first level into the adaptive module ASM to obtain the adaptive mask A1;

[0014] Step 5: Perform a dot product between the adaptive mask A1 and the original image I to obtain the modulated input image I1;

[0015] Step 6: The input image I1 is divided into blocks, and based on the adaptive mask A1 obtained in step 4, the measurement matrix φ2 of this level is used to compress and measure some image blocks to obtain the measurement result y2 of this level.

[0016] Step 7: Use a 1×1 convolutional layer to perform dimensionality transformation on the measurement result y2 of this level, and perform data rearrangement operation. Compare the obtained result with the initial reconstruction result of Step 2. Add them together to obtain the initial reconstruction result for this level.

[0017] Step 8: Use a fully convolutional neural network N2 to reconstruct the initial results of this stage. Further optimization and reconstruction were carried out to obtain the final reconstruction result at this level.

[0018] Level 3 to Level k:

[0019] Step 9: Repeat steps 4-8 until the final reconstruction result at the target compression ratio is obtained.

[0020] Furthermore, the measurement matrices used are φ1, φ2, ..., φ kThe dimension is M k ×N, where N is the data size of each image patch, and M k The number of samples for each image patch, M k <N, where the sampling rate implemented by each measurement matrix is ​​[missing information].

[0021] Furthermore, the dimensionality transformation is to transform the input as... The measurement results, after passing through a 1×1 convolutional layer, output dimension is... The data rearrangement operation consists of two steps: dimension transformation and data concatenation. The dimension transformation operation involves transforming data with dimensions of... Each 1×1×N vector in the data is converted to p×p×C in Image blocks, data stitching operation is to... The image blocks are sequentially stitched together to form an H×W×C shape. in The image, i.e., the initial reconstruction result. Among them, H, W and C in denoted as the height, width, and number of channels of the original image I, respectively, and p is the size of the image block when dividing the image into blocks.

[0022] Furthermore, the adaptive module ASM includes the following calculation steps:

[0023] (1) Calculate measurement error

[0024] (2) Calculate the measurement error for each image block L2 norm

[0025]

[0026] (3) Calculate the number of image blocks that are not sampled during the second-level sampling. Where α is the sampling scale factor.

[0027] (4) Sort the norm V in ascending order, V sorted =sort ↑ (V),

[0028] (5) Calculate the binarization threshold T = V sorted (N a ),

[0029] (6) Binarize the norm V according to the threshold T. Values ​​greater than T are quantized to 1, and values ​​less than or equal to T are quantized to 0, thus obtaining the binary mask B. a ,

[0030] (7) For the binary mask B a The image is then expanded so that each value is the same size as the corresponding image patch, resulting in an adaptive mask A.i-1 , where 2≤i≤k represents a series.

[0031] Furthermore, in step 6, based on the mask output by the adaptive module ASM, compression measurement is performed only on a portion of the image blocks, and the compression measurement values ​​of the unsampled image blocks are filled with 0.

[0032] This invention introduces an adaptive sampling module (ASM) based on error estimation, which enables adaptive compressed sensing imaging that changes with the target scene. The beneficial effects of this invention are: (1) Since the entire model is a fully convolutional neural network, it can perform compressed reconstruction for inputs of any scale without causing blockiness; (2) Through multi-level structural design, it is possible to complete compressed sensing imaging at different sampling rates with a single network; (3) By using the adaptive sampling module (ASM) based on error estimation, adaptive compressed sampling can be achieved, thereby improving sampling efficiency and reconstruction quality. Attached Figure Description

[0033] Figure 1 This is a flowchart of the method of the present invention;

[0034] Figure 2 This is a schematic diagram of the specific structure of the adaptive sampling module in an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of the specific structure of the reconstructed network in an embodiment of the present invention. Detailed Implementation

[0036] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0037] Reference Figure 1 This embodiment presents an adaptive compressed sensing imaging method based on error estimation, which employs a hierarchical sampling and reconstruction structure. During the second and subsequent sampling stages, the sampling strategy is adaptively adjusted based on the error estimation of the current reconstruction result. Using a grayscale image as input, the original image is divided into 8×8 image blocks during sampling, specifically including the following steps:

[0038] Level 1:

[0039] Step 1, Compression Measurement: The first-level compression measurement uses a linear fully convolutional neural network ψ1, whose input is an H×W×1 grayscale image I, where H and W are the height and width of the image, and the output is... The compressed measurement result y1 contains only one convolutional layer with an 8×8 kernel size, a stride of 8, no bias units, and a corresponding sampling rate of ψ1 can be equivalent to dividing the image into multiple non-overlapping 8×8 image blocks, sampling each image block separately, with M1 samples per image block, and the measurement matrix can be learned end-to-end during training.

[0040] Step 2, Initial Reconstruction: The input for the first-level initial reconstruction process is... The compressed measurement result y1 is output as an initial reconstruction result of H×W×1. It first performs a 1×1 convolutional layer to transform the dimensions of the measurement result y1, resulting in... The output is then transformed into a 1×1×64 vector into a 1×1×8×8 image patch through a dimension transformation operation. By stitching together the image patches in sequence, an initial reconstruction result of H×W×1 can be obtained.

[0041] Step 3, Optimization Reconstruction: The input for the first-level optimization reconstruction process is the initial reconstruction result of H×W×1. The output is the reconstruction result of H×W×1. The optimized reconstruction process employs a fully convolutional neural network, such as... Figure 3 As shown, the specific structure of N1 is as follows:

[0042] The first layer is a convolutional layer with 1 input channel and 128 output channels. The second layer is a convolutional layer with 128 input channels and 32 output channels. The third to fifth layers are three dense residual blocks with the same structure, each with 32 input channels and 32 output channels. The sixth layer is a convolutional layer with 32 input channels and 128 output channels. The seventh layer is a convolutional layer with 32 input channels and 1 output channel. There is also a skip connection between the input and output layers. The kernel size of each convolutional layer is 3×3, and each convolutional layer is followed by a LeakyReLU non-linear activation function.

[0043] Level 2:

[0044] Step 4, Generate Adaptive Mask: Use the Adaptive Module ASM to generate an adaptive mask A1, which can be divided into the following steps: (1) Calculate the measurement error (2) Calculate the measurement error for each image block L2 norm (3) Calculate the number of image blocks that are not sampled during the second-level sampling. Where α is the sampling scale factor, (4) sort V from smallest to largest, V sorted =sort ↑ (V), (5) Calculate the binarization threshold T = V sorted (N a(6) Binarize V according to the threshold. Values ​​greater than T are quantized as 1, and values ​​less than or equal to T are quantized as 0, to obtain the binary B. a (7) For B a The image is expanded so that each value is the same size as the corresponding image block, resulting in an adaptive mask A1.

[0045] Step 5, Modulation: Perform a dot product between the adaptive mask A1 obtained in step 4 and the input image I to obtain the modulated input image I1.

[0046] Step 6, Second-stage compression measurement: The second-stage compression measurement uses a linear fully convolutional neural network ψ2, whose input is an H×W×1 image I1, and whose output is... The compressed measurement result y2 contains only one convolutional layer with an 8×8 kernel size, a stride of 8, no bias units, and a corresponding sampling rate of ψ2 can be equivalent to dividing the image into multiple non-overlapping 8×8 image blocks, sampling a subset of these blocks, with each block having M2 samples, and filling unsampled blocks with 0s. The measurement matrix can be learned end-to-end during training.

[0047] Step 7, Initial Reconstruction: The input for the second-level initial reconstruction process is... The compressed measurement result y2 is output as the initial reconstruction result of H×W×1. It first performs a 1×1 convolutional layer to transform the dimensions of the measurement result y2, resulting in... The output is then transformed into a 1×1×64 vector, which is converted into a 1×1×8×8 image patch. By stitching together image blocks in sequence, a residual map of size H×W×1 can be obtained. This residual map is then compared with... By adding them together, we can obtain the initial reconstruction result of H×W×1.

[0048] Step 8, Optimize Reconstruction: The input for the optimized reconstruction process is the initial reconstruction result of H×W×1. The output is the reconstruction result of H×W×1. The optimized reconstruction process employs a fully convolutional neural network N2, such as... Figure 3 As shown, the specific structure of N2 is as follows:

[0049] The first layer is a convolutional layer with 1 input channel and 128 output channels. The second layer is a convolutional layer with 128 input channels and 32 output channels. The third layer is a dense residual block layer with 32 input channels and 32 output channels. The fourth layer is a Cat feature fusion layer, fusing the feature map output from the third layer of the first layer, resulting in a 64 output channel. The fifth layer is a convolutional layer with 64 input channels and 32 output channels. The sixth layer is a dense residual block layer with 32 input channels and 32 output channels. The seventh layer is a Cat feature fusion layer, fusing the feature map output from the fourth layer of the first layer. The first layer outputs a feature map with 64 channels. The eighth convolutional layer has 64 channels as input and 32 channels as output. The ninth layer is a dense residual block with 32 channels as input and 32 channels as output. The tenth layer is a cat feature fusion layer, fusing the feature map output from the fifth layer of the first layer, with 64 channels as output. The eleventh layer is a convolutional layer with 64 channels as input and 32 channels as output. The twelfth layer is a convolutional layer with 32 channels as input and 128 channels as output. The eleventh layer is a convolutional layer with 128 channels as input and 1 channel as output. There is also a skip connection between the input and output layers. Each convolutional layer has a 3×3 kernel size and is followed by a LeakyReLU non-linear activation function.

[0050] Step 9, as follows Figure 1 As shown, repeat steps 4 to 8 up to level K, using the reconstruction result from the previous level and the measurement matrix φ1 and measurement result y1 from the first level, until the reconstruction result at the target compression ratio is obtained. The sampling rate of level i is The total sampling rate for the first i levels is:

[0051] In this embodiment, the loss function used during training the i-th level is the sum of the L1 loss between the final reconstruction result of the i-th level and the original image, and the L1 loss between the initial reconstruction result of the i-th level and the original image. During training, after training the compressed sampling module and the reconstruction module in each iteration, the parameters of all previous iterations are fixed, and the parameters for the next iteration are trained, until the entire network is trained. In this embodiment, the image patch size used during training is 128×128, the optimizer is Adamoptimizer, and the learning rate is set to 2×10⁻⁶. -4 .

Claims

1. An adaptive compressive sensing imaging method based on error estimation, characterized in that, The method includes the following steps: Level 1: Step 1: Divide the original image I into blocks and use the first-level measurement matrix φ1 to perform compressed measurement on each image block to obtain the first-level measurement result y1; Step 2, using a 1x1 convolution layer to transform the dimension of the first level measurement result y1, and output the initial reconstruction result of the first level through data rearrangement operation Step 3: Use a fully convolutional neural network N1 to reconstruct the initial results of the first stage. Further optimization and reconstruction were performed to obtain the final reconstruction result of the first level. Level 2: Step 4: Input the final reconstruction result of the previous level, the measurement matrix φ1 of the first level, and the measurement result y1 of the first level into the adaptive module ASM to obtain the adaptive mask A1; Step 5: Perform a dot product between the adaptive mask A1 and the original image I to obtain the modulated input image I1; Step 6: The input image I1 is divided into blocks, and based on the adaptive mask A1 obtained in step 4, the measurement matrix φ2 of this level is used to compress and measure some image blocks to obtain the measurement result y2 of this level. Step 7, using a 1x1 convolution layer to transform the dimension of the measurement result y2 of this level, and performing data rearrangement operation, and obtaining the result and the initial reconstruction result of step 2 Adding to obtain the initial reconstruction result of this level Step 8, using a fully convolutional neural network N2 on the initial reconstruction result of the current level performing further optimization reconstruction to obtain the final reconstruction result of the current level Level 3 to Level k: Step 9, repeat steps 4-8 until the final reconstruction result at the target compression ratio is obtained 2. The method of claim 1, wherein, The measurement matrices used are φ1, φ2, ..., φ k The dimension is M k ×N, where N is the data size of each image patch, and M k The number of samples for each image patch, M k <N, where the sampling rate implemented by each measurement matrix is ​​[missing information].

3. The method of claim 1, wherein, The dimensionality transformation is to transform the input as... The measurement results, after passing through a 1×1 convolutional layer, output dimension is... The data rearrangement operation consists of two steps: dimension transformation and data concatenation. The dimension transformation operation involves transforming data with dimensions of... Each 1×1×N vector in the data is converted to p×p×C in Image blocks, data stitching operation is to... The image blocks are sequentially stitched together to form an H×W×C shape. in The image, i.e., the initial reconstruction result. Among them, H, W and C in denoted as the height, width, and number of channels of the original image I, respectively, and p is the size of the image block when dividing the image into blocks.

4. The method of claim 1, wherein, The adaptive module ASM includes the following calculation steps: (1) Calculate measurement error (2) calculating the L2 norm of the measurement error for each image patch of the L2 norm (3) the number of image blocks not sampled at the second level of sampling where a is a sampling ratio factor, (4) sort the norms V in increasing order, V sorted = sort ↑ (V), (5) Calculate the threshold value T = V for binarization sorted (N a ), (6) The norm V is binarized according to a threshold T, values greater than T are quantized as 1, values less than or equal to T are quantized as 0, to obtain a binary mask B a , (7) For the binary mask B a The image is then expanded so that each value is the same size as the corresponding image patch, resulting in an adaptive mask A. i-1 , where 2≤i≤k represents a series.

5. The method of claim 1, wherein, In step 6, based on the mask output by the adaptive module ASM, compression measurement is performed only on a portion of the image blocks, and the compression measurement values ​​of the unsampled image blocks are filled with 0.

Images