Image compressed sensing method based on multistage sampling and self-attention network

Abstract

The invention discloses an image compressed sensing method based on multistage sampling and a self-attention network, and the method comprises the following steps: 1, inputting images, and employing a data set as a training set which comprises 40000 images; 2, preprocessing an input image through a sampling network, and carrying out preliminary feature extraction and down-sampling on an input signal; 3, designing a self-adaptive multi-stage sampling module, performing multi-scale feature extraction on the low-sampling-rate signal by combining fuzzy pooling and maximum pooling, and performing weighted fusion on features of different scales through an attention mechanism; and step 4, initializing a reconstruction subnet, generating a preliminary feature map through convolution and pixel rearrangement operation, and performing up-sampling on low-dimensional representation. According to the invention, detail information of the image can be effectively maintained at a low sampling rate. The fuzzy pooling is smooth pooling through a Gaussian filter, and low-frequency information is effectively reserved, so that the loss of low-frequency components in the compressed sensing process is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of compressed sensing, and in particular to an image compressed sensing method based on multi-level sampling and self-attention network. Background Art

[0002] Compressed sensing technology achieves signal recovery by acquiring signals at low sampling rates, thereby reducing sampling and storage costs. However, signal recovery still faces many challenges under low sampling rate conditions, especially how to effectively recover image details and structural information from low-dimensional sampled signals. Existing compressed sensing methods mostly rely on shallow networks and fail to fully capture image details, resulting in image blur and detail loss at low sampling rates.

[0003] In recent years, deep learning has been introduced into compressed sensing image reconstruction and has significantly improved the quality of reconstruction. Deep neural networks can automatically learn image features and achieve relatively accurate restoration at low sampling rates. However, most deep learning methods still use a single sampling strategy and a unified feature processing method, which lacks flexibility and fails to dynamically adjust the processing method according to the complexity of the image area. Taking convolutional neural networks as an example, each pixel is usually processed uniformly without considering the difference between local and global information.

[0004] In addition, low-sampled images often contain strong noise. Although denoising modules have been applied, the noise problem still affects the reconstruction effect, especially in low signal-to-noise ratio environments. In recent years, methods using multi-scale sampling and attention mechanisms have gradually gained attention. Multi-scale methods capture detailed information of images at different levels by processing features of different scales simultaneously, thereby improving image blur and distortion problems. The attention mechanism dynamically allocates weights, allowing the network to focus more on key areas in the image, thereby improving the reconstruction effect. These innovative methods have effectively improved some problems in traditional methods, but there are still the following shortcomings:

[0005] 1. Limitations of sampling strategies: Many methods still rely on uniform or shallow sampling strategies and fail to fully utilize important regional information in the image, especially under low sampling rate conditions.

[0006] 2. Noise problem: Although some methods introduce denoising modules, the image restoration effect is still limited in high-noise environments, especially in low signal-to-noise ratio situations.

[0007] 3. Insufficient information fusion: Existing multi-scale information fusion methods fail to effectively handle the interactions between scales, resulting in unsatisfactory image detail restoration effects.

[0008] 4. Inflexible feature processing: Some deep learning methods process features of all regions too uniformly and fail to optimize based on the differences between different regions and channels. Summary of the invention

[0009] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0010] In view of the above-mentioned problems existing in the existing image compression sensing method based on multi-level sampling and self-attention network, the present invention is proposed.

[0011] Therefore, the purpose of the present invention is to provide an image compression sensing method based on multi-level sampling and self-attention network, which is used to solve the problems of sampling strategy limitations, noise interference, insufficient information fusion and inflexible feature processing in existing methods, so as to achieve higher quality image restoration in low sampling rate and high noise environment.

[0012] In order to solve the above technical problems, the present invention provides the following technical solutions: an image compression sensing method based on multi-level sampling and self-attention network, comprising the following steps:

[0013] Step 1: Input the image and use the dataset as the training set, which contains 40,000 images;

[0014] Step 2: Preprocess the input image through the sampling network, perform preliminary feature extraction and downsampling on the input signal;

[0015] Step 3: Design an adaptive multi-level sampling module to extract multi-scale features from low sampling rate signals by combining fuzzy pooling and maximum pooling, and weightedly fuse features of different scales through the attention mechanism;

[0016] Step 4: Initialize the reconstruction subnet, generate a preliminary feature map through convolution and pixel rearrangement operations, and upsample the low-dimensional representation to restore the high-resolution initial reconstruction result;

[0017] Step 5: Deep reconstruction network, including multiple multi-scale feature enhancement modules, uses the window self-attention mechanism to fine-tune the features, capture local and global information, and optimize the image reconstruction effect.

[0018] As a preferred solution of the image compression sensing method based on multi-level sampling and self-attention network described in the present invention, wherein: the sampling network in step 2 is S AMSM (·) The feature extraction capability of low sampling rate signals is enhanced by combining fuzzy pooling and maximum pooling through an adaptive multi-level sampling module. A custom neural network layer is used to further fuse multi-scale features and enhance the image restoration effect. The sampling process is expressed as:

[0019] y=Φx=S AMSM (x)

[0020] Where y is the low-resolution measurement, Φ is the sampling matrix, x is the input signal, and S AMSM (·) is the sampling network; in the case of 1.25% sampling rate,

[0021] As a preferred solution of the image compression sensing method based on multi-level sampling and self-attention network described in the present invention, the adaptive multi-level sampling module includes: two convolution layers for feature extraction; a multi-scale pooling and fuzzy pooling module, wherein fuzzy pooling uses Gaussian filtering to smooth the feature map, and maximum pooling is used to extract local features, and weighted fusion is performed through the attention mechanism to obtain a weighted feature representation.

[0022] As a preferred solution of the image compression sensing method based on multi-level sampling and self-attention network of the present invention, the weighted fusion formula of the output F1 of the fuzzy pooling module and the output F2 of the maximum pooling module is:

[0023] y=o1·F1+o2·F2

[0024] Among them, o1 and o2 are two learnable weight parameters.

[0025] As a preferred solution of the image compression sensing method based on multi-level sampling and self-attention network described in the present invention, wherein: the input feature map is subjected to feature extraction through a module including three convolutional layers and a ReLU activation function; then, the input feature map is fuzzy pooled through BlurPool(·) to obtain a smoothed feature map; then, the result of the fuzzy pooling is added to the features extracted by the convolution to obtain an intermediate feature map; finally, the feature map is weighted through an attention mechanism and output F1;

[0026] The output process of F2 is similar to that of F1, but the operation performed on the input feature map is the maximum pooling MaxPool(·);

[0027] Among them, the specific definitions of F1 and F2 are as follows:

[0028] F1=(Conv(x)+BlurPool(x))×A blur

[0029] F2=(Conv(x)+MaxPool(x))×A max

[0030] Among them, A blur and A maxThey are the fuzzy attention and maximum attention modules, which consist of convolution and non-linear activation functions.

[0031] As a preferred solution of the image compression sensing method based on multi-level sampling and self-attention network of the present invention, the operation of initializing the reconstruction subnet is completed by the following formula:

[0032] I(x)=PS(Conv(S AMSM (·)))

[0033] Among them, PS is the pixel rearrangement operation, Conv is the convolution operation, S AMSM (·) is the output of the adaptive multi-stage sampling module.

[0034] As a preferred solution of the image compression sensing method based on multi-level sampling and self-attention network described in the present invention, each multi-scale feature enhancement module in the deep reconstruction network operates through the following process: the input feature map is downsampled through two convolution operations to extract feature information of different scales; the downsampled features are spliced ​​with the multi-scale feature map, and the resolution is gradually restored after convolution fusion to enhance the detail information of the image.

[0035] As a preferred solution of the image compression sensing method based on multi-level sampling and self-attention network described in the present invention, the data flow process of the multi-scale feature enhancement module is as follows:

[0036] S1, input data y generates three feature representations y0, y1, and y2 of different scales through the fusion module, where

[0037] S2, through two convolution downsampling operations, the input features are gradually reduced in resolution to extract feature information of different scales;

[0038] S3, by concatenating these downsampled features with the features y1 and y2 obtained from the multi-scale feature extraction module, and then gradually upsampling after convolution fusion, the resolution is gradually restored;

[0039] S4, through multiple splicing and convolution operations, further fused high-resolution features, combined with information of different scales, and finally generated rich feature representation with multi-level details. This process is expressed as follows:

[0040] f ↓ =Conv1(f t ), f ↓↓ =Conv2(f ↓ )

[0041] f1=Conv3(Concat(f ↓↓,y2)),f ↑ =Conv4(Convat(f1,f ↓↓ ))

[0042] f2=Conv5(Concat(f ↑ ,y1)),f ↑↑ =Conv6(Convat(f2,f ↓ ))

[0043] Among them, is the concatenation operation on the channel dimension of concat(·),

[0044] S5. After the window self-attention module, multi-scale window division is performed and self-attention is calculated to capture richer local and global information. Multi-scale window division and self-attention weight calculation are performed to fine-tune the weighted results of the input features. The formula is:

[0045] WSSA(·)=Concat(Attn1,Attn2,...,Attn n )+f ↑↑

[0046] Among them, Attn i Represents the attention of the sub-feature map of each window;

[0047] S6. Then calculate the characteristic error Δ and initialize Δ to obtain Δ Init The measured values ​​are used to guide the reconstruction. The compensated features, measured features and original input are fused to generate the final reconstructed image f out :

[0048] Δ Init =PS(S csmn (conv(f3))-y)

[0049] f4=f3+Res(Δ Init )

[0050] f out =Concat(x,y0,f4)

[0051] Among them, PS(·) is the pixel rearrangement operation, and Res(·) represents the residual connection.

[0052] As a preferred solution of the image compression sensing method based on multi-level sampling and self-attention network described in the present invention, the specific process of the multi-scale feature enhancement module is as follows:

[0053] First, input feature map f ↑↑ Perform feature projection through 1×1 convolution and batch normalization to generate feature maps with more channels;

[0054] Next, these channels are divided into multiple sub-feature maps, each of which corresponds to a different window size. For each sub-feature map, after being divided into small windows, the query (Q), key (K) and value (V) are extracted respectively and the self-attention weight is calculated, and the feature information is re-weighted using the attention mechanism;

[0055] Then, the weighted feature map is reconstructed back to its original shape, and multiple sub-feature maps are concatenated in the channel dimension;

[0056] Finally, the original number of channels is restored through a 1×1 convolution, and the result is connected to the input feature map f through a residual connection. ↑↑ Add together to form the final output.

[0057] Beneficial effects of the present invention:

[0058] 1. Enhance the ability to retain image details at low sampling rates

[0059] By introducing the adaptive multi-scale sampling module (AMSM), the present invention can effectively maintain the image detail information at a low sampling rate. Fuzzy pooling uses a Gaussian filter to smooth the pooling and effectively retains low-frequency information, thereby avoiding the loss of low-frequency components in the compressed sensing process. And maximum pooling combines multi-scale pooling and interpolation strategies to extract features at multiple scales, further enhancing the retention of detail information.

[0060] 2. Fine-tune the capture of feature information

[0061] The Windowed Self-Attention Mechanism (WSSA) is introduced in the sparse self-attention reconstruction module, using convolution windows of different sizes and adaptive attention to fine-tune the capture of feature information. This method enables the network to more flexibly weight different areas of the image, avoiding information loss or noise interference, especially in high-noise or low-sampling situations, and is particularly outstanding in restoring image details.

[0062] 3. Excellent reconstruction effect and generalization ability

[0063] Experimental results show that the present invention has better reconstruction effects than existing methods on multiple public data sets, especially in low sampling rate and noisy environments, and can provide higher quality image reconstruction. In addition, the present invention has strong generalization ability and can stably provide superior performance on different compressed sensing tasks and data sets, further proving its wide applicability in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0065] Figure 1 It is a schematic diagram of the overall process of the image compression sensing method based on multi-level sampling and self-attention network of the present invention.

[0066] Figure 2 It is a schematic diagram of a window-based sparse self-attention reconstruction module of the image compression sensing method based on multi-level sampling and self-attention network of the present invention.

[0067] Figure 3 Schematic diagram of the adaptive multi-scale sampling module of the image compression sensing method based on multi-level sampling and self-attention network of the present invention. DETAILED DESCRIPTION

[0068] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0069] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0070] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0071] Secondly, the present invention is described in detail with reference to the schematic diagram. When describing the embodiments of the present invention in detail, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0072] Reference Figure 1 - Figure 3 , provides an image compression sensing method based on multi-level sampling and self-attention network, including the following steps:

[0073] Step 1: Input the image and use the dataset as the training set, which contains 40,000 images;

[0074] Step 2: Preprocess the input image through the sampling network, perform preliminary feature extraction and downsampling on the input signal;

[0075] Step 3: Design an adaptive multi-level sampling module to extract multi-scale features from low sampling rate signals by combining fuzzy pooling and maximum pooling, and weightedly fuse features of different scales through the attention mechanism;

[0076] Step 4: Initialize the reconstruction subnet, generate a preliminary feature map through convolution and pixel rearrangement operations, and upsample the low-dimensional representation to restore the high-resolution initial reconstruction result;

[0077] Step 5: Deep reconstruction network, including multiple multi-scale feature enhancement modules, uses the window self-attention mechanism to fine-tune the features, capture local and global information, and optimize the image reconstruction effect.

[0078] Among them, the sampling network in step 2 is S AMSM (·) The feature extraction capability of low sampling rate signals is enhanced by combining fuzzy pooling and maximum pooling through an adaptive multi-level sampling module. A custom neural network layer is used to further fuse multi-scale features and enhance the image restoration effect. The sampling process is expressed as:

[0079] y=Φx=S AMSM (x)

[0080] Where y is the low-resolution measurement, Φ is the sampling matrix, x is the input signal, and S AMSM (·) is the sampling network; in the case of 1.25% sampling rate,

[0081] Furthermore, the adaptive multi-level sampling module includes: two convolutional layers for feature extraction; a multi-scale pooling and fuzzy pooling module, wherein fuzzy pooling uses Gaussian filtering to smooth the feature map, and maximum pooling is used to extract local features, and weighted fusion is performed through an attention mechanism to obtain a weighted feature representation.

[0082] Among them, the weighted fusion formula of the output F1 of the fuzzy pooling module and the output F2 of the maximum pooling module is:

[0083] y=o1·F1+o2·F2

[0084] Among them, o1 and o2 are two learnable weight parameters.

[0085] Specifically, the input feature map is extracted through a module containing three convolutional layers and a ReLU activation function. Then, the input feature map is fuzzy pooled through BlurPool(·) to obtain a smoothed feature map. Then, the result of the fuzzy pooling is added to the features extracted by the convolution to obtain an intermediate feature map. Finally, this feature map is weighted through an attention mechanism and output as F1.

[0086] The output process of F2 is similar to that of F1, but the operation performed on the input feature map is the maximum pooling MaxPool(·);

[0087] Among them, the specific definitions of F1 and F2 are as follows:

[0088] F1=(Conv(x)+BlurPool(x))×A blur

[0089] F2=(Conv(x)+MaxPool(x))×A max

[0090] Among them, A blur and A max They are the fuzzy attention and maximum attention modules, which consist of convolution and non-linear activation functions.

[0091] Furthermore, the operation of initializing and rebuilding the subnet is completed by the following formula:

[0092] I(x)=PS(Conv(S AMSM (·)))

[0093] Among them, PS is the pixel rearrangement operation, Conv is the convolution operation, S AMSM (·) is the output of the adaptive multi-stage sampling module.

[0094] Among them, each multi-scale feature enhancement module in the deep reconstruction network operates through the following process: the input feature map is downsampled through two convolution operations to extract feature information of different scales; the downsampled features are spliced ​​with the multi-scale feature map, and the resolution is gradually restored through convolution fusion to enhance the detail information of the image.

[0095] As a preferred solution of the image compression sensing method based on multi-level sampling and self-attention network described in the present invention, the data flow process of the multi-scale feature enhancement module is as follows:

[0096] S1, input data y generates three feature representations y0, y1, and y2 of different scales through the fusion module, where

[0097] S2, through two convolution downsampling operations, the input features are gradually reduced in resolution to extract feature information of different scales;

[0098] S3, by concatenating these downsampled features with the features y1 and y2 obtained from the multi-scale feature extraction module, and then gradually upsampling after convolution fusion, the resolution is gradually restored;

[0099] S4, through multiple splicing and convolution operations, further fused high-resolution features, combined with information of different scales, and finally generated rich feature representation with multi-level details. This process is expressed as follows:

[0100] f ↓ =Conv1(f t ), f ↓↓ =Conv2(f↓)

[0101] f1=Conv3(Concat(f ↓↓ ,y2)),f ↑ =Conv4(Convat(f1,f ↓↓ ))

[0102] f2=Conv5(Concat(f ↑ ,y1)),f ↑↑ =Conv6(Convat(f2,f ↓ ))

[0103] Among them, is the concatenation operation on the channel dimension of concat(·),

[0104] S5. After the window self-attention module, multi-scale window division is performed and self-attention is calculated to capture richer local and global information. Multi-scale window division and self-attention weight calculation are performed to fine-tune the weighted results of the input features. The formula is:

[0105] WSSA(·)=Concat(Attn1,Attn2,...,Attn n )+f ↑↑

[0106] Among them, Attn i Represents the attention of the sub-feature map of each window;

[0107] S6. Then calculate the characteristic error Δ and initialize Δ to obtain Δ Init The measured values ​​are used to guide the reconstruction. The compensated features, measured features and original input are fused to generate the final reconstructed image f out :

[0108] ΔInit =PS(S csmn (conv(f3))-y)

[0109] f4=f3+Res(Δ Init )

[0110] f out =Concat(x,y0,f4)

[0111] Among them, PS(·) is the pixel rearrangement operation, and Res(·) represents the residual connection.

[0112] Specifically, the specific process of the multi-scale feature enhancement module is as follows:

[0113] First, input feature map f ↑↑ Perform feature projection through 1×1 convolution and batch normalization to generate feature maps with more channels;

[0114] Next, these channels are divided into multiple sub-feature maps, each of which corresponds to a different window size. For each sub-feature map, after being divided into small windows, the query (Q), key (K) and value (V) are extracted respectively and the self-attention weight is calculated, and the feature information is re-weighted using the attention mechanism;

[0115] Then, the weighted feature map is reconstructed back to its original shape, and multiple sub-feature maps are concatenated in the channel dimension;

[0116] Finally, the original number of channels is restored through a 1×1 convolution, and the result is connected to the input feature map f through a residual connection. ↑↑ Add together to form the final output.

[0117] Example:

[0118] Input: In the embodiment of the present invention, the input part uses the CO CO 2014 dataset as the training set, which contains 40,000 images. In order to improve the computational efficiency and model robustness, the images are converted to the YC bC r color space, and only the Y channel is used for training and testing. The test datasets include Set5, Setl 1, BSD S68 and BSD S200, and the test images are center cropped and resized to 128×128.

[0119] Training details: The model was implemented using PyTorch and trained on an Nvidia RTX 3060G PU. The Adam optimizer was used to update the model parameters, with momentum and weight decay set to 0.9 and 0.999 respectively. The initial learning rate was set to 1e-4 and decayed at the 60th, 90th, and 120th epochs.

[0120] Loss function: The loss function uses the mean square error between the reconstructed image and the original image as the target, which can be

[0121] It can be expressed as:

[0122]

[0123]

[0124] Where L I represents the initial reconstruction error, L D represents the depth reconstruction error, n is the number of parameters, S AMSM (·), I(·), R(·), are the sampling, initial reconstruction, and deep reconstruction subnetworks mentioned above, x j is the reconstruction result of the jth reconstruction block. The model parameters are optimized during training by minimizing the loss function.

[0125] Training method: The model is trained for 150 epochs using a batch size of 6. The learning rate decay strategy is to multiply the learning rate by 0.25 at the 60th, 90th, and 120th epochs.

[0126] Test details: In the test phase, test images with different sampling rates are reconstructed, and the reconstruction quality is evaluated using PSNR and SSIM. At the same time, quantitative and intuitive comparisons are made with other advanced methods, including CSNet+, RK-CCSNet, NL-CSNet, MR-CCSNet, DPC-DUN, OCTUF, TransCS, etc.

[0127] Experimental platform: The experimental platform uses the PyTorch framework to implement the model and is trained on the Nvidia RTX 3060 GPU.

[0128] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An image compression sensing method based on multi-level sampling and self-attention network, characterized in that: The following steps are involved: Step 1: Input the image and use the dataset as the training set, which contains 40,000 images; Step 2: Preprocess the input image through the sampling network, perform preliminary feature extraction and downsampling on the input signal; Step 3: Design an adaptive multi-level sampling module to extract multi-scale features from low sampling rate signals by combining fuzzy pooling and maximum pooling, and weightedly fuse features of different scales through the attention mechanism; Step 4: Initialize the reconstruction subnet, generate a preliminary feature map through convolution and pixel rearrangement operations, and upsample the low-dimensional representation to restore the high-resolution initial reconstruction result; Step 5: Deep reconstruction network, including multiple multi-scale feature enhancement modules, uses the window self-attention mechanism to fine-tune the features, capture local and global information, and optimize the image reconstruction effect.

2. The image compression sensing method based on multi-level sampling and self-attention network according to claim 1, characterized in that: The sampling network in step 2 is S AMSM (·) The feature extraction capability of low sampling rate signals is enhanced by combining fuzzy pooling and maximum pooling through an adaptive multi-level sampling module. A custom neural network layer is used to further fuse multi-scale features and enhance the image restoration effect. The sampling process is expressed as: y=Φx=S AMSM (x) Where y is the low-resolution measurement, Φ is the sampling matrix, x is the input signal, and S AMSM (·) is the sampling network; in the case of 1.25% sampling rate, 3. The image compression sensing method based on multi-level sampling and self-attention network according to claim 1, characterized in that: The adaptive multi-level sampling module includes: two convolutional layers for feature extraction; a multi-scale pooling and fuzzy pooling module, wherein fuzzy pooling uses Gaussian filtering to smooth feature maps, and maximum pooling is used to extract local features, and weighted fusion is performed through an attention mechanism to obtain a weighted feature representation.

4. The image compression sensing method based on multi-level sampling and self-attention network according to claim 3, characterized in that: The weighted fusion formula of the output F1 of the fuzzy pooling module and the output F2 of the maximum pooling module is: y=o1·F1+o2·F2 Among them, o1 and o2 are two learnable weight parameters.

5. The image compression sensing method based on multi-level sampling and self-attention network according to claim 4, characterized in that: The input feature map is extracted through a module containing three convolutional layers and a ReLU activation function. Then, the input feature map is fuzzy pooled through BlurPool(·) to obtain a smoothed feature map. The result of the fuzzy pooling is then added to the features extracted by the convolution to obtain an intermediate feature map. Finally, the feature map is weighted through an attention mechanism and output as F1. The output process of F2 is similar to that of F1, but the operation performed on the input feature map is the maximum pooling MaxPool(·); Among them, the specific definitions of F1 and F2 are as follows: F1=(Conv(x)+BlurPool(x))×A blur F2=(Conv(x)+MaxPool(x))×A max Among them, A blur and A max They are the fuzzy attention and maximum attention modules, which consist of convolution and non-linear activation functions.

6. The image compression sensing method based on multi-level sampling and self-attention network according to claim 1, characterized in that: The operation of initializing and rebuilding the subnet is completed by the following formula: I(x)=PS(Conv(S AMSM (·))) Among them, PS is the pixel rearrangement operation, Conv is the convolution operation, S AMSM (·) is the output of the adaptive multi-stage sampling module.

7. The image compression sensing method based on multi-level sampling and self-attention network according to claim 6, characterized in that: Each multi-scale feature enhancement module in the deep reconstruction network operates through the following process: the input feature map is downsampled through two convolution operations to extract feature information of different scales; the downsampled features are spliced ​​with the multi-scale feature map, and the resolution is gradually restored through convolution fusion to enhance the detail information of the image.

8. The image compression sensing method based on multi-level sampling and self-attention network according to claim 7, characterized in that: The data flow process of the multi-scale feature enhancement module is as follows: S1, input data y generates three feature representations y0, y1, and y2 of different scales through the fusion module, where S2, through two convolution downsampling operations, the input features are gradually reduced in resolution to extract feature information of different scales; S3, by concatenating these downsampled features with the features y1 and y2 obtained from the multi-scale feature extraction module, and then gradually upsampling after convolution fusion, the resolution is gradually restored; S4, through multiple splicing and convolution operations, further fused high-resolution features, combined with information of different scales, and finally generated rich feature representation with multi-level details. This process is expressed as follows: f ↓ =Conv1(f t ),f ↓↓ =Conv2(f ↓ ) f1=Conv3(Concat(f ↓↓ ,y2))),f ↑ =Conv4(Conv(f1,f ↓↓ )) f2=Conv5(Concat(f ↑ ,y1)),f ↑↑ =Conv6(Convat(f2,f ↓ )) Among them, is the concatenation operation on the channel dimension of concat(·), S5. After the window self-attention module, multi-scale window division is performed and self-attention is calculated to capture richer local and global information. Multi-scale window division and self-attention weight calculation are performed to fine-tune the weighted results of the input features. The formula is: WSSA(·)=Concat(Attn1,Attn2,...,Attn n )+f ↑↑ Among them, Attn i Represents the attention of the sub-feature map of each window; S6. Then calculate the characteristic error Δ and initialize Δ to obtain Δ Init The measured values ​​are used to guide the reconstruction. The compensated features, measured features and original input are fused to generate the final reconstructed image f out : Δ Init =PS(S csmn (conv(f3))-y) f4=f3+Res(A Init ) f out =Concat(x,y0,f4) Among them, PS(·) is the pixel rearrangement operation, and Res(·) represents the residual connection.

9. The image compression sensing method based on multi-level sampling and self-attention network according to claim 8, characterized in that: The specific process of the multi-scale feature enhancement module is as follows: First, input feature map f ↑↑ Perform feature projection through 1×1 convolution and batch normalization to generate feature maps with more channels; Next, these channels are divided into multiple sub-feature maps, each of which corresponds to a different window size. For each sub-feature map, after being divided into small windows, the query (Q), key (K) and value (V) are extracted respectively and the self-attention weight is calculated, and the feature information is re-weighted using the attention mechanism; Then, the weighted feature map is reconstructed back to its original shape, and multiple sub-feature maps are concatenated in the channel dimension; Finally, the original number of channels is restored through a 1×1 convolution, and the result is connected to the input feature map f through a residual connection. ↑↑ Add together to form the final output.

Images