Image depth compressed sensing reconstruction method, system, device and storage medium

Abstract

This invention relates to the field of image reconstruction technology, specifically to an image depth compression-sensing reconstruction method, system, device, and storage medium. To address the problem of poor image reconstruction results in existing technologies, this invention first performs compression processing on the image to be processed based on both the spatial and frequency domains to obtain compression effect values. When the compression effect value is greater than a compression effect threshold, the local spatial features and multi-scale attention fusion features of the initial reconstructed image are merged and then convolved to obtain the reconstructed image. When the compression effect value is not greater than the compression effect threshold, the initial reconstructed image is sequentially subjected to sparsification processing, and the sparse feature map is clustered to group similar features into one category, resulting in a sparse feature cluster map. After several noise addition processes, the sparse feature cluster map undergoes denoising and reconstruction processing based on an overcomplete dictionary to more accurately approximate the original image, thus obtaining the reconstructed image.

Description

Technical Field

[0001] This invention relates to the field of image reconstruction technology, specifically to an image depth compression sensing reconstruction method, system, device, and storage medium. Background Technology

[0002] Images are important carriers of information, generating massive amounts of data and typically occupying significant storage space, posing challenges for rapid transmission and real-time processing. Image compression can effectively reduce data storage requirements and transmission costs, improving data processing efficiency, but it relies on the Nyquist sampling theorem. Compressed sensing overcomes the limitations of the Nyquist sampling theorem, enabling image reconstruction at low sampling rates; however, traditional compressed sensing methods suffer from high computational costs and relatively low reconstruction quality when processing images.

[0003] With the rapid development of deep learning, deep neural networks have shown significant advantages in image representation learning and processing. When applied to compressed sensing, they can adaptively extract image information for compression and reconstruct the image using deep learning-driven reconstruction methods, thus improving the efficiency of compressed sensing. However, existing deep compressed sensing methods can only learn local image features. When processing large and complex images, they cannot fully capture global image information, easily leading to information loss and reduced quality of the reconstructed image. Summary of the Invention

[0004] The purpose of this invention is to provide an image depth compression sensing reconstruction method, system, device, and storage medium.

[0005] The technical solution of this invention is as follows:

[0006] An image depth compressed sensing reconstruction method includes the following operations:

[0007] S1. Perform spatial domain compression and frequency domain compression on the image to be processed to obtain a spatial domain compressed image and a frequency domain compressed image. Merge the spatial domain compressed image and the frequency domain compressed image according to the number of channels after measurement to obtain a compression evaluation image. Perform inverse transform processing based on the inverse measurement operator on the spatial domain compressed image and the frequency domain compressed image to obtain an initial reconstructed image. Obtain the compression effect value based on the compression evaluation image and the image to be processed. Determine whether the compression effect value is greater than the compression effect threshold. If it is greater, proceed to S2; if it is not greater, proceed to S3.

[0008] S2. Obtain the local spatial features and multi-scale attention fusion features of the initial reconstructed image respectively, merge them and then perform convolution processing to obtain the reconstructed image;

[0009] S3. The initial reconstructed image is sequentially subjected to high-dimensional feature sparsification and channel feature sparsification to obtain a sparse feature map. A set of neurons is initialized, and the sparse feature map is subjected to clustering mapping to obtain a sparse feature cluster map. After several noise additions, the sparse feature cluster map is subjected to denoising processing based on the U-Net network to obtain a denoised image. The denoised image is then processed using a reconstruction method based on an overcomplete dictionary to obtain a reconstructed image.

[0010] The spatial domain-based compression processing in S1 specifically involves: performing convolution processing, ReLU activation function processing, and convolution processing on the low-level features of the image to be processed to obtain the feature map to be processed; performing residual connection between the image to be processed and the feature map to be processed to extract global features to obtain a global deep feature map; and performing convolution and average pooling processing on the image to be processed, followed by fusion processing and masking function processing on the global deep feature map to obtain a spatial domain compressed image.

[0011] The frequency domain-based compression operation in S1 is achieved by mapping the image to be processed from the spatial domain to the frequency domain and then performing masking function processing.

[0012] The specific operation to obtain the compression effect value in S1 is as follows: after performing multi-scale decomposition on the compression evaluation image and the image to be processed, several compression evaluation scale decomposition maps and several scale decomposition maps to be processed are obtained.

[0013] At the same scale, the evaluation scale decomposition map, the scale decomposition map to be processed, and the residual map of the evaluation scale decomposition map and the scale decomposition map to be processed are merged to obtain several scale merged maps at different scales. The scale merged maps at different scales are then processed by batch normalization and ReLU activation function several times, followed by residual connection, convolution and average pooling to obtain scale pooling maps at different scales. After merging the scale pooling maps at all scales, they are processed by a multilayer perceptron to obtain the compression effect value.

[0014] In S2, the multi-scale attention fusion feature is obtained by extracting multi-scale attention features from the initial reconstructed image several times. Specifically, the multi-scale attention feature extraction process is as follows: the initial reconstructed image is split into channel features to obtain a first-channel feature map and a second-channel feature map; the first-channel feature map is processed by convolution, ReLU activation function, and then residually connected to the first-channel feature map to obtain a residual connected feature map; the second-channel feature map is divided into several windows, and the multi-head attention features of all windows are weighted and summed to obtain the initial attention fusion feature; the multi-head attention fusion feature is arranged according to the window order in the second-channel feature map to obtain the multi-head attention fusion feature; the multi-head attention fusion feature is merged with the residual connected feature map and then processed by convolution to obtain the initial multi-scale attention fusion feature; the initial multi-scale attention fusion feature is then residually connected to the initial reconstructed image to obtain the multi-scale attention fusion feature.

[0015] In S3, the conditional probability distribution during several noise addition processes is obtained using the following formula:

[0016] ,

[0017] For conditional probability distribution, , They are respectively in wheel, Output after -1 round of noise addition processing Indicates a Gaussian distribution. The mean value after adding noise. Indicates the first Wheel noise intensity, The variance of the noise term. It is the center of a sparse neighborhood.

[0018] The sparse neighborhood center is obtained using the following formula:

[0019] ,

[0020] For sparse neighborhood centers, For a set of neighborhood neurons, It is the first in the neighborhood j One neuron, This is a sparse feature clustering diagram.

[0021] An image depth compressed sensing reconstruction system, used to implement the above-mentioned image depth compressed sensing reconstruction method, includes:

[0022] The compression module and compression effect judgment module are used to perform spatial domain compression and frequency domain compression on the image to be processed, respectively, to obtain a spatial domain compressed image and a frequency domain compressed image; the spatial domain compressed image and the frequency domain compressed image are merged by the number of channels according to the measurement values ​​to obtain a compression evaluation image; the spatial domain compressed image and the frequency domain compressed image are subjected to inverse transform processing based on the inverse measurement operator to obtain an initial reconstructed image; based on the compression evaluation image and the image to be processed, a compression effect value is obtained; it is judged whether the compression effect value is greater than the compression effect threshold; if it is greater, the first reconstructed image generation module is executed; if it is not greater, the second reconstructed image generation module is executed.

[0023] The first reconstructed image generation module is used to obtain the local spatial features and multi-scale attention fusion features of the initial reconstructed image, and then perform convolution processing after merging to obtain the reconstructed image;

[0024] The second reconstructed image generation module is used to sequentially perform high-dimensional feature sparsification and channel feature sparsification on the initial reconstructed image to obtain a sparse feature map; by initializing a set of neurons, the sparse feature map is subjected to clustering mapping to obtain a sparse feature cluster map; after several noise addition processes, the sparse feature cluster map is subjected to denoising processing based on a U-Net network to obtain a denoised image; the denoised image is processed based on an overcomplete dictionary reconstruction method to obtain a reconstructed image.

[0025] An image depth compressed sensing reconstruction device includes a processor and a memory, wherein the processor implements the above-described image depth compressed sensing reconstruction method when executing a computer program stored in the memory.

[0026] A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described image depth compressed sensing reconstruction method.

[0027] The beneficial effects of this invention are as follows:

[0028] This invention provides an image depth compression-sensing reconstruction method. First, the image to be processed undergoes spatial domain compression and frequency domain compression, resulting in a spatial domain compressed image that considers both global and local information, and a frequency domain compressed image that retains the low-frequency components reflecting the global information of the image. Then, based on the spatial and frequency domain compressed images, a compression effect value is obtained. When the compression effect value is greater than a compression effect threshold, it indicates good compression quality. To improve computational efficiency while ensuring reconstruction quality, the local spatial features and multi-scale attention fusion features of the initial reconstructed image are merged and then convolved to obtain the reconstructed image. When the compression effect value is not greater than the compression effect threshold, it indicates good compression quality. The initial image compression quality was poor. To ensure reconstruction quality, the initial reconstructed image underwent high-dimensional feature sparsification and channel feature sparsification processes sequentially. This removed redundant information, highlighted key features, and mined the correlation between channels to obtain a sparse feature map. A set of neurons was initialized, and the sparse feature map was then subjected to clustering mapping to group similar features into one category, resulting in a sparse feature cluster map. After several rounds of noise addition, the sparse feature cluster map underwent denoising processing based on a U-Net network to restore the original image's inherent structure and texture information, yielding a denoised image. Finally, an overcomplete dictionary-based reconstruction method was used to process the denoised image, more accurately approximating the original image to obtain the reconstructed image, thus improving the overall image reconstruction quality. Detailed Implementation

[0029] This embodiment provides an image depth compressed sensing reconstruction method, including the following operations:

[0030] S1. Perform spatial domain compression and frequency domain compression on the image to be processed to obtain a spatial domain compressed image and a frequency domain compressed image. Merge the spatial domain compressed image and the frequency domain compressed image according to the number of channels after measurement to obtain a compression evaluation image. Perform inverse transform processing based on the inverse measurement operator on the spatial domain compressed image and the frequency domain compressed image to obtain an initial reconstructed image. Obtain the compression effect value based on the compression evaluation image and the image to be processed. Determine whether the compression effect value is greater than the compression effect threshold. If it is greater, proceed to S2; if it is not greater, proceed to S3.

[0031] S2. Obtain the local spatial features and multi-scale attention fusion features of the initial reconstructed image respectively, merge them and then perform convolution processing to obtain the reconstructed image;

[0032] S3. The initial reconstructed image is sequentially subjected to high-dimensional feature sparsification and channel feature sparsification to obtain a sparse feature map. A set of neurons is initialized, and the sparse feature map is subjected to clustering mapping to obtain a sparse feature cluster map. After several noise additions, the sparse feature cluster map is subjected to denoising processing based on the U-Net network to obtain a denoised image. The denoised image is then processed using a reconstruction method based on an overcomplete dictionary to obtain a reconstructed image.

[0033] The spatial domain-based compression processing in S1 specifically involves: performing convolution processing, ReLU activation function processing, and convolution processing on the low-level features of the image to be processed to obtain the feature map to be processed; performing residual connection between the image to be processed and the feature map to be processed to extract global features to obtain a global deep feature map; and performing convolution and average pooling processing on the image to be processed, followed by fusion processing and masking function processing on the global deep feature map to obtain a spatial domain compressed image.

[0034] S1. Perform spatial domain compression and frequency domain compression on the image to be processed to obtain a spatial domain compressed image and a frequency domain compressed image. Combine the spatial domain compressed image and the frequency domain compressed image according to the number of channels by measurement to obtain a compression evaluation image. Perform inverse transform processing based on the inverse measurement operator on the spatial domain compressed image and the frequency domain compressed image to obtain an initial reconstructed image. Obtain the compression effect value based on the compression evaluation image and the image to be processed. Determine whether the compression effect value is greater than the compression effect threshold. If it is greater, proceed to S2; if it is not greater, proceed to S3.

[0035] The image to be processed is subjected to spatial domain compression and frequency domain compression respectively, resulting in a spatial domain compressed image that takes into account both global and local information, and a frequency domain compressed image that retains the low-frequency part reflecting the global information of the image. Then, based on the spatial domain compressed image and the frequency domain compressed image, the compression effect value is obtained. According to the comparison between the compression effect value and the compression effect threshold, different image reconstruction methods are executed to avoid wasting computational resources and improve the flexibility of image reconstruction methods.

[0036] First, the image to be processed is compressed in both the spatial domain and the frequency domain to obtain a spatial domain compressed image and a frequency domain compressed image.

[0037] The steps for spatial domain-based compression processing are as follows.

[0038] Step 1: Perform convolution processing, ReLU activation function processing, and convolution processing on the low-level features of the image to be processed to obtain the feature map to be processed.

[0039] Specifically, in order to extract low-level features from the image to be processed, the image is processed by convolution. Mapping to the feature space yields convolutional feature maps , , Represents convolution transformation, This represents the number of channels in the image after convolution. It allows for the extraction of deeper or richer feature representations through continuous convolution without altering the spatial structure. In other words, it... After convolution transformation The activation function is applied, followed by convolution to obtain the feature map to be processed. , .

[0040] Step 2: After performing residual connection between the image to be processed and the feature map to be processed, extract global features to obtain a global deep feature map.

[0041] Specifically, to avoid losing information from the original image, a residual connection feature map is obtained by performing a residual connection between the image to be processed and the feature map to be processed. , And using a step size of The convolution transformation will from Each channel is mapped to Each channel, while preserving global information, performs deeper feature extraction to prevent information loss during the process, resulting in a global deep feature map. , ,in, .

[0042] Step 3: After convolution and average pooling, the image to be processed is fused with the global deep feature map and processed by a masking function to obtain a spatial domain compressed image.

[0043] Specifically, first, the image to be processed... Perform convolutional transformation, and then use average pooling to obtain the average pooling feature map to be processed. , This ensures that there is global or near-global statistical information during the compression process, assisting in the subsequent reconstruction process; the average pooling feature map to be processed... and global deep feature map The measurements of the residual network are obtained by fusion. , , For element-wise multiplication, This is a mask function used to control the actual measurement values ​​retained during the compression process.

[0044] The aforementioned frequency-domain compression operation is achieved by mapping the image to be processed from the spatial domain to the frequency domain and then performing masking function processing. Specifically, the image to be processed... This includes, but is not limited to, processing the image to be processed through discrete cosine transform, mapping the image from the spatial domain to the frequency domain, and obtaining the discrete transformed image to be processed. This facilitates more efficient compression. , Discrete cosine transform; image to be processed (discrete transform). The measured values ​​obtained through a masking function are used as a frequency domain compressed image. , ,in, .

[0045] Then, to evaluate the compression quality, the spatial domain compressed image and the frequency domain compressed image are merged by channel number based on the measured values ​​to obtain the compressed evaluation image. The compressed evaluation image can be obtained by... and Combined into measurement operators (measured values) get, , This indicates that the measured values ​​will be combined according to the number of channels.

[0046] Simultaneously, the spatial domain compressed image and the frequency domain compressed image are subjected to inverse transform processing based on the inverse measurement operator to obtain the initial reconstructed image, which is used to perform subsequent image reconstruction processes. The inverse measurement operator... Obtain the initial reconstructed image , .

[0047] Finally, based on the compression evaluation image and the image to be processed, the compression effect value is obtained. The specific operation steps are as follows.

[0048] Step 1: After performing multi-scale decomposition on the compressed evaluation image and the image to be processed, several compressed evaluation scale decomposition maps and several image to be processed scale decomposition maps are obtained. Specifically, for the compressed evaluation image... and the image to be processed Performing multi-scale decomposition can better guide how the network quantifies its performance at different scales. , Indicates the first The pixel values ​​of the image to be processed at each scale. Indicates the first Compressed evaluation image measurements at various scales, Indicates the image to be processed conduct Sub-sampling Represents compressed evaluation image conduct Sub-sampling.

[0049] Step 2: Merge the evaluation scale decomposition map, the scale decomposition map to be processed, and the residual map of the evaluation scale decomposition map and the scale decomposition map to be processed at the same scale to obtain several scale merged maps at different scales.

[0050] Will , , (The residual plots of the evaluation scale decomposition plot and the scale decomposition plot to be processed) are merged to obtain the first... l Scale merged graph of multiple scales , .

[0051] Step 3: Merge the scaled images of different scales, perform batch normalization and ReLU activation function processing several times, then perform residual connection, convolution, and average pooling to obtain scaled pooling images of different scales. Specifically, taking the first... l Scale merged graph of multiple scales For example, for Perform batch normalization, then apply. Activation function , Indicates batch normalization, Repeat the above two steps to obtain the result through residual connection. , ,right Perform a convolution transformation, and then obtain the output through average pooling. This helps reduce redundancy and better concentrate important information when performing aggregation at multiple scales in the future, resulting in the first... l Scale pooling graphs at various scales .

[0052] Step 3: After merging the scale pooling maps of all scales, process them through a multilayer perceptron to obtain the compression effect value. (This involves merging the scale pooling maps of all scales.) The scale pooling maps are fused to obtain the scale pooling fused map. , Scale pooling fusion graph After processing by a multilayer perceptron, Through multilayer perceptron (MLP) Projecting onto the real number field yields the compression effect value. .

[0053] Finally, determine whether the compression effect value is greater than the compression effect threshold; if it is greater, execute S2; if it is not greater, execute S3.

[0054] S2. Obtain the local spatial features and multi-scale attention fusion features of the initial reconstructed image respectively, merge them and then perform convolution processing to obtain the reconstructed image.

[0055] When the compression effect value is greater than the compression effect threshold, it proves that the compression quality is good. In order to improve the computational efficiency while ensuring the reconstruction quality, the local spatial features and multi-scale attention fusion features of the initial reconstructed image are merged and then convolved to obtain the reconstructed image.

[0056] First, obtain the local spatial features of the initial reconstructed image. This can be achieved through convolution transformation of the initial reconstructed image. from Each channel is mapped to One channel is used to obtain local spatial features. , .

[0057] Meanwhile, multi-scale attention fusion features are obtained by extracting multi-scale attention features from the initial reconstructed image several times (preferably 3 times).

[0058] The steps for multi-scale attention feature extraction are as follows.

[0059] Step 1: The initial reconstructed image is split into channel feature maps to obtain the first channel feature map and the second channel feature map. Specifically, the initial reconstructed image is processed by convolution transformation. Channel splitting is performed to obtain the feature map of the first channel. Second channel feature map This parallel splitting allows the network to process features from different channels using convolution and self-attention respectively, thereby integrating two complementary information extraction methods. , , And satisfy C represents the number of channels.

[0060] Step 2: On one hand, the first channel feature map, after being processed by convolution, ReLU activation function, and then convolution again, is residually connected to the first channel feature map to obtain a residual connected feature map. Specifically, the first channel feature map... After convolution transformation, the application is... The activation function is then used to perform a convolution transformation to obtain the first channel convolutional feature map. , Next, the first channel convolutional feature map and the first channel feature map The output of the convolutional part and the residual connection feature map are obtained through residual connections. This effectively alleviates the gradient vanishing problem in deep networks. .

[0061] On the other hand, after the second channel feature map is divided into several windows, the multi-head attention features of all windows are weighted and summed to obtain the initial attention fusion features; according to the window order in the second channel feature map, the multi-head attention fusion features are arranged to obtain the multi-head attention fusion features.

[0062] Specifically, firstly, the second channel feature map Divided into multiple sizes By limiting attention computation to within each small window, the computational complexity of self-attention is reduced, while also preserving local spatial structure, which is beneficial for detecting or reconstructing textures and edges in local regions. Feature map of the i-th window , Functions for dividing the window, Let be the window size, and let each window have dimensions of . There are a total of Each window has a query. Then, the query is calculated within each window. ,key Sum : , , ,in, , The total number of all windows, For the number of self-attention heads, This represents the dimension of each head. This is a linear projection matrix. Before reaching the cutoff point, self-attention weights are calculated within each window, and adaptive weights are assigned to each pixel to highlight key regions. , For normalization function, This is the relative position bias matrix. The multi-head attention from each window is merged to obtain... , ,in, The linear projection matrix is ​​used; all windows are merged to obtain the initial attention fusion features. Improve the model's ability to capture complex textures and details. , This indicates the window merging function; finally, to prevent the feature order from being disrupted, [the function is described]. Rearrangement yields multi-head attention fusion features , This represents a rearrangement function used to rearrange features processed by window back to their original combination.

[0063] Step 3: After merging the multi-head attention fusion features and residual connection feature maps, convolution processing is performed to obtain the initial multi-scale attention fusion features. Specifically, the multi-head attention fusion features are... and The initial multi-scale attention fusion features are obtained by merging and performing convolutional transformations. , .

[0064] Step 4: The initial multi-scale attention fusion features and the initial reconstructed image are processed through residual connections to obtain the multi-scale attention fusion features. , .

[0065] Finally, the local spatial features and multi-scale attention fusion features are merged and then convolved to obtain the reconstructed image. The local spatial features... Features with multi-scale attention fusion Merge the images and obtain the reconstructed images through convolution transformation. , .

[0066] S3. The initial reconstructed image is sequentially subjected to high-dimensional feature sparsification and channel feature sparsification to obtain a sparse feature map. A set of neurons is initialized, and the sparse feature map is subjected to clustering mapping to obtain a sparse feature cluster map. After several noise additions, the sparse feature cluster map is subjected to denoising processing based on the U-Net network to obtain a denoised image. The denoised image is then processed using a reconstruction method based on an overcomplete dictionary to obtain a reconstructed image.

[0067] When the compression effect value is not greater than the compression effect threshold, it indicates poor compression quality. To ensure reconstruction quality, the initial reconstructed image is sequentially subjected to high-dimensional feature sparsification and channel feature sparsification to remove redundant information, highlight key features, and mine the correlation between channels to obtain a sparse feature map. Then, by initializing a set of neurons, the sparse feature map is subjected to clustering mapping to group similar features into one category, resulting in a sparse feature clustering map. After several noise additions, the sparse feature clustering map is subjected to denoising processing based on a U-Net network to restore the original image's inherent structure and texture information, resulting in a denoised image. Finally, a reconstruction method based on an overcomplete dictionary is used to process the denoised image to more accurately approximate the original image, resulting in a reconstructed image.

[0068] First, the initial reconstructed image is subjected to high-dimensional feature sparsification and channel feature sparsification in sequence to obtain a sparse feature map.

[0069] Specifically, for the initial reconstructed image High-dimensional feature sparse encoding is performed by representing the high-dimensional input with a small number of basis vectors and removing redundant information to obtain high-dimensional sparse features. , , , For a complete dictionary, and , Noise is added; then, to improve representation quality and reduce cross-channel aliasing, the high-dimensional sparsity features are... Each channel Perform sparse coding. , For sparse feature maps, For the first The pixel values ​​of each channel. express Regularization, is the sparsity coefficient.

[0070] Then, by initializing a set of neurons As a sparse network, the sparse feature map is subjected to clustering mapping, so that similar features are mapped to adjacent neurons, resulting in a sparse feature clustering map. This represents the number of neurons.

[0071] The clustering mapping process is implemented using the following formula:

[0072] ,

[0073] This is a sparse feature clustering diagram. For sparse feature maps, Indicates the first Neurons in each channel.

[0074] Next, after several rounds of noise addition to the sparse feature clustering map, a denoising process based on the U-Net network is performed to obtain the denoised image. The noise addition process does not simply involve randomly adding noise, but rather spreading it along the structure of the sparse network, making the diffusion result more consistent with the data distribution.

[0075] During several noise-adding processes, the conditional probability distribution is obtained using the following formula:

[0076] ,

[0077] For conditional probability distribution, , They are respectively in wheel, -1 round, output after noise processing Indicates a Gaussian distribution. The mean value after adding noise. Indicates the first Wheel noise intensity, The variance of the noise term. , which represents the sparse neighborhood center of the coefficient network.

[0078] The sparse neighborhood center is obtained using the following formula:

[0079] ,

[0080] For sparse neighborhood centers, For a set of neighborhood neurons, It is the first in the neighborhood j One neuron, This is a sparse feature clustering diagram.

[0081] The noise addition process is implemented step by step, followed by forward diffusion, to obtain a sparse noisy feature map. , .

[0082] Meanwhile, in the denoising process based on the U-Net network, the conditional probability distribution is obtained through the following formula:

[0083] ,

[0084] The noise mean value predicted by the U-Net network after denoising. The denoised covariance matrix; Step by step, reverse the process until... The output denoised image is obtained. , .

[0085] Finally, the denoised image is processed using a reconstruction method based on an overcomplete dictionary to obtain the reconstructed image. Transpose and Denoising Images with Complete Dictionary Multiply to obtain the reconstructed image , .

[0086] This embodiment also provides an image depth compressed sensing reconstruction system for implementing the above-described image depth compressed sensing reconstruction method, including:

[0087] The compression module and compression effect judgment module are used to perform spatial domain compression and frequency domain compression on the image to be processed, respectively, to obtain a spatial domain compressed image and a frequency domain compressed image; the spatial domain compressed image and the frequency domain compressed image are merged by the number of channels according to the measurement values ​​to obtain a compression evaluation image; the spatial domain compressed image and the frequency domain compressed image are subjected to inverse transform processing based on the inverse measurement operator to obtain an initial reconstructed image; based on the compression evaluation image and the image to be processed, a compression effect value is obtained; it is judged whether the compression effect value is greater than the compression effect threshold; if it is greater, the first reconstructed image generation module is executed; if it is not greater, the second reconstructed image generation module is executed.

[0088] The first reconstructed image generation module is used to obtain the local spatial features and multi-scale attention fusion features of the initial reconstructed image, and then perform convolution processing after merging to obtain the reconstructed image;

[0089] The second reconstructed image generation module is used to sequentially perform high-dimensional feature sparsification and channel feature sparsification on the initial reconstructed image to obtain a sparse feature map; by initializing a set of neurons, the sparse feature map is subjected to clustering mapping to obtain a sparse feature cluster map; after several noise addition processes, the sparse feature cluster map is subjected to denoising processing based on a U-Net network to obtain a denoised image; the denoised image is processed based on an overcomplete dictionary reconstruction method to obtain a reconstructed image.

[0090] This embodiment also provides an image depth compressed sensing reconstruction device, including a processor and a memory, wherein the processor implements the above-described image depth compressed sensing reconstruction method when executing a computer program stored in the memory.

[0091] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described image depth compressed sensing reconstruction method.

[0092] This embodiment provides an image depth compression-sensing reconstruction method. First, the image to be processed undergoes spatial domain compression and frequency domain compression, resulting in a spatial domain compressed image that considers both global and local information, and a frequency domain compressed image that retains the low-frequency components reflecting the global information of the image. Then, based on the spatial and frequency domain compressed images, a compression effect value is obtained. When the compression effect value is greater than a compression effect threshold, it indicates good compression quality. To improve computational efficiency while ensuring reconstruction quality, the local spatial features and multi-scale attention fusion features of the initial reconstructed image are merged and then convolved to obtain the reconstructed image. When the compression effect value is not greater than the compression effect threshold, it indicates good compression quality. The initial image compression quality was poor. To ensure reconstruction quality, the initial reconstructed image underwent high-dimensional feature sparsification and channel feature sparsification processes sequentially. This removed redundant information, highlighted key features, and mined the correlation between channels to obtain a sparse feature map. A set of neurons was initialized, and the sparse feature map was then subjected to clustering mapping to group similar features into one category, resulting in a sparse feature cluster map. After several rounds of noise addition, the sparse feature cluster map underwent denoising processing based on a U-Net network to restore the original image's inherent structure and texture information, yielding a denoised image. Finally, an overcomplete dictionary-based reconstruction method was used to process the denoised image, more accurately approximating the original image to obtain the reconstructed image, thus improving the overall image reconstruction quality.

Claims

1. An image depth compressed sensing reconstruction method, characterized in that, This includes the following operations: S1. Perform spatial domain compression and frequency domain compression on the image to be processed to obtain spatial domain compressed image and frequency domain compressed image respectively. The spatial domain compressed image and the frequency domain compressed image are merged by the number of channels according to the measured values ​​to obtain the compressed evaluation image; The spatial domain compressed image and the frequency domain compressed image are subjected to inverse transform processing based on the inverse measurement operator to obtain the initial reconstructed image. Based on the compressed evaluation image and the image to be processed, the compression effect value is obtained. The specific operation to obtain the compression effect value is as follows: the compressed evaluation image and the image to be processed are decomposed into multiple scales to obtain several compressed evaluation scale decomposition maps and several unprocessed scale decomposition maps; at the same scale, the evaluation scale decomposition map, the unprocessed scale decomposition map, and the residual map of the evaluation scale decomposition map and the unprocessed scale decomposition map are merged to obtain several scale merged maps of different scales; the scale merged maps of different scales are processed by batch normalization and ReLU activation function several times, and then residual connection, convolution and average pooling are performed to obtain scale pooling maps of different scales; after merging the scale pooling maps of all scales, they are processed by a multilayer perceptron to obtain the compression effect value. Determine if the compression effect value is greater than the compression effect threshold; if it is, execute S2; if it is not, execute S3. S2. Obtain the local spatial features and multi-scale attention fusion features of the initial reconstructed image respectively, merge them and then perform convolution processing to obtain the reconstructed image; S3. Perform high-dimensional feature sparsification and channel feature sparsification on the initial reconstructed image in sequence to obtain a sparse feature map; By initializing a set of neurons, the sparse feature map is subjected to clustering mapping to obtain a sparse feature clustering map. After several rounds of noise addition, the sparse feature clustering map is denoised using a U-Net-based network to obtain a denoised image. The denoised image is processed using a reconstruction method based on an overcomplete dictionary to obtain the reconstructed image.

2. The image depth compression sensing reconstruction method according to claim 1, characterized in that, In S1, the spatial domain-based compression operation specifically includes: The low-level features of the image to be processed are processed by convolution, ReLU activation function, and convolution again to obtain the feature map to be processed; the image to be processed and the feature map to be processed are then residually connected, and global features are extracted to obtain the global deep feature map. After convolution and average pooling, the image to be processed is fused with the global deep feature map and processed by a masking function to obtain a spatial domain compressed image.

3. The image depth compression sensing reconstruction method according to claim 1, characterized in that, In S1, the frequency domain-based compression operation is achieved by mapping the image to be processed from the spatial domain to the frequency domain and then performing masking function processing.

4. The image depth compression sensing reconstruction method according to claim 1, characterized in that, In S2, the multi-scale attention fusion feature is obtained by extracting multi-scale attention features from the initial reconstructed image several times. The specific operation of multi-scale attention feature extraction is as follows: The initial reconstructed image is split into channel features to obtain the first channel feature map and the second channel feature map; After the first channel feature map is processed by convolution, ReLU activation function, and convolution again, it is residually connected with the first channel feature map to obtain the residual connected feature map. After the second channel feature map is divided into several windows, the multi-head attention features of all windows are weighted and summed to obtain the initial attention fusion features; According to the window order in the second channel feature map, the multi-head attention fusion features are arranged to obtain the multi-head attention fusion features. The multi-head attention fusion feature and the residual connection feature map are merged and then processed by convolution to obtain the initial multi-scale attention fusion feature; the initial multi-scale attention fusion feature and the initial reconstructed image are processed by residual connection to obtain the multi-scale attention fusion feature.

5. The image depth compression sensing reconstruction method according to claim 1, characterized in that, In step S3, the conditional probability distribution is obtained through the following formula during several noise addition processes: ， For conditional probability distribution, , They are respectively in wheel, -1 round, output after noise processing Indicates a Gaussian distribution. The mean value after adding noise. Indicates the first Wheel noise intensity, The variance of the noise term. It is the center of a sparse neighborhood.

6. The image depth compressed sensing reconstruction method according to claim 1, characterized in that, The sparse neighborhood center is obtained using the following formula: ， For sparse neighborhood centers, For a set of neighborhood neurons, It is the first in the neighborhood j One neuron, This is a sparse feature clustering diagram.

7. An image depth compressed sensing reconstruction system, used to implement the image depth compressed sensing reconstruction method of claim 1, characterized in that, include: The compression module and the compression effect judgment module are used to perform spatial domain compression processing and frequency domain compression processing on the image to be processed, respectively, to obtain spatial domain compressed image and frequency domain compressed image. The spatial domain compressed image and the frequency domain compressed image are merged by the number of channels according to the measured values ​​to obtain the compressed evaluation image; The spatial domain compressed image and the frequency domain compressed image are subjected to inverse transform processing based on the inverse measurement operator to obtain the initial reconstructed image; the compression effect value is obtained based on the compression evaluation image and the image to be processed; it is determined whether the compression effect value is greater than the compression effect threshold; if it is greater, the first reconstructed image generation module is executed. If it is not greater than, execute the second reconstructed image generation module; The first reconstructed image generation module is used to obtain the local spatial features and multi-scale attention fusion features of the initial reconstructed image, and then perform convolution processing after merging to obtain the reconstructed image; The second reconstructed image generation module is used to sequentially perform high-dimensional feature sparsification and channel feature sparsification on the initial reconstructed image to obtain a sparse feature map. By initializing a set of neurons, the sparse feature map is subjected to clustering mapping to obtain a sparse feature clustering map. After several rounds of noise addition, the sparse feature clustering map is denoised using a U-Net network to obtain a denoised image. The denoised image is then processed using an overcomplete dictionary-based reconstruction method to obtain a reconstructed image.

8. An image depth compression sensing reconstruction device, characterized in that, It includes a processor and a memory, wherein the processor implements the image depth compressed sensing reconstruction method as described in any one of claims 1-6 when executing a computer program stored in the memory.

9. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the image depth compressed sensing reconstruction method as described in any one of claims 1-6.

Images