Image compressed sensing joint reconstruction method and system, storage medium and electronic equipment

Abstract

The invention provides an image compressed sensing joint reconstruction method and system, a storage medium and electronic equipment. The image compressed sensing joint reconstruction method comprises the following steps: acquiring an original image, and acquiring an image vector based on the original image; performing primary reconstruction and secondary reconstruction on the image vector to obtain an initial reconstruction image; and further performing image post-processing on the initial reconstructed image based on the fine tuning network to obtain a target reconstructed image. The method not only pays attention to observation in an ideal environment, but also considers the influence of noise and other degradation factors in a reconstruction process, so that the capability of accurately deducing image features and modes in a noisy environment of the model is remarkably improved, and the robustness of a reconstruction algorithm is enhanced; moreover, the quality of the initial reconstructed image is optimized, the reconstruction error is reduced, the target reconstructed image is closer to the original image, and the reconstruction accuracy and visual effect are improved.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and particularly relates to an image compressive sensing joint reconstruction method, system, storage medium, and electronic device. Background Art

[0002] Nowadays, with the continuous development of sensor technology, communication technology, and information processing technology, a large amount of data is continuously collected and processed. The early process of digitizing analog signals was inseparable from the traditional Nyquist-Shannon sampling theorem, which states that if one wants to completely retain the information in the original signal or accurately reconstruct the original signal, the sampling frequency must be more than twice the highest frequency of the original signal. However, due to the limitations of storage and transmission costs, how to efficiently collect and process data has become an urgent problem to be solved. The theory of image compressive sensing (Compressed Sensing) breaks through the traditional Shannon sampling theorem and is a major revolution in the field of signal processing. It utilizes the sparsity or low-dimensional characteristics of signals to achieve efficient sampling and reconstruction of signals through a small number of non-uniform samplings and compressive measurements.

[0003] When dealing with a large amount of data, especially when processing images in a large sensing matrix, existing compressive sensing methods usually adopt the block compressive sensing (Block Compressed Sensing, BCS) method. However, this block processing may introduce problems. When applying the zero-padding technique, if the width and height of the image cannot be divisible by the block size, zeros need to be added at the end of the image to meet the block size requirements. This operation usually leads to the problem of uneven diffusion on the image, thus generating obvious artifacts in the reconstructed image. They are two main factors leading to reconstruction errors, which may seriously affect the quality of image reconstruction. At the same time, existing methods only focus on observations in an ideal environment and ignore the influence of noise and other degradation factors during the reconstruction process. In a noisy environment, the presence of measurement noise significantly affects the pre-trained model and hinders its ability to accurately infer image features and patterns. The increasing randomness in the measured signal poses a challenge to the reconstruction algorithm in accurately recovering the original image. Summary of the Invention

[0004] To solve the above technical problems, this application provides an image compressive sensing joint reconstruction method, system, storage medium, and electronic device.

[0005] Specifically, this application provides an image compressive sensing joint reconstruction method, including the following steps:

[0006] Obtain an original image, and obtain an image vector based on the original image.

[0007] Perform primary reconstruction and secondary reconstruction on the image vectors respectively to obtain an initial reconstructed image.

[0008] And, perform image post-processing on the initial reconstructed image based on a fine-tuning network to obtain a target reconstructed image.

[0009] In the above technical solution, not only the observations in the ideal environment are concerned, but also the impacts of noise and other degradation factors during the reconstruction process are considered. By performing post-processing on the initial reconstructed image through the fine-tuning network, the ability of the model to accurately infer image features and patterns in a noisy environment is significantly improved, and the robustness of the reconstruction algorithm is enhanced.

[0010] In addition, through the post-processing step based on the fine-tuning network, the quality of the initial reconstructed image is further optimized, the reconstruction error is reduced, the target reconstructed image is made closer to the original image, and the accuracy and visual effect of the reconstruction are improved.

[0011] Further, the fine-tuning network at least includes an input layer, a network backbone, residual blocks, and an output layer; the network backbone includes multiple scales; each scale at least includes a downsampling layer and an upsampling layer, and an identity skip connection established between the downsampling layer and the upsampling layer; the number of output channels between each scale presents a geometric sequence; each scale introduces the residual blocks, each residual block includes an activation function, and the downsampling layer and the upsampling layer do not include the activation function.

[0012] In the above technical solution, establishing an identity skip connection between each scale helps to alleviate the problem of gradient disappearance, accelerate the network training process, and improve the stability and convergence speed of the network; the design of the residual blocks allows the network to learn the residuals (differences) between the input and the output instead of directly learning the output. This design not only helps the network converge faster but also improves the sensitivity of the network to subtle changes, thus better restoring image details; the downsampling layer is used to reduce the resolution to capture global features, and the upsampling layer is used to restore the resolution to reconstruct image details. Such a combination can more effectively handle the problem of image compressive sensing.

[0013] Further, the obtaining of the target reconstructed image includes:

[0014] Receive the initial reconstructed image through the input layer.

[0015] Perform a downsampling operation on the initial reconstructed image based on the downsampling layer using a 2*2 stride convolutional operation to output a first feature map through the output channels.

[0016] Perform an upsampling operation on the initial reconstructed image based on the upsampling layer using a 2*2 stride transposed convolutional operation to output a second feature map through the output channels.

[0017] Add the first feature map and the second feature map to form an identity skip connection, and obtain a target feature map.

[0018] In the above technical solution, through the combination of downsampling and upsampling operations, the network can extract and fuse image features at different levels. This multi-level feature extraction and fusion strategy can capture the global and local features of the image, improving the detail performance and overall quality of the reconstructed image; adding the first feature map and the second feature map to form an identity skip connection. This design helps to fuse image features at different resolutions, improving the richness and diversity of features. The identity skip connection allows information to be directly transmitted, reducing information loss, enabling the network to better learn image features, and improving the accuracy and consistency of the reconstructed image.

[0019] Further, obtaining the target reconstructed image further includes:

[0020] Based on the activation function, add the initial reconstructed image and the target feature map to form residual learning, obtain the target reconstructed image, and output the target reconstructed image through the output layer.

[0021] In the above technical solution, adding the initial reconstructed image and the target feature map and processing through the activation function realizes the fusion and enhancement of information. This design strategy can effectively combine feature information at different levels, improving the richness and diversity of features; through the non-linear transformation of the activation function, the network can better fuse and enhance this feature information, improving the accuracy and consistency of the reconstructed image.

[0022] Among them, the application of the activation function usually has a small amount of calculation and can quickly complete non-linear transformation, improving the calculation efficiency; by introducing the activation function at the appropriate position in the network, the network can enhance its expression ability and feature extraction ability while maintaining high computing efficiency, improving the overall computing efficiency and performance.

[0023] In addition, the selection of the activation function has high flexibility and adaptability and can be adjusted and optimized according to different image compressive sensing requirements and noise environments. By selecting different types of activation functions, such as ReLU, Leaky ReLU, Sigmoid, etc., the reconstruction effect can be further improved to meet the requirements of different application scenarios; among them, ReLU is preferably used in this application.

[0024] Further, obtaining the image vector includes:

[0025] Perform a block operation on the original image and use a flattening function to project the blocked original image into an image vector.

[0026] In the above technical solution, the operation of dividing the original image into blocks can decompose a large-sized image into multiple smaller blocks, which helps to simplify the processing process and reduce the computational complexity; using the flattening function to project the original image after block division into an image vector realizes the dimensionality reduction from a two-dimensional image to a one-dimensional vector. This dimensionality reduction operation reduces the data dimension, simplifies the subsequent calculation and storage requirements, and improves the efficiency of data processing.

[0027] Among them, the block division and flattening operations have high adaptability and flexibility and can be adjusted according to different image sizes and processing requirements; by adjusting the block size and flattening method, the processing effect can be further improved to meet the requirements of different application scenarios.

[0028] Further, the first reconstruction includes:

[0029] Jointly training the image vector based on the normal distribution and the backpropagation algorithm to form a Gaussian distribution; transposing the image vector to obtain a reconstruction matrix, and jointly training the reconstruction matrix based on the backpropagation algorithm to obtain a target matrix; and obtaining the first reconstructed image based on the Gaussian distribution and the target matrix.

[0030] In the above technical solution, the Gaussian distribution has a wide range of applications in statistics and can describe the distribution characteristics of many natural phenomena. By utilizing the characteristics of the Gaussian distribution, the network can better capture the potential distribution characteristics of the image vector and improve the quality of the reconstructed image; the backpropagation algorithm is a commonly used optimization algorithm in deep learning and can effectively adjust the network parameters to minimize the loss function. By jointly training the image vector and the reconstruction matrix using the backpropagation algorithm, the network can converge quickly and improve the training efficiency; the transpose operation helps to reconstruct image features, improve the expression ability and diversity of features. Through the transpose operation, the network can better capture the global and local features of the image and improve the detail performance and overall quality of the reconstructed image;

[0031] Among them, the first reconstruction process has high flexibility and adaptability and can be adjusted and optimized according to different image compressive sensing requirements and noise environments; by adjusting network parameters and structures, such as the distribution type, transpose method, training strategy, etc., the reconstruction effect can be further improved to meet the requirements of different application scenarios.

[0032] Further, the second reconstruction includes:

[0033] Perform gradient descent processing on the primary reconstructed image based on the transformer model to obtain image patches, and perform stitching processing on the image patches to obtain a pre-reconstructed image; extract features from the pre-reconstructed image through a convolutional neural network to obtain a basic feature map; input the basic feature map into the transformer model to obtain a detailed feature map; and, perform a convolution operation on the detailed feature map through the convolutional neural network to obtain a final feature map; fuse the final feature map with the primary reconstructed image to obtain a secondary reconstructed image as the initial reconstructed image.

[0034] In the above technical solution, gradient descent is a commonly used optimization algorithm that can adjust model parameters to minimize the loss function. Through gradient descent processing, the transformer model can better capture local and global features in the image, improving the quality of the pre-reconstructed image; performing stitching processing on the image patches obtained through gradient descent processing can restore the overall structure of the image. The stitching operation combines multiple small patches into a complete image, helping to reconstruct the global information and context relationship of the image, and improving the consistency and coherence of the pre-reconstructed image; the convolutional neural network is good at processing image data and can effectively extract information such as the texture, edges, and shapes of the image. The generation of the basic feature map helps to enhance the detail performance and structural information of the image; the transformer model is good at processing sequence data and can capture long-range dependencies. Through the processing of the transformer model, the detailed feature map can better express the high-level semantic information of the image and further refine the image features;

[0035] Among them, the secondary reconstruction process has high flexibility and adaptability and can be adjusted and optimized according to different image compressive sensing requirements and noise environments; by adjusting network parameters and structures, such as the number of layers of the transformer model, the size of the convolutional kernel, the training strategy, etc., the reconstruction effect can be further improved to meet the requirements of different application scenarios.

[0036] Based on the same concept, the present application also provides an image compressive sensing joint reconstruction system, including:

[0037] A sampling module, configured to obtain an original image and obtain an image vector based on the original image.

[0038] A reconstruction module, configured to perform primary reconstruction and secondary reconstruction on the image vector respectively to obtain an initial reconstructed image.

[0039] And an artifact removal module, configured to perform image post-processing on the initial reconstructed image based on a fine-tuning network to obtain a target reconstructed image.

[0040] In the above technical solution, each module of the image compressive sensing joint reconstruction system forms an efficient, stable and flexible image processing solution through the high efficiency of data acquisition and processing, the advantages of multi-level reconstruction, the richness of feature information, the optimization of the fine-tuning network, the refinement of image post-processing, the enhancement of robustness, and the improvement of computational efficiency. This system can significantly improve the quality and robustness of image compressive sensing joint reconstruction and meet the requirements of different application scenarios.

[0041] Based on the same concept, the present application also provides a storage medium storing a computer program, wherein the computer program is configured to execute the image compressive sensing joint reconstruction method when running.

[0042] Based on the same concept, the present application also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the image compressive sensing joint reconstruction method.

[0043] Compared with the prior art, the beneficial effects of the present application are as follows:

[0044] The present application first obtains an original image and obtains an image vector based on the original image; then performs primary reconstruction and secondary reconstruction on the image vector respectively to obtain an initial reconstructed image; and further performs image post-processing on the initial reconstructed image based on a fine-tuning network to obtain a target reconstructed image. The present application not only focuses on observations in an ideal environment but also considers the influence of noise and other degradation factors during the reconstruction process, significantly improving the model's ability to accurately infer image features and patterns in a noisy environment and enhancing the robustness of the reconstruction algorithm; and optimizing the quality of the initial reconstructed image, reducing the reconstruction error, making the target reconstructed image closer to the original image, and improving the accuracy and visual effect of the reconstruction. Description of the Drawings

[0045] Figure 1 It is a flowchart of the image compressive sensing joint reconstruction method described in the present application.

[0046] Figure 2 is Figure 1 the operation framework diagram of the image compressive sensing joint reconstruction method.

[0047] Figure 3 is for Figure 1 the system framework diagram of the image compressive sensing joint reconstruction method described.

[0048] Figure 4 is Figure 3Data flow diagram of the reconstruction module in the described image compressive sensing joint reconstruction system.

[0049] Figure 5 Schematic diagram comparing PSNR (dB) and SSIM of this application and the TransCS method on datasets Set11, Kodak24, and BSD68 at various sampling rates.

[0050] Figure 6 Effect diagram comparing the visual quality of TransCS and PSNR (dB) / SSIM of this application at a sampling rate of 0.25. Detailed implementation mode

[0051] The following further describes in detail an image compressive sensing joint reconstruction method, system, storage medium, and electronic device of this application in combination with specific embodiments and the accompanying drawings.

[0052] Embodiment 1:

[0053] Please refer to Figure 1-2 , this application provides an image compressive sensing joint reconstruction method, including the following steps:

[0054] First, obtain the original image and obtain an image vector based on the original image.

[0055] Furthermore, the obtaining of the image vector includes:

[0056] Perform a blocking operation on the original image and use a flattening function to project the blocked original image into an image vector.

[0057] In some embodiments, obtain the original image and perform a blocking operation on the original image to obtain C×S×S non-overlapping image blocks; where C represents the number of image channels and S represents the image block size.

[0058] The blocking operation on the obtained original image further includes:

[0059] Define the sampling rate τ = M / N (0 ≤ τ ≤ 1), then obtain the sensing matrix where M = τ×B 2 , N = B 2 ;

[0060] The sampling rate refers to the density or frequency of sampling a continuous signal per unit time. Usually, the higher the sampling rate, the greater the sampling density of the signal or image, and more details and information can be captured. In the fields of image reconstruction and compressive sensing, a high sampling rate can provide more original data, which helps to improve the quality and accuracy of the reconstructed image.

[0061] Use the flattening function Fv (·) Flatten the image block into a vector (i.e., the image vector). This vectorization process not only facilitates the execution of efficient mathematical operations but also helps in applying machine learning models, such as training the sensing matrix, and thus enables image reconstruction and analysis tasks.

[0062] In the above technical solution, performing a blocking operation on the original image can decompose a large-sized image into multiple smaller blocks, which helps to simplify the processing process and reduce the computational complexity; using a flattening function to project the blocked original image into an image vector realizes the dimensionality reduction from a two-dimensional image to a one-dimensional vector. This dimensionality reduction operation reduces the data dimension, simplifies the subsequent calculation and storage requirements, and improves the efficiency of data processing.

[0063] Among them, the blocking and flattening operations have high adaptability and flexibility and can be adjusted according to different image sizes and processing requirements; by adjusting the block size and flattening method, the processing effect can be further improved to meet the requirements of different application scenarios.

[0064] Furthermore, perform primary reconstruction and secondary reconstruction on the image vector respectively to obtain an initial reconstructed image.

[0065] Furthermore, the primary reconstruction includes:

[0066] Jointly train the image vector based on the normal distribution and the backpropagation algorithm to form a Gaussian distribution; transpose the image vector to obtain a reconstruction matrix, and jointly train the reconstruction matrix based on the backpropagation algorithm to obtain a target matrix; and obtain the primary reconstructed image based on the Gaussian distribution and the target matrix.

[0067] In some embodiments, the elements of the sensing matrix H are initialized with a normal distribution, and H is automatically jointly trained from the training images through backpropagation and finally forms a Gaussian-like distribution: y = H·F vec (F B (x)).

[0068] Specifically, in the training stage, first feed the image to be trained (i.e., the image vector) into the model. Subsequently, set the parameters of the sensing matrix as trainable variables. Evaluate and calculate the difference between the model output determined by the current parameters and the actual image through the backpropagation algorithm, and this difference is represented by a loss function. Then, use optimization techniques such as gradient descent to fine-tune the parameters of the sensing matrix according to the feedback of the loss function. As the training iterates, these parameters will be continuously optimized, enabling the model to better learn and represent the features of the training images. This process ultimately enables the model to achieve higher accuracy in tasks such as image reconstruction or classification.

[0069] Among them, an adaptive learning reconstruction matrix is used to complete the initial image reconstruction, specifically as follows:

[0070] Transpose the sensing matrix H to obtain a reconstruction matrix And perform automatic training through the backpropagation algorithm; obtain the initial reconstruction process, expressed as a reconstructed image in one time:

[0071]

[0072] In the above technical solution, the Gaussian distribution has a wide range of applications in statistics and can describe the distribution characteristics of many natural phenomena. By utilizing the characteristics of the Gaussian distribution, the network can better capture the potential distribution characteristics of the image vector and improve the quality of the reconstructed image; the backpropagation algorithm is a commonly used optimization algorithm in deep learning and can effectively adjust the network parameters to minimize the loss function. By jointly training the image vector and the reconstruction matrix through the backpropagation algorithm, the network can converge quickly and improve the training efficiency; the transpose operation helps to reconstruct the image features, improve the expression ability and diversity of the features. Through the transpose operation, the network can better capture the global and local features of the image and improve the detail performance and overall quality of the reconstructed image;

[0073] Among them, the reconstruction process in one time has high flexibility and adaptability and can be adjusted and optimized according to different image compressive sensing requirements and noise environments; by adjusting network parameters and structures, such as the distribution type, transpose method, training strategy, etc., the reconstruction effect can be further improved to meet the requirements of different application scenarios.

[0074] Furthermore, the secondary reconstruction includes:

[0075] Perform gradient descent processing on the reconstructed image in one time based on the transformer model to obtain image patches, and splice the image patches to obtain a pre-reconstructed image; extract features from the pre-reconstructed image through a convolutional neural network to obtain a basic feature map; input the basic feature map into the transformer model to obtain a detailed feature map; and, perform a convolutional operation on the detailed feature map through the convolutional neural network to obtain a final feature map; fuse the final feature map with the reconstructed image in one time to obtain a secondary reconstructed image as the initial reconstructed image.

[0076] In some embodiments, introducing a Transformer model (i.e., the varying period model) that solves parameters based on the iterative shrinkage threshold method for image secondary reconstruction includes:

[0077] First, perform gradient descent processing on each input sub-vector (i.e., the reconstructed image in one time), expressed as:

[0078]

[0079] Among them, k is the number of layers, and a trainable parameter λ is applied in the k-th layer k to optimize the result of gradient descent; its output result will be reorganized into image patches and stitched together (at this time, a pre-reconstructed image is obtained) to restore the size of the original image.

[0080] Among them, x k and respectively represent the input and output of the k-th layer gradient descent, and λ k represents the learnable step size of the k-th layer. The initial value x 1 of the first layer is initialized as x init , and the number of iterative layers is n.

[0081] Furthermore, for the secondary image reconstruction by introducing a Transformer model that solves parameters based on the iterative shrinkage threshold method, it also includes: extracting the basic feature maps of the pre-reconstructed image through a convolutional neural network (CNN), and these basic feature maps are then sent to a Transformer module for further processing to capture finer image details (at this time, detailed feature maps are obtained). The processed detailed feature maps go through another round of convolution operations and are fused and superimposed with the original input image (i.e., the first reconstructed image) to generate the final output image (i.e., the second reconstructed image and used as the initial reconstructed image).

[0082] In the above technical solution, gradient descent is a commonly used optimization algorithm that can adjust model parameters to minimize the loss function. Through gradient descent processing, the transformer model can better capture local and global features in the image, improving the quality of the pre-reconstructed image; stitching the image patches obtained through gradient descent processing can restore the overall structure of the image. The stitching operation combines multiple small patches into a complete image, helping to reconstruct the global information and context relationship of the image, and improving the consistency and coherence of the pre-reconstructed image; the convolutional neural network is good at processing image data and can effectively extract information such as the texture, edges, and shapes of the image. The generation of basic feature maps helps to enhance the detail performance and structural information of the image; the transformer model is good at processing sequence data and can capture long-range dependencies. Through the processing of the transformer model, the detailed feature maps can better express the high-level semantic information of the image and further refine the image features;

[0083] Among them, the secondary reconstruction process has high flexibility and adaptability and can be adjusted and optimized according to different image compressive sensing requirements and noise environments; by adjusting network parameters and structures, such as the number of layers of the transformer model, the size of the convolutional kernel, the training strategy, etc., the reconstruction effect can be further improved to meet the requirements of different application scenarios.

[0084] Further, perform image post - processing on the initial reconstructed image based on the fine - tuning network to obtain the target reconstructed image.

[0085] Further, the fine - tuning network at least includes an input layer, a network backbone, residual blocks, and an output layer; the network backbone includes multiple scales; each scale at least includes a down - sampling layer and an up - sampling layer, and an identity skip connection established between the down - sampling layer and the up - sampling layer; the number of output channels between each scale presents a geometric sequence; each scale introduces the residual blocks, each residual block includes an activation function, and the down - sampling layer and the up - sampling layer do not include the activation function.

[0086] In some embodiments, preferably, the fine - tuning network is composed of four scales, and each scale has an identity skip connection between a 2×2 stride convolution (SConv) down - sampling and a 2×2 transposed convolution (TConv) up - sampling operation. From the first scale to the fourth scale, the number of channels in each layer is 64, 128, 256, and 512 respectively. There is no activation function after the first and the last convolution (Conv) layer, as well as the SConv and TConv layers, and each residual block only includes one activation function.

[0087] Among them, the convolution layer in the fine - tuning network does not use bias terms, and the network introduces residual blocks to enhance the feature learning ability, and can adapt to different noise levels. By receiving a noise level map as input, it can process multiple noise conditions with a single model.

[0088] In the above - mentioned technical solution, establishing an identity skip connection between each scale helps to alleviate the problem of gradient disappearance, accelerate the network training process, and improve the stability and convergence speed of the network; the design of the residual block allows the network to learn the residual (difference) between the input and the output instead of directly learning the output. This design not only helps the network converge faster, but also improves the sensitivity of the network to subtle changes, so as to better restore image details; the down - sampling layer is used to reduce the resolution to capture global features, and the up - sampling layer is used to restore the resolution to reconstruct image details. Such a combination can more effectively process the problem of image compressive sensing.

[0089] Further, the obtaining of the target reconstructed image includes:

[0090] Receive the initial reconstructed image through the input layer.

[0091] Based on the down - sampling layer, perform a down - sampling operation on the initial reconstructed image using a 2*2 stride convolution operation to output a first feature map through the output channel.

[0092] The initial reconstructed image is upsampled based on the transposed convolution operation with a stride of 2*2 in the upsampling layer, so as to output a second feature map through the output channels.

[0093] The first feature map and the second feature map are added together to form an identity skip connection, and a target feature map is obtained.

[0094] In the above technical solution, through the combination of downsampling and upsampling operations, the network can extract and fuse image features at different levels. This multi-level feature extraction and fusion strategy can capture the global and local features of the image, improving the detail performance and overall quality of the reconstructed image; adding the first feature map and the second feature map to form an identity skip connection. This design helps to fuse image features at different resolutions, improving the richness and diversity of features. The identity skip connection allows information to be directly transmitted, reducing information loss, enabling the network to better learn image features, and improving the accuracy and consistency of the reconstructed image.

[0095] Further, obtaining the target reconstructed image further includes:

[0096] Based on the activation function, the initial reconstructed image and the target feature map are added together to form residual learning, and the target reconstructed image is obtained, and the target reconstructed image is output through the output layer.

[0097] In the above technical solution, adding the initial reconstructed image and the target feature map and processing them through the activation function realizes the fusion and enhancement of information. This design strategy can effectively combine feature information at different levels, improving the richness and diversity of features; through the nonlinear transformation of the activation function, the network can better fuse and enhance this feature information, improving the accuracy and consistency of the reconstructed image.

[0098] Among them, the application of the activation function usually has a small amount of calculation and can quickly complete the nonlinear transformation, improving the calculation efficiency; by introducing the activation function at the appropriate position in the network, the network can enhance its expression ability and feature extraction ability while maintaining high computational efficiency, improving the overall computational efficiency and performance.

[0099] In addition, the selection of the activation function has high flexibility and adaptability, and can be adjusted and optimized according to different image compressive sensing requirements and noise environments. By selecting different types of activation functions, such as ReLU, Leaky ReLU, Sigmoid, etc., the reconstruction effect can be further improved to meet the requirements of different application scenarios; among them, the present application preferably uses ReLU.

[0100] Embodiment 2:

[0101] Please refer to Figure 3-4, this application also provides a system using the image compressive sensing joint reconstruction method described in Embodiment 1. The system includes:

[0102] A sampling module, configured to obtain an original image and obtain an image vector based on the original image.

[0103] A reconstruction module, configured to perform primary reconstruction and secondary reconstruction on the image vector respectively to obtain an initial reconstructed image.

[0104] And an artifact removal module, configured to perform image post-processing on the initial reconstructed image based on a fine-tuning network to obtain a target reconstructed image.

[0105] In the above technical solution, each module of the image compressive sensing joint reconstruction system forms an efficient, stable and flexible image processing solution through the high efficiency of data acquisition and processing, the advantages of multi-level reconstruction, the richness of feature information, the optimization effect of the fine-tuning network, the refinement of image post-processing, the enhancement of robustness, and the improvement of computational efficiency. This system can significantly improve the quality and robustness of image compressive sensing joint reconstruction and meet the requirements of different application scenarios.

[0106] Embodiment 3:

[0107] This application also provides a storage medium storing a computer program, wherein the computer program is configured to execute the image compressive sensing joint reconstruction method described in Embodiment 1 when running.

[0108] In some embodiments, the storage medium stores a number of computer programs for enabling a device to execute all or part of the steps of the methods described in various embodiments of this application.

[0109] The medium may include various media such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disc that can store program codes.

[0110] Embodiment 4:

[0111] This application also provides an electronic device including a processor and a memory. The memory stores at least one instruction, at least one segment of program, a code set or an instruction set, and the at least one instruction, the at least one segment of program, the code set or the instruction set is loaded and executed by the processor to implement the image compressive sensing joint reconstruction method described in Embodiment 1.

[0112] In some embodiments, the memory and the processor are interconnected; the processor can be one or more CPUs. When the processor is a single CPU, it can be a single-core CPU or a multi-core CPU. The processor is used to control each functional module of the image compressive sensing joint reconstruction system and process signals.

[0113] The memory includes, but is not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), CD-ROM (Compact Disc Read-Only Memory). This memory is used to store computer programs, operating systems, various applications and data, such as storing the computer program for implementing the image compressive sensing joint reconstruction method.

[0114] Furthermore, in order to verify the effect of this application, a simulation experiment was conducted to compare the reconstruction performance of this application with the TranCS method. The evaluation criteria used are visual perception, PSNR, and SSIM. The higher the PSNR and SSIM, the better the performance of the method. For the comparison method, the code was downloaded from their respective websites and executed according to their default settings. All experiments in the simulation were configured as follows: Win11 operating system, Intel I9-13900HX CPU, GeForce RTX4060 GPU platform, and the simulation software was Pytorch1.9.0.

[0115] 1. Experimental dataset: The BSD500 was used to construct the training set. All images in the training set were randomly cropped into 200 sub-images (64×64 in size). To achieve image data augmentation, random horizontal flipping, vertical flipping, rotation, and scaling were used to increase the training images to 100,000 training sub-images. The validation set was set14, and the experimental results were tested using 3 widely used natural image sets, namely Set11, BSD68, and Urban100.

[0116] 2. Training Details: During the training phase, the Adam optimizer was used for the DRUNet deep learning model, and a learning rate scheduler was adopted. The model aims to minimize the squared loss function to facilitate the recovery of different noise levels in the input image. To prevent excessive learning rates and overfitting problems during training, we introduced a gradient clipping mechanism, setting a threshold to ensure that the gradient norm does not exceed a predefined limit during the gradient update process. Additionally, a relatively small initial learning rate was carefully selected and systematically reduced using the learning rate scheduler to maintain the stability of training. To address the overfitting problem, a dataset containing thousands of images was used to enhance the generalization ability of the model. Meanwhile, a regularization term containing L1 or L2 regularization was applied to limit the complexity of the model parameters and prevent overfitting to the training data. By monitoring the model performance on the validation set and stopping the training in a timely manner to prevent loss of generalization ability in the face of overfitting. To enhance the generalization ability of the model, embedded defect noise from the TranCS model was dynamically injected during training. During the training process, a learning rate scheduler was used to adjust the learning rate to optimize the convergence of the model. In the validation phase, the PSNR was used to evaluate the model performance on the validation set, demonstrating the organic combination of the deep learning model and the noise injection strategy in the challenges of real-world image denoising and artifact removal.

[0117] 3. Specific Implementation: First, the size of the image was set to 64 pixels wide by 64 pixels high. Considering that the size of the validation image is 128 pixels wide by 256 pixels high, the entire input image needs to be divided into 8 regions, each with a size of 64 pixels wide by 64 pixels high, and these regions will serve as the input to the network. In the sampling phase, the batch size B was set to 32, and at this time the image would be re-divided into 32 batches, each batch containing 4 image patches of 64×64 pixels. When the sampling rate was set to 0.1, the images in each batch would be further divided into a matrix H of 102 rows and 1024 columns, where the dimension of the observed value y is 102 rows by 4 columns. In the transformer module, the size of the input image is 1×1×64×64, and its dilation ratio is set to 48.

[0118] 4. Simulation Results: In the experiment, the sampling rates were set to 0.04, 0.1, 0.25, 0.3, 0.4, and 0.5 respectively. Table 1 shows the average PSNR and SSIM of the reconstructed images of this application and existing advanced methods on different datasets. As Figure 5 This is a comparison of the PSNR (dB) and SSIM of this application and the TransCS method on the datasets Set11, Kodak24, and BSD68 at each sampling rate.

[0119] As can be seen from the results, under the conditions of multiple datasets and different sampling rates, the performance of this application is better than other methods for comparison. This means that this application has high efficiency and effectiveness in the field of image compressive sensing. Since it can maintain its advantages under different conditions, it shows strong adaptability and robustness. Therefore, it can be inferred that this application may have broader applicability and better market prospects in practical applications.

[0120] Please refer to Figure 6 , which shows the reconstruction results of various methods for baby images and bird images at a 25% sampling rate. By comparing the reconstruction results, it can be found that the PSNR value and SSIM value of the method proposed in this application are higher than those of TransCS. Whether from the subjective visual perception or objective data, it indicates that the method of this application is better than other methods.

[0121] In summary, this application provides an image compressive sensing joint reconstruction method, system, storage medium, and electronic device; first, obtain the original image, and obtain the image vector based on the original image; then perform primary reconstruction and secondary reconstruction on the image vector respectively to obtain the initial reconstructed image; and further perform image post-processing on the initial reconstructed image based on the fine-tuning network to obtain the target reconstructed image. This application not only focuses on the observations in the ideal environment but also considers the influence of noise and other degradation factors during the reconstruction process, significantly improving the model's ability to accurately infer image features and patterns in a noisy environment and enhancing the robustness of the reconstruction algorithm; and optimizing the quality of the initial reconstructed image, reducing the reconstruction error, making the target reconstructed image closer to the original image, and improving the accuracy and visual effect of the reconstruction.

[0122] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of this application. Those of ordinary skill in the art can make various changes and modifications therein without departing from the scope and spirit of this application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.

[0123] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising said element.

[0124] Although the description of the present application is made in conjunction with the above specific embodiments, it is obvious that those skilled in the art can make many substitutions, modifications and changes based on the above content. Therefore, all such substitutions, improvements and changes are included within the spirit and scope of the appended claims.

Claims

1. A method for image compression sensing joint reconstruction, characterized in that: The following steps are involved: Acquire an original image, and acquire an image vector based on the original image; Performing primary reconstruction and secondary reconstruction on the image vector respectively to obtain an initial reconstructed image; And, performing image post-processing on the initial reconstructed image based on the fine-tuning network to obtain a target reconstructed image.

2. The image compression sensing joint reconstruction method according to claim 1, characterized in that: The fine-tuning network at least includes an input layer, a network backbone, a residual block and an output layer; The network backbone includes multiple scales; each scale includes at least a downsampling layer and an upsampling layer, and an identical skip connection established between the downsampling layer and the upsampling layer; The number of output channels between each scale is presented as a geometric progression; The residual block is introduced at each scale, each residual block includes an activation function, and the downsampling layer and the upsampling layer do not include the activation function.

3. The image compression sensing joint reconstruction method according to claim 2, characterized in that: The obtaining of the target reconstructed image comprises: Receiving the initial reconstructed image through the input layer; Performing a downsampling operation on the initial reconstructed image based on the downsampling layer using a convolution operation with a step size of 2*2 to output a first feature map through the output channel; Performing an upsampling operation on the initial reconstructed image based on the upsampling layer using a 2*2 step transposed convolution operation to output a second feature map through the output channel; The first feature map and the second feature map are added to form an identity skip connection to obtain a target feature map.

4. The image compression sensing joint reconstruction method according to claim 3 is characterized in that: The obtaining of the target reconstructed image further comprises: The initial reconstructed image and the target feature map are added based on the activation function to form residual learning, obtain a target reconstructed image, and output the target reconstructed image through the output layer.

5. The image compression sensing joint reconstruction method according to claim 4, characterized in that: The obtaining of the image vector comprises: The original image is divided into blocks, and a flattening function is used to project the divided original image into image vectors.

6. The image compression sensing joint reconstruction method according to claim 5, characterized in that: The primary reconstruction includes: jointly training the image vector based on a normal distribution and a back propagation algorithm to form a Gaussian distribution; Transposing the image vector to obtain a reconstruction matrix, and jointly training the reconstruction matrix based on the back propagation algorithm to obtain a target matrix; A reconstructed image is obtained based on the Gaussian distribution and the target matrix.

7. The image compression sensing joint reconstruction method according to claim 6, characterized in that: The secondary reconstruction includes: performing gradient descent processing on the primary reconstructed image based on a transformer model to obtain image blocks, and performing splicing processing on the image blocks to obtain a pre-reconstructed image; Extracting features from the pre-reconstructed image using a convolutional neural network to obtain a basic feature map; Inputting the basic feature map into the transformer model to obtain a detailed feature map; Performing a convolution operation on the detailed feature map through the convolutional neural network to obtain a final feature map; The final feature map is fused with the first reconstructed image to obtain a second reconstructed image as an initial reconstructed image.

8. A system using the image compression sensing joint reconstruction method according to any one of claims 1 to 7, characterized in that: include: A sampling module, used for acquiring an original image and acquiring an image vector based on the original image; A reconstruction module, used for performing primary reconstruction and secondary reconstruction on the image vector respectively to obtain an initial reconstructed image; And, a de-artifact module is used to perform image post-processing on the initial reconstructed image based on a fine-tuning network to obtain a target reconstructed image.

9. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the image compression sensing joint reconstruction method according to any one of claims 1 to 7 when running.

10. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the image compressed sensing joint reconstruction method as described in any one of claims 1 to 7.

Images