Compressed sensing image reconstruction method, device, storage medium and system

Abstract

The present invention discloses a compressed sensing image reconstruction method, device, storage medium, and system. The method includes: obtaining image data to be reconstructed, inputting the image data to be reconstructed into a pre-trained image reconstruction model to obtain a reconstructed image; the image reconstruction model uses a semi-quadratic splitting deep reconstruction network to iteratively process the image data to be reconstructed multiple times to obtain a reconstructed image; the semi-quadratic splitting deep reconstruction network includes an x-subproblem network, a b-subproblem network, and a q-subproblem network connected in sequence, wherein the x-subproblem network includes a multi-layer convolutional neural network; the b-subproblem network includes a soft threshold module; and the q-subproblem network includes a non-local neural network. The complex image reconstruction problem is decomposed into three simple sub-optimization problems: the x-subproblem, the b-subproblem, and the q-subproblem, and these problems are solved, thereby improving image reconstruction efficiency. Furthermore, the image reconstruction, which combines sparsity and non-local priors, has higher visual quality, clearer image texture, and more accurate reconstructed images.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a compressed sensing image reconstruction method, device, storage medium and system. Background Art

[0002] Compressed sensing (CS) theory breaks the constraints of the traditional Nyquist sampling theorem, eliminating the requirement for a sampling frequency twice the maximum frequency. The theory states that, given signal sparsity, signal compression and sampling can be achieved directly by mapping high-dimensional signals to low-dimensional ones. Finally, accurate reconstruction of the original signal can be achieved by solving a nonlinear optimization problem. Since its introduction, CS theory, which combines compression and sampling, has quickly become a research hotspot in fields such as image and video processing, radar, and medical magnetic resonance imaging.

[0003] Reconstruction is a key area of ​​research in compressed sensing theory. Accurate signal reconstruction is achieved by solving an NP-Hard problem. However, this problem does not have a unique solution. Consequently, scholars have conducted extensive research on reconstructing this problem. Over the years, research on compressed sensing reconstruction algorithms has been divided into two main categories: traditional reconstruction algorithms and deep learning-based reconstruction algorithms.

[0004] Among traditional reconstruction algorithms, model-based reconstruction is the most common, alongside earlier non-convex optimization algorithms, greedy matching pursuit algorithms, and convex optimization algorithms. The first three reconstruction algorithms all rely on signal sparsity, but the recovery results are often unsatisfactory. Model-based reconstruction algorithms, however, focus on prior information about the signal, leveraging different prior information to establish different reconstruction models. Their reconstruction results are superior to the other three types of algorithms. However, these traditional reconstruction algorithms require time-consuming image block searches or matrix inversion operations, significantly increasing computational time and reducing algorithm efficiency.

[0005] Deep learning's impressive performance in various advanced image processing tasks has earned it widespread acclaim and research from researchers across various fields. In recent years, some scholars have proposed applying deep learning frameworks to the field of compressed sensing. This idea has been widely recognized, and many excellent deep compressed sensing reconstruction algorithms have been developed. Ali Mousavi proposed the Stacked Denoising Autoencoder (SDA), which uses an unsupervised approach to achieve end-to-end mapping between measured values ​​and true values. The non-iterative reconstruction network (ReconNet) first applied convolutional neural networks to compressed sensing, achieving effective image reconstruction through fully connected and convolutional layers. Later, the Deep Residual Reconstruction Network (DR2-Net), developed based on ReconNet, combined the ReconNet structure with the residual concept to further improve the quality of reconstructed images. Although the existing deep learning-based image reconstruction algorithm has effectively overcome the drawbacks of traditional compressed sensing algorithms to a certain extent, it still suffers from the blocking effect problem (block-based transform coding is widely used in image compression coding. As the bit rate decreases, quantization becomes coarse, and discontinuities appear at the boundaries of blocks, forming obvious defects in the reconstructed image, which is called blocking effect), and does not make good use of the prior information of the image. Summary of the Invention

[0006] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a compressed sensing image reconstruction method, device, storage medium and system.

[0007] In order to achieve the above-mentioned purpose of the present invention, according to the first aspect of the present invention, the present invention provides a compressed sensing image reconstruction method, including: obtaining image data to be reconstructed, inputting the image data to be reconstructed into a pre-trained image reconstruction model to obtain a reconstructed image; the image reconstruction model includes a semi-quadratic splitting deep reconstruction network, and the image reconstruction model uses the semi-quadratic splitting deep reconstruction network to perform multiple iterative processes on the image data to be reconstructed to obtain a reconstructed image; the semi-quadratic splitting deep reconstruction network includes an x-subproblem network, a b-subproblem network and a q-subproblem network connected in sequence, the x-subproblem network includes a multi-layer convolutional neural network; the b-subproblem network includes a soft threshold module; the q-subproblem network includes a non-local neural network.

[0008] The above technical solution: uses semi-quadratic splitting to decompose the complex image reconstruction problem into three simple sub-optimization problems: x sub-problem, b sub-problem and q sub-problem for solution, which simplifies the solution process and improves the image reconstruction efficiency. The three sub-problems combine sparsity prior and non-local prior to reconstruct the image, thereby improving the reconstruction accuracy. In addition, the image reconstruction model uses a semi-quadratic splitting deep reconstruction network to perform multiple iterative processes on the reconstructed image data to obtain a reconstructed image, which further improves the reconstruction accuracy. Compared with existing reconstruction algorithms, this method has higher visual quality, clearer image texture, and more accurate reconstructed images.

[0009] In a preferred embodiment of the present invention, the x-subproblem network includes 5 layers of convolutional neural networks connected in sequence; the first convolutional neural network includes a first x-subproblem convolution layer and a first x-subproblem activation function layer, and the convolution kernel size of the first x-subproblem convolution layer is 3×3×1; the second convolutional neural network, the third convolutional neural network, and the fourth convolutional neural network all include a second x-subproblem convolution layer and a second x-subproblem activation function layer, and the convolution kernel size of the second x-subproblem convolution layer is 3×3×32; the fifth convolutional neural network includes a fifth x-subproblem convolution layer and a fifth x-subproblem activation function layer, and the convolution kernel size of the fifth x-subproblem convolution layer is 3×3×32.

[0010] The above technical solution: based on the powerful approximation ability of neural networks, uses shallow convolutional neural networks to approximate the solution of the x subproblem, learns the relationship between the degraded image and the original image through the convolutional neural network, makes full use of sparse priors, and can accelerate the reconstruction speed and reconstruction accuracy.

[0011] In a preferred embodiment of the present invention, the b-subproblem network includes a first b-subproblem convolution layer, a second b-subproblem convolution layer, a first b-subproblem activation function layer, a third b-subproblem convolution layer, a soft threshold module, a fourth b-subproblem convolution layer, a second b-subproblem activation function layer, a fifth b-subproblem convolution layer and a sixth b-subproblem convolution layer, which are connected in sequence.

[0012] The above technical solution: The network structure of sub-problem b is simple, and it makes full use of the sparse prior of the image to achieve efficient processing.

[0013] In a preferred embodiment of the present invention, the q-subproblem network includes a first q-subproblem convolutional layer, a non-local neural network, and a second q-subproblem convolutional layer connected in sequence.

[0014] The above technical solution makes full use of the non-local similarity prior of the image and can achieve efficient and accurate image reconstruction.

[0015] In a preferred embodiment of the present invention, the image reconstruction model utilizes the semi-quadratic splitting deep reconstruction network to iteratively process the image data to be reconstructed for a preset number of times to obtain a reconstructed image, and the preset number of times is 6 to 12.

[0016] The above technical solution: multiple iterations are performed to increase the accuracy of the reconstructed image, and the preset number of iterations is within the range of the optimal number of iterations selected in preliminary experiments, ensuring that the amount of calculation is appropriate while increasing the accuracy of the reconstructed image.

[0017] In a preferred embodiment of the present invention, the image reconstruction model also includes an initial reconstruction network arranged before the semi-quadratic splitting deep reconstruction network, which reconstructs the transposed matrix of the sampling matrix into multiple filters and reshapes the tensor output by the filter to obtain an initial reconstructed image.

[0018] The above technical solution: by initially reconstructing the network, it can amplify image features and significantly reduce the number of network parameters compared to existing deep learning-based methods.

[0019] In a preferred embodiment of the present invention, the image reconstruction model training process includes: constructing a sampling network, an initial reconstruction network, and a semi-quadratic split deep reconstruction network, wherein the sampling network is used to sample the original image; obtaining multiple original images to construct a training set; and jointly training the sampling matrix, the initial reconstruction network, and the semi-quadratic split deep reconstruction network based on the training set until a preset condition is met, wherein the preset condition is that the number of training times reaches a preset target number or the value of the loss function is less than or equal to a preset loss threshold.

[0020] The above technical solution: sampling through a sampling network, and jointly training the sampling network, the initial reconstruction network and the semi-quadratic split deep reconstruction network, is more conducive to training multiple image blocks.

[0021] In order to achieve the above-mentioned purpose of the present invention, according to the second aspect of the present invention, the present invention provides an image restoration device, including: an acquisition module, acquiring image data to be reconstructed; an image reconstruction model module, the image reconstruction model module including a semi-quadratic splitting deep reconstruction network, the image reconstruction model module using the semi-quadratic splitting deep reconstruction network to perform multiple iterative processes on the image data to be reconstructed to obtain a reconstructed image; the semi-quadratic splitting deep reconstruction network includes an x-subproblem network, a b-subproblem network and a q-subproblem network connected in sequence, the x-subproblem network includes a multi-layer convolutional neural network; the b-subproblem network includes a soft threshold module; the q-subproblem network includes a non-local neural network.

[0022] The above technical solution: uses semi-quadratic splitting to decompose the complex image reconstruction problem into three simple sub-optimization problems: x sub-problem, b sub-problem and q sub-problem for solution, which simplifies the solution process and improves the image reconstruction efficiency. The three sub-problems combine sparsity prior and non-local prior to reconstruct the image, thereby improving the reconstruction accuracy. In addition, the image reconstruction model uses a semi-quadratic splitting deep reconstruction network to perform multiple iterative processes on the reconstructed image data to obtain a reconstructed image, which further improves the reconstruction accuracy. Compared with existing reconstruction algorithms, this method has higher visual quality, clearer image texture, and more accurate reconstructed images.

[0023] In order to achieve the above-mentioned purpose of the present invention, according to the third aspect of the present invention, the present invention provides a computer-readable storage medium, in which computer-executable instructions are stored. When the processor executes the computer-executable instructions, the method described in the first aspect of the present invention is implemented.

[0024] The above technical solution: uses semi-quadratic splitting to decompose the complex image reconstruction problem into three simple sub-optimization problems: x sub-problem, b sub-problem and q sub-problem for solution, which simplifies the solution process and improves the image reconstruction efficiency. The three sub-problems combine sparsity prior and non-local prior to reconstruct the image, thereby improving the reconstruction accuracy. In addition, the image reconstruction model uses a semi-quadratic splitting deep reconstruction network to perform multiple iterative processes on the reconstructed image data to obtain a reconstructed image, which further improves the reconstruction accuracy. Compared with existing reconstruction algorithms, this method has higher visual quality, clearer image texture, and more accurate reconstructed images.

[0025] In order to achieve the above-mentioned objectives of the present invention, according to the fourth aspect of the present invention, the present invention provides an image compression sensing system, including an image sampling device and an image restoration device as described in the second aspect of the present invention, the image sampling device includes a sampling network module, and the sampling network module is jointly trained with the semi-quadratic splitting deep reconstruction network in the image restoration device.

[0026] The above technical solution: This system jointly trains a sampling network, an initial reconstruction network, and a semi-quadratic split deep reconstruction network. It first uses convolutional layers to simulate the sampling process, which is more conducive to training multiple image blocks compared to traditional sampling methods. It then uses convolutional layers and pixel shuffling for initial reconstruction, significantly reducing the number of parameters compared to existing deep learning-based methods. This system combines image sparsity priors and non-local similarity priors within a deep image compressed sensing network to establish a reconstruction model. Using semi-quadratic splitting, it decomposes the model into multiple subproblems, each of which is solved within the deep learning framework, resulting in significantly improved quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 1 is a flow chart of a compressed sensing image reconstruction method according to a specific embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of the structure of a compressed sensing system in a specific embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of the network structure of the x-subproblem in a specific embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram of the network structure of sub-problem b in a specific embodiment of the present invention;

[0031] Figure 5 This is a schematic diagram of the network structure of the q sub-problem in a specific embodiment of the present invention;

[0032] Figure 6 This is a schematic diagram of a non-local neural network structure in a specific embodiment of the present invention;

[0033] Figure 7 is a comparison diagram of a reconstructed image output by an image reconstruction model and an original image in a specific embodiment of the present invention;

[0034] Figure 8 It is a curve showing the PANR variation of the image reconstruction model under three compression ratios with the iteration of training data in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0036] In the description of the present invention, it should be understood that the terms "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention.

[0037] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal communication between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0038] The present invention discloses a compressed sensing image reconstruction method. In a preferred embodiment, Figure 1 As shown, the compressed sensing image reconstruction method includes:

[0039] Step S1, obtain the image data to be reconstructed. The image data to be reconstructed is the data obtained after compressed sensing of the original image. In order to adapt to the computing power of the computer, the entire original image is usually divided into multiple image blocks, and the image block size is B×B, where B is a positive integer, preferably but not limited to 33. Let the block sampling matrix be A B ∈R M×N , M<<N, R represents the real number domain. N represents the square of the image block size, N=B 2 , which represents the dimension of the image block. M is a positive integer, the product of the sampling rate and N, representing the dimension of the compressed data after sampling. The block sampling matrix is ​​preferably, but not limited to, a random Gaussian matrix. Image blocks are sampled using the block sampling matrix to obtain compressed data corresponding to the image blocks. This compressed data is transmitted to the reconstruction device as the image data to be reconstructed.

[0040] Step S2: Input the image data to be reconstructed into the pre-trained image reconstruction model to obtain the reconstructed image. Figure 2 As shown, the image reconstruction model includes a semi-quadratic split deep reconstruction network. The image reconstruction model uses the semi-quadratic split deep reconstruction network to iteratively process the image data to be reconstructed to obtain a reconstructed image. Specifically, the image data to be reconstructed is input into the semi-quadratic split deep reconstruction network for processing to obtain a first intermediate result. The first intermediate result is then input into the semi-quadratic split deep reconstruction network for processing to obtain a second intermediate result. This process is repeated several times to obtain multiple iterative results as the reconstructed image output. The semi-quadratic split deep reconstruction network includes an x-subproblem network, a b-subproblem network, and a q-subproblem network, which are connected in sequence. The x-subproblem network includes a multi-layer convolutional neural network; the b-subproblem network includes a soft threshold module; and the q-subproblem network includes a non-local neural network.

[0041] In this embodiment, preferably, the image data to be reconstructed is image compression data obtained through a sampling network. The sampling network is a convolutional network, and a layer of unbiased convolutional network is used to simulate the sampling process. The sampling network input is set to an image block of size B×B, and the block sampling matrix A is used. B ∈R M×NReshape into M learnable filters, each with a kernel size of The sampling process can be expressed as: Among them, x i ∈R N represents the i-th image block, y i ∈R M represents the measurement value of the i-th image block (image compression data), * is the convolution operation, is the filter weight matrix, which is learned and changed during training.

[0042] In this embodiment, preferably, the image reconstruction model also includes an initial reconstruction network arranged before the semi-quadratic splitting deep reconstruction network, which reconstructs the transposed matrix of the sampling matrix into multiple filters and reshapes the tensor output by the filter to obtain an initial reconstructed image.

[0043] In this embodiment, the initial reconstruction network uses a convolutional layer, firstly transforming the block sampling matrix A B The transposed matrix A B T ∈R M×N Reshape into N filters, each filter kernel size is 1×1×M, and then use pixel shuffling to reshape the tensor of size N×1×1 into a size of The image is used as the initial reconstructed image, and the initial reconstruction process can be expressed as:

[0044]

[0045] Among them, x i 0 represents the initial reconstructed image of the i-th original image block; PixelShuffle(·) represents the pixel shuffling operation, which is a method commonly used in super-resolution to amplify images and feature maps. Represents the convolutional layer weight matrix of the initial reconstruction network, which is learned and changed during training.

[0046] In this embodiment, the following optimized image reconstruction model is established by combining the sparse prior and the non-local similarity prior:

[0047]

[0048] Among them, x represents the reconstructed image; R(x) represents the regularization term, that is, the prior information, R l1 (x) is L1 sparse regularization, R non-local(x) represents the nonlocal regularization, and y represents the measurement value corresponding to the original image. argmin represents the maximum value. Solving the above equation allows for accurate image reconstruction, but directly solving it is computationally very complex. To simplify the solution, we use Half Quadratic Splitting (HQS).

[0049] The process of introducing a single variable semi-quadratic split includes: assuming that the semi-quadratic split is for the following optimization goal:

[0050]

[0051] The first term in the above formula is the data fidelity term, and the second term is the constraint term. First, introduce the variable z and rewrite the above formula to obtain:

[0052]

[0053] Its cost function is:

[0054]

[0055] Where μ is a non-increasing penalty parameter and A is the sampling matrix from x to y.

[0056] The calculation of the model is carried out by alternating solution. The expression of alternating solution is:

[0057]

[0058] The above-mentioned semi-quadratic splitting can effectively simplify the solution. Therefore, for the reconstruction model established by the present invention, the semi-quadratic splitting algorithm is adopted, and the auxiliary variables b and q are introduced. The optimized image reconstruction model can be equivalent to the following three sub-problems to be solved alternately:

[0059]

[0060] Where k represents the number of iterations, ranging from 6 to 12, preferably 9; x i represents the i-th image block to be estimated; μ1 represents the first penalty parameter; μ2 represents the second penalty parameter; λ1 represents the first penalty coefficient, which is 0.01; λ2 represents the second penalty coefficient, which is 0.01; b i Represents the input of the b-subproblem network when reconstructing the i-th image block; Indicates the square of the 2-norm; R l1 (b i ) means b i is the L1 norm regularization term of the input; q i Represents the input of the q-subproblem network when reconstructing the i-th image block; R non-local (qi ) represents q i is the non-local regularization term of the input.

[0061] represents the value of the auxiliary variable b obtained at the kth iteration during the reconstruction of the i-th image block; represents the value of the auxiliary variable q obtained at the kth iteration during the reconstruction of the i-th image block; if a direct mathematical solution is used, the computational complexity is high and the model performance is low. The present invention converts the solution of the three equations in the above formula into the solution of three subproblems: the first equation is the x-subproblem, which is implemented using a convolutional neural network (CNN); the second equation is the b-subproblem, which is solved by learning; and the third equation is the q-subproblem, which is solved based on a learnable non-local network.

[0062] For the x subproblem, its expression is:

[0063]

[0064] Formula (1) is a convex problem and can be solved directly with a closed solution, but the amount of computation required is very large. Due to the powerful approximation ability of neural networks, the present invention directly uses a shallow CNN network to approximate Formula (1). From Formula (1), it can be seen that x i is the i-th original image block to be estimated, and are all degraded image blocks of the i-th original image block, is the reconstructed image block corresponding to the i-th original image block in the k+1th iteration. Specifically, the convolutional neural network CNN is used to learn the relationship between the degraded image block and the original image block. The CNN structure based on convolution (Conv) and activation function (ReLU) is used to solve the above problem. The specific network structure is as follows: Figure 3 shown.

[0065] like Figure 3 As shown, the x-subproblem network includes five layers of convolutional neural networks connected in sequence; the first convolutional neural network includes the first x-subproblem convolution layer and the first x-subproblem activation function layer, and the convolution kernel size of the first x-subproblem convolution layer is 3×3×1; the second convolutional neural network, the third convolutional neural network, and the fourth convolutional neural network all include the second x-subproblem convolution layer and the second x-subproblem activation function layer, and the convolution kernel size of the second x-subproblem convolution layer is 3×3×32; the fifth convolutional neural network includes the fifth x-subproblem convolution layer and the fifth x-subproblem activation function layer, and the convolution kernel size of the fifth x-subproblem convolution layer is 3×3×32.

[0066] like Figure 3As shown, solving the x subproblem is a nonlinear reconstruction process. In this invention, a CNN is used to implement this function. The network consists of five layers, with the exception of the first and last layers. Each of the three intermediate layers has an identical network structure. Each intermediate layer outputs a feature map with 32 channels. The first layer operates on the initial reconstructed output and has 32 feature maps. The last layer consists of a single filter with 32 channels.

[0067] In this embodiment, the expression of subproblem b is:

[0068]

[0069] For R in the above formula (2), l1 (b i ), also using a general nonlinear transformation function Instead, the expression becomes:

[0070]

[0071] There exists a nonlinear transformation such that:

[0072]

[0073] where α is only related to The relevant parameters are set as the contraction threshold θ = λ1, and the value of θ can be taken as 0.01. Therefore, equation (3) can be written as:

[0074]

[0075] represents the 1-norm of the nonlinear transformation function; therefore The closed-form solution is:

[0076]

[0077] Therefore, the solution expression for b is:

[0078]

[0079] The parameters are learnable at each iteration, so:

[0080]

[0081] θ k Represents the value of θ at the kth iteration; for the solution process of the above b sub-problem, its network structure is as follows Figure 4As shown, the b-subproblem network includes a first b-subproblem convolutional layer, a second b-subproblem convolutional layer, a first b-subproblem activation function layer, a third b-subproblem convolutional layer, a soft threshold module, a fourth b-subproblem convolutional layer, a second b-subproblem activation function layer, a fifth b-subproblem convolutional layer, and a sixth b-subproblem convolutional layer, which are connected in sequence. The soft threshold module is preferably, but not limited to, a Softmax function module.

[0082] In this embodiment, a non-local neural network (Non-local Net) is used to solve the q-subproblem. The non-local neural network is based on the principle of the non-local mean method and realizes the non-local operation of calculating the response of a certain position as the weighted sum of the features of all positions. The q-subproblem network includes a first q-subproblem convolution layer, a non-local neural network, and a second q-subproblem convolution layer connected in sequence. For the q-subproblem, it is regarded as a nonlinear optimization problem, and the signal reconstruction process is realized by the non-local neural network. The network parameters are as follows: Figure 6 As shown in Figure 2, ξ, ψ, and δ are all convolution kernels of size 1×1, and × represents tensor product. The input x and output z maintain the same dimension.

[0083] In this embodiment, in order to achieve better reconstruction accuracy and efficiency, the image reconstruction model uses a semi-quadratic splitting deep reconstruction network to iterate the image data to be reconstructed for a preset number of times to obtain a reconstructed image, and the preset number of times is 6 to 12. Figure 2 As shown, the loop iteratively executes the semi-quadratic splitting depth reconstruction network, and the number of iterations is 6 to 12, preferably 9, which is obtained based on multiple experiments.

[0084] In a preferred embodiment, the image reconstruction model training process includes: constructing a sampling network, an initial reconstruction network, and a semi-quadratic split deep reconstruction network, where the sampling network is used to sample the original image; obtaining multiple original images to construct a training set; and jointly training the sampling matrix, the initial reconstruction network, and the semi-quadratic split deep reconstruction network based on the training set until a preset condition is met, wherein the preset condition is that the number of training times reaches a preset target number or the value of the loss function is less than or equal to a preset loss threshold.

[0085] In this embodiment, the i-th image block x is given i As input, i is a positive integer, and the joint sampling network, initial reconstruction network, and semi-quadratic split deep reconstruction network are trained to generate x i N R Iterative reconstruction results Represents x i The reconstruction result is output. Try to reduce x i and The difference between the two, while satisfying the symmetry constraint symbol is a symmetric constraint symbol. Therefore, the end-to-end loss function of the algorithm proposed in the present invention is designed as follows:

[0086]

[0087] Among them, N P Represents the total number of image blocks involved in training. N represents the number of image blocks x i The size of B 2 . N R The number of iterations of the preset semi-quadratic splitting deep reconstruction network is preferably 9. γ represents the regularization parameter, and γ is set to 0.01.

[0088] The present invention also discloses an image restoration device, including: an acquisition module, which acquires image data to be reconstructed; an image reconstruction model module, which includes a semi-quadratic splitting deep reconstruction network, and the image reconstruction model module uses the semi-quadratic splitting deep reconstruction network to perform multiple iterative processes on the image data to be reconstructed to obtain a reconstructed image; the semi-quadratic splitting deep reconstruction network includes an x-subproblem network, a b-subproblem network and a q-subproblem network connected in sequence, the x-subproblem network includes a multi-layer convolutional neural network; the b-subproblem network includes a soft threshold module; and the q-subproblem network includes a non-local neural network.

[0089] The present invention also discloses a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the image reconstruction method provided by the present invention is implemented.

[0090] The present invention also discloses an image compression sensing system, comprising an image sampling device and an image restoration device provided by the present invention. The image sampling device includes a sampling network module, which is jointly trained with a semi-quadratic splitting deep reconstruction network in the image restoration device. The image sampling device and the image restoration device are communicatively connected, sampling an original image and transmitting the sampled data to the image restoration device, which then reconstructs the original image based on the sampled data.

[0091] The compressed sensing image reconstruction method provided by this invention was validated in an application scenario. To verify the effectiveness of the proposed network and the quality of image reconstruction, a training set consisting of 88,912 randomly cropped image patches (33×33 pixels) was extracted from the 91image dataset, a dataset consistent with several mainstream deep learning-based reconstruction algorithms. The test set was constructed from three widely used benchmark datasets: Set5, Set11, and BSD68, containing 5, 11, and 68 grayscale images, respectively.

[0092] All experiments in this application scenario were run on an Intel i7-8700K processor with a 3.2GHz clock speed and 15.6 GiB of memory, accelerated by a GTX1080Ti GPU, and running the Ubuntu 16.04 operating system. Peak Signal to Noise Ratio (PSNR) was used as an objective quality evaluation metric.

[0093] In order to verify the effectiveness of the reconstruction network proposed in this invention, three pictures of parrots, houses and butterflies were selected from the data set for testing. First, the original pictures were input into the sampling network to obtain compressed sampling data, where the parrot and house pictures were compressed at a compression rate of 10%, and the butterfly picture was compressed at a compression rate of 20%. The compressed sampling data corresponding to the three original pictures were respectively input into the image reconstruction model provided by this invention for processing to obtain reconstructed images. Figure 7 As shown in the figure, the left column from top to bottom are the original images of the parrot, house, and butterfly, and the right column from top to bottom are the reconstructed images of the parrot, house, and butterfly. Figure 7 In the figure, the image in the larger box in the lower right corner is an enlargement of the image in the smaller box. Figure 7 As can be seen, the reconstructed image differs little from the original image, with PSNRs of 28.95dB, 33.02dB, and 32.88dB for the parrot, house, and butterfly, respectively. This demonstrates that the reconstruction method provided by the present invention has high visual quality, with very clear image textures. In summary, the compressed image reconstruction algorithm proposed in the present invention is an effective and accurate reconstruction algorithm.

[0094] Figure 8 The curves of PSNR changing with training data iteration under different compression ratios are given. Figure 8 As can be seen from the figure, with the increase of the number of training times, the PSNR value of the image reconstruction model provided by the present invention gradually stabilizes, that is, the reconstruction algorithm proposed by the present invention is a stable and effective reconstruction algorithm. MR represents the compression ratio.

[0095] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0096] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A compressed sensing image reconstruction method, characterized in that: include: Combining sparse prior and non-local similarity prior, an optimized image reconstruction model is established. Using a semi-quadratic splitting algorithm, the optimized image reconstruction model is equivalent to solving the x-subproblem, b-subproblem and q-subproblem alternately. Acquiring image data to be reconstructed, and inputting the image data to be reconstructed into a pre-trained image reconstruction model to obtain a reconstructed image; The image reconstruction model includes a semi-quadratic splitting deep reconstruction network, and the image reconstruction model uses the semi-quadratic splitting deep reconstruction network to perform multiple iterative processes on the image data to be reconstructed to obtain a reconstructed image; The semi-quadratic splitting deep reconstruction network includes an x-subproblem network, a b-subproblem network and a q-subproblem network connected in sequence, wherein the x-subproblem network includes a multi-layer convolutional neural network; the b-subproblem network includes a soft threshold module; and the q-subproblem network includes a non-local neural network. The b-subproblem network includes a first b-subproblem convolution layer, a second b-subproblem convolution layer, a first b-subproblem activation function layer, a third b-subproblem convolution layer, a soft threshold module, a fourth b-subproblem convolution layer, a second b-subproblem activation function layer, a fifth b-subproblem convolution layer, and a sixth b-subproblem convolution layer, which are connected in sequence. The q-subproblem network includes a first q-subproblem convolutional layer, a non-local neural network and a second q-subproblem convolutional layer connected in sequence.

2. The compressed sensing image reconstruction method according to claim 1, wherein: The x-subproblem network includes five layers of convolutional neural networks connected in sequence; The first convolutional neural network layer includes the first x sub-problem convolution layer and the first x sub-problem activation function layer. The convolution kernel size of the first x sub-problem convolution layer is 3×3×1; The second convolutional neural network, the third convolutional neural network, and the fourth convolutional neural network all include a second x sub-problem convolutional layer and a second x sub-problem activation function layer. The convolution kernel size of the second x sub-problem convolutional layer is 3×3×32. The fifth convolutional neural network includes a fifth x sub-problem convolution layer and a fifth x sub-problem activation function layer. The convolution kernel size of the fifth x sub-problem convolution layer is 3×3×32.

3. The compressed sensing image reconstruction method according to claim 1 or 2, wherein: The image reconstruction model utilizes the semi-quadratic splitting deep reconstruction network to iterate the image data to be reconstructed for a preset number of times to obtain a reconstructed image, where the preset number of times is 6 to 12.

4. The compressed sensing image reconstruction method according to claim 3, wherein: The image reconstruction model also includes an initial reconstruction network arranged before the semi-quadratic splitting deep reconstruction network, which reconstructs the transposed matrix of the sampling matrix into multiple filters and reshapes the tensor output by the filter to obtain an initial reconstructed image.

5. The compressed sensing image reconstruction method according to claim 3, wherein: The image reconstruction model training process includes: Constructing a sampling network, an initial reconstruction network, and a semi-quadratic splitting deep reconstruction network, wherein the sampling network is used to sample the original image; Obtain multiple original images to construct a training set; Based on the training set, the sampling matrix, the initial reconstruction network, and the semi-quadratic split deep reconstruction network are jointly trained until a preset condition is met, where the preset condition is that the number of training times reaches a preset target number or the value of the loss function is less than or equal to a preset loss threshold.

6. An image restoration device, characterized in that: For implementing the compressed sensing image reconstruction method according to any one of claims 1 to 5, the image restoration device comprises: An acquisition module, which acquires image data to be reconstructed; An image reconstruction model module includes a semi-quadratic splitting deep reconstruction network, and the image reconstruction model module uses the semi-quadratic splitting deep reconstruction network to perform multiple iterative processes on the image data to be reconstructed to obtain a reconstructed image; the semi-quadratic splitting deep reconstruction network includes an x-subproblem network, a b-subproblem network and a q-subproblem network connected in sequence, the x-subproblem network includes a multi-layer convolutional neural network; the b-subproblem network includes a soft threshold module; the q-subproblem network includes a non-local neural network.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method according to any one of claims 1 to 5 is implemented.

8. An image compression sensing system, characterized in that: The method comprises an image sampling device and an image restoration device as claimed in claim 6, wherein the image sampling device comprises a sampling network module, and the sampling network module is jointly trained with a semi-quadratic splitting deep reconstruction network in the image restoration device.

Images