Image super-resolution reconstruction method based on compressed sensing

Abstract

The invention discloses an image super-resolution reconstruction method based on compressed sensing, and belongs to the field of image reconstruction. The problems that the updating speed of existing hardware cannot meet the requirement for signal precision, and on the premise that the hardware cost is not increased, an image with higher definition cannot be reconstructed are solved. The invention discloses an image super-resolution reconstruction method based on compressed sensing, and the method comprises the steps: designing a dictionary learning process, and obtaining high-frequency component parameters needed in image restoration; comprising the steps of constructing a training data set and performing dictionary training; wherein a corresponding low-resolution image is obtained by carrying out quality reduction and scale reduction calculation on an image so as to carry out a process of training a data set; constructing an over-complete dictionary pair by using a K-SVD algorithm, and training the dictionary; firstly, a reduced low-frequency component is obtained, and then the resolution is reconstructed. According to the method, the GPSR algorithm is utilized, the limitation of fixed sparseness of a traditional OMP algorithm is overcome, a more accurate sparse coefficient is obtained, and the precision of the reconstructed image is further improved by adopting a bilateral filtering mode.

Description

Technical Field

[0001] The present invention relates to an image super-resolution reconstruction method, and in particular to an image super-resolution reconstruction method based on compressed sensing. Background Art

[0002] Today, the camera systems on smartphones are getting higher and higher in pixels, which is the result of the high development of integrated sensing and computing. In addition, the application range of integrated sensing and computing has expanded to a considerable range of fields, from drones, factory robots and IoT devices to self-driving cars. Therefore, it is safe to assume with the above examples that the real potential of sensing and computing integration can still be explored.

[0003] In order to improve the accuracy of the image, the program often requires more data to be collected and the system to support complex calculations. However, in resource-constrained sensor devices, such as microprocessors, wearable devices, and IoT devices, it is often unable to cope with the sampling and computational burden of sensor data in modern machine learning. Smart devices equipped with multi-core CPUs and GPUs and relatively large storage space, such as smartphones, can run certain deep learning models to a certain extent, but can only support the use of carefully optimized models under the premise of occasional use. Even without considering the computational burden during use, sensor sampling requires mobile devices to pay huge energy. In mobile devices, batteries are a precious resource in mobile computing, and the development of battery technology lags seriously behind the improvement of processing and components. With further breakthroughs in computing systems, this problem cannot be solved at present.

[0004] In summary, without a new generation of hardware breakthroughs, in order to support advanced applications, it is necessary to rely on reducing the burden of sampling and calculation, which cannot be solved by traditional solutions. According to the Nyquist sampling theorem, the signal can only be reliably restored when the sampling rate of the sensor is twice the highest frequency in the signal. However, in reality, the signal changes very quickly, and objects can only be identified in images with sufficient resolution. Wireless radars require millions of acquisitions to detect tiny movements, etc. However, in the 2000s, there was a method called compressed sensing that can successfully reconstruct even at a rate lower than the Nyquist sampling theorem. When the sampled signal meets the sparsity of the signal, that is, in a certain projection, most of the components of the signal are zero and incoherent, the sampling rate can be greatly reduced through compressed sensing, and then the signal is reconstructed through data processing, which is usually simplified to the fuzzy solution of the underdetermined linear system.

[0005] However, the method of using compressed sensing to reduce the sampling rate also has its limitations, and the application of compressed sensing is still challenging. First, not all reduced rate sampling is equal. The compression rate interacts with the characteristics of the input, affecting the ability to successfully reconstruct the signal from a finite number of samples. In addition, in early practice, the sampling method of compressed sensing focused on random sampling, but recent research has shown that designed sampling strategies are more practical. Second, in theory, reconstructing signals from sparse components requires solving the NP problem of non-zero signals. In current practice, the problem is solved by iteration, but the iteration method requires large calculations. Therefore, in order to achieve high-resolution reconstruction of signals with less sampling, it is urgent to construct a suitable network structure for the application of compressed sensing.

[0006] The low resolution level of an image refers to the small number of pixels representing the target and the lack of high-frequency component information (i.e. blur or noise) on the other hand. Interpolation and magnification can solve the problem of image size; image deblurring and image denoising can solve the problem of image blur and noise; image registration can solve the problem of information fusion of multiple frames. Image super-resolution reconstruction technology can solve all the above problems and improve image quality comprehensively. The concept of image super-resolution reconstruction was proposed in 1964 and 1968, but it was not recognized by the public at that time. It was not until 1984 that a method based on image sequences in the frequency domain was proposed, and image reconstruction achieved a key breakthrough. After that, a large number of scholars began to conduct in-depth research on this technology. From different perspectives, image super-resolution reconstruction technology can be divided into many types, including interpolation-based, reconstruction-based and learning-based methods.

[0007] For super-resolution reconstruction, the image obtained based on the traditional method is fast, but the reconstructed image is relatively blurry. The image super-resolution method based on reconstruction improves the quality of the reconstructed image, but requires multiple frames of low-resolution images with sub-pixel displacement and accurate image registration methods, and has poor stability. The image super-resolution reconstruction method based on learning uses machine learning algorithms to learn the correspondence between low-resolution images and high-resolution images from a large number of training samples, and obtains better quality and reconstruction speed, which is a significant improvement compared to traditional methods.

[0008] Unlike the traditional Nyquist sampling theorem, the sampling rate of compressed sensing technology CS no longer depends on the bandwidth of the signal, but is based on the structure and content characteristics of the effective information in the signal. After more than ten years of continuous development and expansion, many new branches have emerged from the theory of compressed sensing. The reconstruction of compressed sensing signals can be simplified to solving the l1 norm under the conditions of a certain set of equations through convex optimization. In the past four decades, neural networks have been used to solve various optimization problems, and different neural network models have been developed to solve convex optimization problems. In the literature on compressed sensing, deep learning solutions for signal reconstruction can be divided into solutions that follow the general principles set by traditional iterative reconstruction algorithms and solutions that directly utilize deep learning modeling capabilities. In the field of image reconstruction, compressed sensing has demonstrated its unique excellence. For high-rate measurement processes, deep learning-related algorithms using compressed sensing have shown good speed and restoration effects. Therefore, using theoretical research on compressed sensing to perform super-resolution reconstruction has better results than traditional algorithms in terms of reconstruction time and quality of reconstruction results.

[0009] Nowadays, with the development of production and life such as industry, military, and entertainment, the application value of high-resolution images is increasing. Using image reconstruction technology, the resolution of images can be improved on a certain hardware basis without increasing the amount of hardware, and the value prospects are huge. Summary of the invention

[0010] The purpose of the present invention is to solve the problem that the update speed of existing hardware cannot meet the requirements of signal accuracy and how to reconstruct a higher definition image without increasing the hardware cost, and to propose an image super-resolution reconstruction method based on compressed sensing.

[0011] The above purpose is achieved through the following technical solutions:

[0012] A method for image super-resolution reconstruction based on compressed sensing is implemented by the following steps:

[0013] Step 1: Design a dictionary learning process to obtain the high-frequency component parameters required in image restoration; including constructing a training data set and performing dictionary training; wherein, the image is degraded and scaled down to obtain the corresponding low-resolution image for the training data set process; an overcomplete dictionary pair is constructed using the K-SVD algorithm to train the dictionary;

[0014] Step 2: reconstructing the image, including: first obtaining the restored low-frequency components, and then obtaining a high-resolution image.

[0015] Furthermore, the process of constructing an overcomplete dictionary pair using the K-SVD algorithm and training the dictionary described in step 1 is specifically as follows:

[0016] Step 1: Through the given original sample Y training and learning, construct the redundant dictionary D and sparse matrix X of the sparse representation sample; each column of the original sample Y represents a sample, the dictionary D stores the features of Y, and the sparse matrix X represents Y through the linear combination of D, as shown in the following formula:

[0017] Y m×n =D m×s ·X s×n

[0018] When s is much smaller than m and n, the dimensions of D and Y will be much smaller than the original Y. The K-SVD algorithm is a commonly used dictionary learning method. It draws on the ideas of K-means clustering and SVD algorithm, and constructs a dictionary for sparse representation through singular value decomposition (SVD). Under the condition that both D and X are unknown, it solves the optimization problem shown in the following formula, namely:

[0019]

[0020] Where Y = [y1, ..., y n ] is the input sample matrix, each column of which represents an input sample, X = [x1, ..., x m ] is a sparse representation matrix, x i for y i The corresponding sparse representation coefficients are, represents the square of the Frobenius norm of the matrix Y-DX, T is the parameter for constraining the sparsity of the coefficients; if you directly restrict ||x i ||0=1, then the above formula is K-means clustering, which updates the dictionary D and the sparse representation coefficient x of the corresponding column column by column through a two-step iterative process i , stop when the convergence condition is met or the maximum number of iterations is reached, and finally obtain a redundant dictionary D and sparse representation matrix X that can accurately express the sample characteristics. The specific steps are as follows:

[0021] Initialization: Randomly select s samples from sample Y to initialize dictionary D. Each column in D is a sample feature vector, and perform L2 normalization on the feature vector;

[0022] Sparse coding: Fixed dictionary D, using OMP algorithm to solve the sparse representation coefficients of all samples, for each sample y i and its corresponding sparse representation coefficient x i have:

[0023]

[0024] Update dictionary: fix the sparse matrix X, update each column of atoms in the dictionary D and update the corresponding row of X; take the kth atom d k For example, the objective function can be rewritten as:

[0025]

[0026] in, is the error matrix. The solution of the above objective function can be transformed into adjusting d k and Multiply it by E k The error is as small as possible; choose E corresponding to the non-zero terms in k Participate in the calculation and define the set of non-zero items:

[0027]

[0028] E k By Omega k The limitation of E k ′, for E k ′ Perform SVD decomposition to obtain E k ′=UΔV T , then the solution d k It is given by the first column of U and the product of the first column of V and Δ(1,1) Update; finally repeat the sparse coding and dictionary update steps until the preset number of iterations or convergence condition is reached;

[0029] Step 1 and 2: Improve the sparse coefficient;

[0030] In the compressed sensing theory reconstruction framework, the gradient projection sparse reconstruction algorithm is used to solve the sparse representation coefficients.

[0031] Furthermore, the process of image reconstruction described in step 2 is specifically as follows:

[0032] First, after completing the training set, the input low-resolution image is interpolated to obtain the enlarged pixel size;

[0033] Secondly, bilateral filtering is used to extract low-frequency information, and then the parameters obtained by dictionary learning and the input image are used to calculate the high-frequency information component;

[0034] Finally, the corresponding high-resolution image is reconstructed through synthesis and superposition;

[0035] Among them, bilateral filtering performs filtering operation and represents the pixel value weight as G r , the spatial distance weight is represented by G s ,have:

[0036]

[0037]

[0038] Then the entire filter can be expressed as BF, and the filtering result is:

[0039]

[0040] Where W q is the weighted sum of each pixel value in the filter window, which is used for weight normalization;

[0041]

[0042] In the flat area, the G of each pixel in the filter r The value is similar, the spatial distance weight G s Dominant filtering effect; in the edge area, G on the same side of the edge r The value is similar and much larger than G on the other side of the edge. r value;

[0043] When noise points appear in a flat area, the weights of the signals around the noise points are very small and are normalized.

[0044] The beneficial effects of the present invention are:

[0045] This paper reviews the main framework of compressed sensing theory, describes the roles of the three main parts of sparse matrix, measurement matrix and reconstruction algorithm in the whole theory, summarizes the corresponding algorithms, and compares the three commonly used algorithms of sparse matrix, measurement matrix and reconstruction algorithm. This paper uses the prior knowledge of local sparsity of images in the existing algorithms based on compressed sensing to study the problem of image super-resolution reconstruction.

[0046] First, the present invention analyzes the commonly used reconstruction algorithms in the reconstruction algorithms under the traditional compressed sensing framework, starting from the commonly used OMP algorithm, and concludes that the OMP algorithm has low recovery rate and reconstruction efficiency in larger samples, especially for two-dimensional image signals, and in general, the original signal is real-time, which leads to uncertainty in the sparsity K of the original signal, and can only rely on previous rules to assume the value range of K, which affects the accuracy of image restoration. Under this premise, the GPSR algorithm is used to overcome the limitations of the fixed sparsity of the traditional OMP algorithm, and a more accurate sparse coefficient is obtained, thereby improving the accuracy of the image.

[0047] Secondly, in the link of image reconstruction, the original reconstruction algorithm uses interpolation to obtain the low-frequency information component of the image. The advantage of this method is that it is fast, but it will lose part of the image information, thus affecting the quality of image reconstruction. The present invention adopts bilateral filtering, which better retains part of the image information than the traditional method, and pays attention to both distance and pixel value at the same time, and uses weight equations to calculate the size of spatial distance and pixel value weight, thereby completing higher quality filtering and obtaining more accurate low-frequency information components, further improving the accuracy of the image, and having better accuracy than the traditional super-resolution reconstruction algorithm.

[0048] The super-resolution reconstruction algorithm based on compressed sensing of the present invention includes two parts: constructing an overcomplete dictionary and sparse representation of the image in the corresponding dictionary. It can realize the construction of a training data set, obtain feasible high-resolution and low-resolution images, and improve the quality of subsequent training. It can obtain an overcomplete dictionary through a dictionary learning algorithm as an important part of image reconstruction. It can obtain the sparse coefficient x′ by solving the l1 minimization problem, and compare x′ with the high-resolution dictionary D h Construct the reconstructed high frequency component HF.

[0049] The present invention combines the low frequency component LF and the high frequency component HF to output a high-resolution image.

[0050] in:

[0051] (1) In the construction of the training data set, choosing an appropriate solution to obtain a high-quality training data set is one of the key issues of the algorithm.

[0052] (2) In solving the overcomplete dictionary pair, a suitable method is used to obtain a high-quality redundant dictionary.

[0053] (3) In solving the sparse coefficient, the solution to the l1 minimization problem also affects the quality of the output image. The obtained sparse coefficient x′ will directly participate in the reconstruction of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of the method of the present invention;

[0055] Figure 2 The present invention relates to a compressed sensing signal sampling process;

[0056] Figure 3 It is a theoretical framework diagram of compressed sensing involved in the present invention;

[0057] Figure 4 It is a schematic diagram of the K-SVD algorithm involved in the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] The traditional image super-resolution reconstruction method is:

[0060] Step 1: Compressed sensing starts from global information, performs information sampling and information transmission operations at the same time. In the known prior knowledge of the original sparse signal, the redundancy in the original signal is used to perform sparse sampling and linear measurement on the original signal, and a small number of observations are used to replace most of the redundant information in the original signal.

[0061] Among them, the sparse representation of the signal requires that the original signal is compressible or can be sparsely represented in a certain transformation domain, which is the basic condition of the compressed sensing theory for signal acquisition;

[0062] Linear measurement of signals refers to pre-constructing a suitable measurement matrix, which is used to linearly project the original signal from a high-dimensional space to a low-dimensional space. This requires that the measurement matrix and the sparse transformation basis of the signal satisfy incoherence. After sparse transformation and linear sampling, the signal will produce a measurement value with a smaller dimension. The measurement value is transmitted to the receiving sensor. In order to obtain the original signal information, a small number of measurement values ​​need to be restored and reconstructed.

[0063] The process of sparsely sampling the original signal by utilizing the redundancy in the original signal is specifically as follows:

[0064] Compressed sensing is a process of sampling and transmitting data based on global information, integrating data sampling and data transmission into one step. Based on the prior knowledge of the original sparse signal, the original signal is sparsely sampled and linearly measured by utilizing the redundancy in the original signal, replacing most of the redundant information in the original signal with a small number of observations. The original signal is restored and reconstructed at the receiving end by solving the under-sampling equations. This method of projecting value information from high-dimensional space to low-dimensional space through sparsely sampled signals greatly reduces the requirements for hardware facilities during the sampling process and improves the utilization rate of sampling resources. Natural signals have varying degrees of redundancy due to their local continuity, smoothness, and regularity. In order to utilize this redundancy, we can set a sparse basis ψ so that it has a sparse expression x′ in the transform domain. A one-dimensional signal x with a length of N and sparseness in a certain transform domain N , x N YesRN The N×1-dimensional column vector in the space, any vector in the space R to which the element x{n}, n=1,2,3,...,N belongs can be represented by the basis vector The combined linear representation of .

[0065]

[0066] The vector x′ is x N In ψ M×N The linear representation in ψ is x′ in M×N The domain is compressible if ψ M×N Basis vectors in the field Represents a one-dimensional signal x N , vector x′ has K elements with relatively large weight values. Set a threshold, and regard the elements in x′ whose absolute values ​​are lower than the threshold as zero elements. Then vector x′ has only K non-zero elements, where K is much smaller than the length of the original signal. The number of non-zero elements is called the sparsity of vector x′, which also indicates that vector x′ has K-order sparsity.

[0067] Compared with one-dimensional signals, the processing of two-dimensional images is relatively complicated, especially in the sparse transformation of the sparse basis ψ. We can represent the image information contained in a two-dimensional signal as X M×N , where n×n=N. If the sparse basis ψ is an orthogonal matrix, the sparse transformation of a two-dimensional image can be described as follows:

[0068] X n×n =ψ T X′ψ

[0069] Compared with the Nyquist sampling method, when compressed sensing transmits the original signal after acquisition, it does not directly transmit the original signal, but the projection value after sparse transformation from high dimension to low dimension, that is, the sparse coefficient x′ (obviously, x′ and x′ are different representations of the same signal in different spatial domains). Therefore, after the original signal is sparsely transformed, whether it is stored or transmitted, the storage cost of hardware facilities is greatly reduced and resource utilization is increased.

[0070] There are many methods for sparse representation of redundant signals, which can be summarized into two parts: the first part is orthogonal sparse transformation, which is the most commonly used sparse orthogonal basis; the second part is redundant dictionary sparse transformation, which requires a learning algorithm to train the dictionary. The trained dictionary can express the original signal most directly with the least atoms.

[0071] Orthogonal sparse transform is the most common and commonly used sparse transform. It roughly conforms to the estimation of row vectors and column vectors in the orthogonal transform part of linear transform. This transform technology has been continuously improved and perfected, and the theoretical knowledge has reached maturity. The transform speed and signal recovery speed are fast, and the stability is relatively good when restoring and reconstructing the signal. However, there are shortcomings in the broad good side. Because it is applicable to most signals, it cannot adaptively adjust a certain signal, and in general, it cannot directly represent the required signal in the sparsest way. There are three commonly used orthogonal sparse transforms.

[0072] Discrete Fourier Transform: The Fourier analysis method is the basic method first proposed for signal analysis. The core content of Fourier analysis is to perform Fourier transform on the original signal. The essence of the Fourier transform process is to transform the signal from the time domain to the frequency domain.

[0073] The two-dimensional discrete Fourier transform can be expressed as

[0074]

[0075] When using discrete Fourier orthogonal basis to sparsely represent two-dimensional image signals, most image signals can be sparsely represented. Because of the unique properties of discrete Fourier, it is more effective when sparsely representing commonly used waveform signals. The disadvantage is that the sparse transformation only targets stationary time domain signals, and it is difficult to extract effective information from discontinuous time domain signals.

[0076] Discrete cosine transform (DCT): Generally speaking, Fourier transform is a complex transform, which includes the imaginary part. However, when operating on one-dimensional signals or two-dimensional images, we only need the real part. Therefore, discrete cosine transform can also be used to perform sparse transformation on the signal, which can shorten the calculation time and reduce the amount of calculation. The two-dimensional discrete cosine transform is:

[0077]

[0078] in

[0079]

[0080] Discrete wavelet transform: In view of the deficiency of Fourier transform in the absence of time domain analysis for sudden changes or non-stationary signals, MORLET proposed wavelet transform. Instead of continuing the idea of ​​using window function, it directly changes the Fourier transform basis. This not only enables adaptive scaling, but also obtains frequency and locates the position in the time domain.

[0081] The two-dimensional wavelet transform formula is

[0082]

[0083] In the formula, It is a kind of wavelet basis with the characteristics of locality and normalization. Parameter a is the scale parameter, b is x , b y is called the time-shift parameter.

[0084] The core of discrete wavelet transform is the discretization of continuous wavelet transform. For one-dimensional signal, wavelet transform can be described in two parts: the overall part and the detail part. For two-dimensional image signal, wavelet transform can transform the image into a series of wavelet coefficients, which can be compressed and stored efficiently, and eliminate the block effect that is common in discrete cosine transform compression.

[0085] If a dictionary's vector basis can support the entire interval or vector space, that is, the vectors of the signal space can be completely represented by the atoms in the dictionary, then the dictionary is said to be complete. For a complete dictionary, if the dimension of its atoms is much smaller than the number of columns of the matrix, it is called a redundant dictionary, also known as an over-complete dictionary.

[0086] For the original signal x N , can be approximated by the linear sum of K (K<<N) atoms. Unlike the orthogonal basis, the atoms in the redundant dictionary are non-orthogonal and can be represented by the remaining atoms. It is this redundancy that can make the signal more sparse when expressed in sparse transformation. For most signals, this redundancy can be used to train the dictionary through a learning algorithm so that the atoms in the dictionary find a sparser expression relative to the signal. The methods for constructing redundant dictionaries are generally divided into two categories: one is to use an orthogonal transformation basis to construct a redundant dictionary, such as a discrete cosine dictionary and a wavelet dictionary. The other is to use a learning algorithm to train atoms, such as the K-SVD algorithm.

[0087] In general, dictionary learning can be achieved in two steps: sparse coding and updating the dictionary set. Given the original signal X N×T , T is the number of signals, find the redundant dictionary Ψ that makes the training set the sparsest N×L and the corresponding sparse coefficient x′ L×T , L represents the length of the sparse coefficient. The optimal solution of the sparse coefficient to restore the original signal can be expressed as:

[0088] Compared with the orthogonal sparse matrix, the optimized dictionary in the training set given by the algorithm training does not need to be calculated in the inverse matrix during the iteration process, which improves the convergence speed. The obtained dictionary can adaptively sparsely represent the sparse coefficients of the image and is more targeted to specific image features.

[0089] The process of linearly measuring the original signal by utilizing the redundancy in the original signal is specifically as follows:

[0090] The original signal x of length N N By setting the measurement matrix Φ M×N Linear projection into low-dimensional space can obtain the M (M < < N)-dimensional measurement value y M , the process is as follows Figure 2 As shown:

[0091] By measuring the vector y M It can be seen from the dimension that using the measurement matrix Φ M×N For the original signal x N After M linear measurements, if the measurement matrix Φ M×N If each row vector of is regarded as an atom (basis), then the essence of linear measurement is to use a specific basis to perceive these original signals, and M measurement values ​​can be obtained. These measurement values ​​are essentially the linear superposition of the original signals on these atoms.

[0092] y=ΦΨx′=Ax′ (2-8)

[0093] The matrix A = ΦΨ, which is called the perception matrix. The sparse basis size is N×N, and the measurement matrix size is M×N. Therefore, it is deduced that the size of the perception matrix A is M×N. At this time, there are K non-zero vectors in x′, and for all K (K<<N), there is an isometric constant δ K ∈(0,1), such that

[0094]

[0095] If the above equation holds, then A is said to satisfy the K-order restricted isometry property (RIP), that is, A∈RIP(K,δ K ) is subject to the parameter δ K The essence of the RIP property is to ensure the isometry invariance between elements, so that the measurement matrix Φ and the sparse matrix ψ are independent of each other. Only then can we say that there is an N×1-dimensional vector X, and the M×1-dimensional vector y is the linear projection of vector X on the perception matrix A. The equivalent representation x′ of vector X in the ψ domain is restored from vector y using M (M>K×LOG(N / K)) measurement values, and then the original signal is restored by inverse transformation.

[0096] The measurement matrix is ​​the bridge that connects the sparse sampling and reconstruction process of the signal. Common measurement matrices can be divided into two types: random measurement matrices and deterministic measurement matrices. The random measurement matrix is ​​the first measurement matrix used in compressed sensing theory. There are many common random measurement matrices, including: Gaussian random measurement matrix, Bernoulli random measurement matrix, partial Hadamard (PH) matrix, etc. The random measurement matrix has great difficulties in hardware storage due to its inherent uncertainty, but the advantages are also obvious. The use of random measurement matrices only needs to meet the RIP criterion and is not limited by conditions such as sampling rate; the deterministic measurement matrix is ​​a type of measurement matrix specifically designed according to the characteristics of the target being measured, such as polynomial deterministic matrix, etc.

[0097] Deterministic matrices generally present the distribution of atoms according to specific rules, are supported by specific functions, and can calculate the specific position of each atom in the matrix. When using deterministic measurement matrices for linear sampling, they generally require a higher sampling rate than random measurement matrices. Their application range is not as wide as that of random measurement matrices, but they are easier to implement in hardware facilities because they are specifically designed, which greatly saves storage space. This section briefly introduces Gaussian random measurement matrices, Bernoulli random measurement matrices, partial Hadamard (PH) matrices, and polynomial deterministic matrices as examples:

[0098] Gaussian random measurement matrix: Gaussian random measurement matrix is ​​the most commonly used random measurement matrix and the earliest measurement matrix used in compressed sensing. The elements in the matrix are independent of each other and satisfy the Gaussian distribution with a mean of zero and a variance of 1 / N:

[0099]

[0100] The elements in the Gaussian random measurement matrix are highly non-fixed, and it is uncorrelated with most sparse matrices, which means that Gaussian is suitable for most sparse signals.

[0101] Bernoulli random matrix: Bernoulli random measurement matrix is ​​also widely used in compressed sensing. The elements of the matrix obey the random Bernoulli distribution. The specific construction is as follows:

[0102]

[0103] Partial Hadamard Matrix: The partial Hadamard measurement matrix is ​​constructed based on the Hadamard array. The characteristic of the Hadamard array is that the size must be a square matrix of N×N, and N=2 K, K is a natural number, so M rows can be randomly selected as the M×N measurement matrix. The relationship between the 2U-order (high-order) Hadamard matrix and the U-order Hadamard matrix is ​​shown below

[0104]

[0105] When the Hadamard matrix is ​​of the lowest order, that is, U = 2

[0106]

[0107] The elements in the Hadamard matrix are +1 or -1, which is binary, and any two rows or columns of atoms are orthogonal, so it is independent of correlation compared to most sparse matrices.

[0108] Polynomial measurement matrix: The polynomial measurement matrix generates each atom in the matrix through a polynomial function. In the finite field F, we can calculate the element value of each position of the coordinate in the polynomial measurement matrix, but this also fixes the sampling rate, requiring the sampling data M = p 2 , p is the number of elements in the polynomial set, assuming a finite field F with p elements, for any given natural number r, 1<r<p with p r represents the set of polynomials whose highest power is less than or equal to r, Q(x) = a0+a1x+…+a r x r ,Q(x)∈P r ,a0,a1,...,a r ∈F, its construction principle is as follows:

[0109] Define a matrix E of size p×p, and record the positions of E as (0,0), (0.1), ... (p-1, p-1) where (p-1, p-1) represents the position coordinates of the p-1 row and p-1 column of the matrix, as shown below:

[0110]

[0111] Operate on the matrix E, set the value of the Q(x)th element in the xth column to 1, and set the remaining elements to 0.

[0112]

[0113] Among them, the polynomial Q(x) formula is

[0114] Q(x)=a0+a1x+…+a r x r

[0115] Convert the matrix E into a column vector V of size M×1 Q

[0116] V Q =(V0,V1,...,V p-1 ) T ,V I = column vector of the i-th column of matrix E, 0≤i≤p-1

[0117] M=p 2 , there are N vectors V Q , at this time, the deterministic polynomial measurement matrix Φ can be represented by a matrix composed of N column vectors Q :

[0118] Φ Q =(V Q1 ,V Q2 ,...,V QV ) T .

[0119] Step 2: At the receiving end, the original signal is restored and reconstructed by solving the under-sampling equations. This method of projecting value information from high-dimensional space to low-dimensional space by sparsely sampling signals greatly reduces the requirements for hardware facilities during the sampling process and reduces unnecessary consumption of resources. The theoretical framework of compressed sensing is shown in the figure below: Figure 3 shown; by the framework Figure 3 It can be seen that the entire data collection process in compressed sensing can be summarized into three parts: sparse representation of signals, linear measurement of signals, and accurate reconstruction of signals. Among them, the process of reconstructing the original signal is the process of solving underdetermined equations. Therefore, the specific form of the reconstruction process and the good choice of the reconstruction algorithm determine the reconstruction quality and efficiency of the original signal.

[0120] The process of restoring and reconstructing the original signal by solving the undersampling equations at the receiving end is specifically:

[0121] Signal recovery is a very important step in the compressed sensing theory framework. The quality of the reconstruction algorithm is directly related to the quality of imaging and the efficiency of reconstruction. The key issue of compressed sensing theory is to use the measurement vector y to accurately reconstruct the original sparse signal. That is, solving the problem of y=Ax′ can generally be solved by solving the l0 optimization problem:

[0122]

[0123] sty=Ax′

[0124] Where x′ is an N-dimensional vector, y is an M-dimensional vector, N>>M is an underdetermined equation, so the recovery process belongs to the NP problem. In the regularization method, the l0 norm solution result is less stable, so the l1 norm optimization is used to find the approximate solution.

[0125]

[0126] st y=Ax′

[0127] This is transformed into solving a convex optimization problem, and the signal to be collected can be obtained by inversely transforming the approximate solution. The core of the reconstruction algorithm is to complete the process of solving the original high-dimensional signal from the collected low-dimensional measurement data. According to different solution methods, common reconstruction algorithms are divided into three categories: greedy algorithm, convex relaxation method and minimum total variation method:

[0128] Greedy algorithm: The core of the greedy algorithm is the matching pursuit method, which is the most effective method for solving problems in the compressed sensing reconstruction algorithm. The basic idea is to match the column atoms in the sensing matrix with the sparse signal, find the nearest column atom for the next iteration, and continuously perform sparse approximation to obtain the residual value during iteration. When the set error value is reached, the iteration is stopped. The whole process is to ensure the optimality of the iteration by orthogonalizing the selected atom set. It mainly includes the Match Pursuit (MP) algorithm, the Orthogonal Match Pursuit (OMP) algorithm and the Sparsity Adaptive Matching Pursuit (SAMP) algorithm.

[0129] Minimum Total Variation Method - TVL3 Algorithm: In order to overcome the disadvantage of the greedy algorithm that the two-dimensional image is slow to recover, the minimum total variation differential method is proposed, which uses the TV criterion to perform image recovery. l1l2 The core formula is as follows:

[0130]

[0131] D h , D v Represents the gradient operation factor in the vertical and horizontal directions, and I represents the entire image value. In order to improve the reconstruction speed and enhance the reconstruction effect, Maleh et al. proposed the matrix inverse operation factor Ω, which is defined as

[0132] ΩI=||D h ||1+||D v I||1

[0133] TV′ l1 It is applied to the continuity of gradient intensity in vertical and horizontal directions, but it cannot be associated with the continuity of gradient in the diagonal direction, and the correlation between some pixel blocks is lost. In order to better utilize the local continuity and local smoothness of the two-dimensional image itself, make up for the TV l1l2 , TV′ l1There are some deficiencies between them. The augmented Lagrangian function is introduced into the total variation method. According to the gradient correlation, the TVAL3 algorithm based on the minimum total variation method is proposed, which improves the reconstruction speed and enhances the reconstruction effect. The reconstruction formula is:

[0134]

[0135] D i x represents the horizontal and vertical discrete gradient calculation of x at pixel point i. If I(i,j) represents the position of the pixel point, then the horizontal and vertical gradients of the pixel block can be expressed as

[0136]

[0137] Solve the following equation to get the measured value of x, adding slack variables as conditions

[0138]

[0139] The Lagrangian enhanced function corresponding to the following formula is

[0140]

[0141] Solve the following equation to convert it into the optimization of solving the Lagrangian function value, that is

[0142]

[0143] The above formula is to solve the minimum value problem of convex function, but because of its non-differentiable and nonlinear characteristics, it is difficult to solve. i Calculate and solve separately.

[0144] The method of the present invention also includes a super-resolution reconstruction step based on compressed sensing, which specifically includes:

[0145] Specifically, the super-resolution reconstructed image (SRI) consists of two parts, namely low frequency (LF) and high frequency (HF), and their relationship can be expressed as follows:

[0146] SRI=LF+HF

[0147] Where LF is the interpolation image of the input low-resolution original image. In this paper, bicubic interpolation is proposed. HF is obtained by a framework based on compressed sensing. The method used by the framework is as follows:

[0148] Collect multiple high-resolution images, and obtain their corresponding low-resolution images through scaling operations to form the training data set of the algorithm. Use the dictionary learning method to construct the relationship between images and form an over-complete dictionary. The high-frequency component can be expressed as follows:

[0149] HF=ψx′

[0150] In the formula, x′ represents the weight coefficient of the high-resolution dictionary, i.e., the sparse representation coefficient. In the compressed sensing framework, x′ directly represents the quality of the high-frequency component, so it is the focus of research. Specific implementation method one:

[0152] This embodiment is a method for image super-resolution reconstruction based on compressed sensing, such as Figure 1 As shown, the method is implemented by the following steps:

[0153] Step 1: Design a dictionary learning process to obtain the high-frequency component parameters required in image restoration; including constructing a training data set and performing dictionary training; wherein, the image is degraded and scaled down to obtain the corresponding low-resolution image for the training data set process; an overcomplete dictionary pair is constructed using the K-SVD algorithm to train the dictionary;

[0154] Step 2: reconstructing the image, including: first obtaining the restored low-frequency components, and then obtaining a high-resolution image. Specific implementation method 2:

[0156] Different from the first specific implementation, in the method for image super-resolution reconstruction based on compressed sensing in this implementation, the process of constructing an overcomplete dictionary pair using the K-SVD algorithm and training the dictionary described in step 1 is specifically as follows:

[0157] Step 1: Through the given original sample Y training and learning, construct the redundant dictionary D and sparse matrix X of the sparse representation sample; each column of the original sample Y represents a sample, the dictionary D stores the features of Y, and the sparse matrix X represents Y through the linear combination of D, as shown in the following formula:

[0158] Y m×n =D m×s ·X s×n

[0159] When s is much smaller than m and n, the dimensions of D and Y will be much smaller than the original Y; this can be applied to data dimensionality reduction. The K-SVD algorithm is a commonly used dictionary learning method. It draws on the ideas of K-means clustering and SVD algorithm, and constructs a dictionary for sparse representation through singular value decomposition (SVD). Under the condition that both D and X are unknown, it solves the optimization problem shown in the following formula, namely:

[0160]

[0161] Where Y = [y1, ..., y n ] is the input sample matrix, each column of which represents an input sample, X = [x1, ..., x m ] is a sparse representation matrix, x i for y i The corresponding sparse representation coefficients are, represents the square of the Frobenius norm of the matrix Y-DX, T is the parameter for constraining the sparsity of the coefficients; if you directly restrict ||x i ||0=1, then the problem shown in the above formula is essentially K-means clustering. The K-SVD algorithm is a generalization of the K-means clustering method. Its solution is to update the dictionary D and the sparse representation coefficient x of the corresponding column column by column through a two-step iterative process. i , stop when the convergence condition is met or the maximum number of iterations is reached, and finally obtain a redundant dictionary D and sparse representation matrix X that can accurately express the sample characteristics. The specific steps are as follows:

[0162] Initialization: Randomly select s samples from sample Y to initialize dictionary D. Each column in D is a sample feature vector, and perform L2 normalization on the feature vector;

[0163] Sparse coding: Fixed dictionary D, using OMP algorithm to solve the sparse representation coefficients of all samples, for each sample y i and its corresponding sparse representation coefficient x i have:

[0164]

[0165] Update dictionary: fix the sparse matrix X, update each column of atoms in the dictionary D and update the corresponding row of X; take the kth atom d k For example, the objective function can be rewritten as:

[0166]

[0167] in, is the error matrix. The solution of the above objective function can be transformed into adjusting d k and Multiply it by E k The error is as small as possible; for this problem, if we directly k , SVD decomposition will result in Not sparse, so just choose E corresponding to the non-zero terms in k Participate in the calculation and define the set of non-zero items:

[0168]

[0169] E k By Omega k The limitation of E k ′, the process diagram is as follows Figure 4 As shown. k ′ Perform SVD decomposition to obtain E k ′=UΔV T , then the solution d k It is given by the first column of U and the product of the first column of V and Δ(1,1) Update; finally repeat the sparse coding and dictionary update steps until the preset number of iterations or convergence condition is reached;

[0170] Step 1 and 2: Improve the sparse coefficient;

[0171] In the reconstruction framework of compressed sensing theory, the orthogonal matching pursuit algorithm (OMP) is used to solve the sparse representation coefficients; the orthogonal matching pursuit algorithm is to approximate the original signal by selecting the most relevant atoms (basis vectors). The core idea of ​​OMP is to gradually establish a sparse representation by selecting the atom with the highest correlation with the current residual in each step. Specifically, OMP includes: first, initialize a sparse coefficient vector to zero, then select an atom with the highest correlation with the current residual in each step, and update the sparse coefficient; this process is iterated until the preset sparsity is reached or the convergence condition is met. The advantages of the OMP algorithm are high computational efficiency, simple implementation, and good performance when processing high-dimensional data. However, a fixed sparsity is required when using OMP, which is not the optimal result under actual conditions. Therefore, the present invention adopts the gradient projection sparse reconstruction (GPSR) algorithm to solve the sparse representation coefficients, which can provide higher accuracy and convergence speed for large-scale problems. Specific implementation method three:

[0173] Different from the second specific implementation mode, in the method for image super-resolution reconstruction based on compressed sensing in this implementation mode, the process of image reconstruction described in step 2 is specifically as follows:

[0174] First, after completing the training of the training set, the reconstruction framework of the present invention can perform image reconstruction. For image reconstruction, the input low-resolution image is interpolated to obtain the enlarged pixel size;

[0175] Secondly, bilateral filtering is used to extract low-frequency information, and then the parameters obtained by dictionary learning and the input image are used to calculate the high-frequency information component;

[0176] Finally, the corresponding high-resolution image is reconstructed through synthesis and superposition;

[0177] Among them, bilateral filtering is a filtering operation. Bilateral filtering is a low-pass filtering, which is proposed on the premise of Gaussian filtering. Bilateral filtering adds pixel value weighting items on the basis of Gaussian filtering, that is, it takes into account both the distance factor and the influence of the pixel value difference. The closer the pixel value, the greater the weight. The pixel value weight is represented as G r , the spatial distance weight is represented by G s ,have:

[0178]

[0179] Then the entire filter can be expressed as BF, and the filtering result is:

[0180]

[0181] Where W q is the weighted sum of each pixel value in the filter window, which is used for weight normalization;

[0182]

[0183] In the flat area, the G of each pixel in the filter r The value is similar, the spatial distance weight G s Dominant filtering effect; in the edge area, G on the same side of the edge r The value is similar and much larger than G on the other side of the edge. r value, at this time, the weight of the pixel on the other side has almost no effect on the filtering result, the edge information is protected, and a certain degree of adaptability is shown;

[0184] When noise points appear in a flat area, the weights of the signals around the noise points are very small. After normalization, these weights are improved, so the noise points are also filtered out.

[0185] Experimental and result evaluation indicators of the method of the present invention

[0186] The reconstruction speed and reconstruction quality of various image reconstruction algorithms are compared by parallel comparison. The reconstruction speed uses the time it takes for each algorithm to reconstruct the image as the comparison basis, in seconds, and the reconstruction quality uses PSNR and MSE to reflect the reconstruction quality. Their calculation formulas are as follows:

[0187]

[0188] Among them, Y i Refers to the original data, Represents the reconstructed data. PSNR represents the quality of reconstruction, and MSE represents the error size of reconstruction.

[0189] The embodiments of the present invention disclose preferred embodiments, but are not limited thereto. A person skilled in the art can easily understand the spirit of the present invention based on the above embodiments and make different extensions and changes. However, as long as they do not deviate from the spirit of the present invention, they are all within the protection scope of the present invention.

Claims

1. A method for image super-resolution reconstruction based on compressed sensing, characterized in that: The method is implemented by the following steps: Step 1: Design the dictionary learning process to obtain the high-frequency component parameters required for image restoration; The method includes constructing a training data set and performing dictionary training; wherein, the image is degraded and scaled down to obtain a corresponding low-resolution image for the training data set; an overcomplete dictionary pair is constructed using the K-SVD algorithm to train the dictionary; Step 2: reconstructing the image, including: first obtaining the restored low-frequency components, and then obtaining a high-resolution image.

2. The method for image super-resolution reconstruction based on compressed sensing according to claim 1, characterized in that: The process of constructing an overcomplete dictionary pair using the K-SVD algorithm and training the dictionary described in step 1 is specifically as follows: Step 1: Through the given original sample Y training and learning, construct the redundant dictionary D and sparse matrix X of the sparse representation sample; each column of the original sample Y represents a sample, the dictionary D stores the features of Y, and the sparse matrix X represents Y through the linear combination of D, as shown in the following formula: Y m×n =D m×s ·X s×n When s is much smaller than m and n, the dimensions of D and Y will be much smaller than the original Y. The K-SVD algorithm is a commonly used dictionary learning method. It draws on the ideas of K-means clustering and SVD algorithm, and constructs a dictionary for sparse representation through singular value decomposition (SVD). Under the condition that both D and X are unknown, it solves the optimization problem shown in the following formula, namely: Where Y = [y1, ..., y n ] is the input sample matrix, each column of which represents an input sample, X = [x1, ..., x m ] is a sparse representation matrix, x i for y i The corresponding sparse representation coefficients are, represents the square of the Frobenius norm of the matrix Y-DX, T is the parameter for constraining the sparsity of the coefficients; if you directly restrict ||x i ||0=1, then the above formula is K-means clustering, which updates the dictionary D and the sparse representation coefficient x of the corresponding column column by column through a two-step iterative process i , stop when the convergence condition is met or the maximum number of iterations is reached, and finally obtain a redundant dictionary D and sparse representation matrix X that can accurately express the sample characteristics. The specific steps are as follows: Initialization: Randomly select s samples from sample Y to initialize dictionary D. Each column in D is a sample feature vector, and perform L2 normalization on the feature vector; Sparse coding: Fixed dictionary D, using OMP algorithm to solve the sparse representation coefficients of all samples, for each sample y i and its corresponding sparse representation coefficient x i have: Update dictionary: fix the sparse matrix X, update each column of atoms in the dictionary D and update the corresponding row of X; take the kth atom d k For example, the objective function can be rewritten as: in, is the error matrix. The solution of the above objective function can be transformed into adjusting d k and Multiply it by E k The error is as small as possible; choose E corresponding to the non-zero terms in k Participate in the calculation and define the set of non-zero items: E k By Omega k The limitation of E k ′, for E k ′ Perform SVD decomposition to obtain E k ′=UΔV T , then the solution d k It is given by the first column of U and the product of the first column of V and Δ(1,1) Update; finally repeat the sparse coding and dictionary update steps until the preset number of iterations or convergence condition is reached; Step 1 and 2: Improve the sparse coefficient; In the compressed sensing theory reconstruction framework, the gradient projection sparse reconstruction algorithm is used to solve the sparse representation coefficients.

3. The method for image super-resolution reconstruction based on compressed sensing according to claim 2, characterized in that: The process of image reconstruction described in step 2 is specifically as follows: First, after completing the training set, the input low-resolution image is interpolated to obtain the enlarged pixel size; Secondly, bilateral filtering is used to extract low-frequency information, and then the parameters obtained by dictionary learning and the input image are used to calculate the high-frequency information component; Finally, the corresponding high-resolution image is reconstructed through synthesis and superposition; Among them, bilateral filtering performs filtering operation and represents the pixel value weight as G r , the spatial distance weight is represented by G s ,have: Then the entire filter can be expressed as BF, and the filtering result is: Where W q is the weighted sum of each pixel value in the filter window, which is used for weight normalization; In the flat area, the G of each pixel in the filter r The value is similar, the spatial distance weight G s Dominant filtering effect; in the edge area, G on the same side of the edge r The value is similar and much larger than G on the other side of the edge. r value; When noise points appear in a flat area, the weights of the signals around the noise points are very small and are normalized.

Images