Two-way spatial-spectral domain joint compressed sensing imaging method and device

Abstract

The invention discloses a double-path spatial-spectral domain joint compressed sensing imaging method and device, and belongs to the field of high-resolution spectral imaging and compressed imaging. Target light is divided into two paths through the beam splitter prism, and one path is imaged to the high-spatial-resolution detector through the imaging lens. And the other path enters a spatial spectrum compression measurement light path. A spatial-spectral joint coding matrix is generated for a spatial-spectral compression measurement light path, a DMD is introduced into the light path to serve as a spatial light modulator to load the joint coding matrix to carry out high-resolution spatial-spectral joint coding on a target scene, and then a prism is used as a dispersion element to carry out dispersion on the coding scene. And finally, carrying out space and spectrum down-sampling measurement on the coded information of dispersion through an imaging lens by using a low-spatial-resolution detector to obtain a low-spatial-resolution two-dimensional spectrum aliasing measurement value. An original high-space and high-spectral-resolution image can be reconstructed through a back-end reconstruction algorithm on the collected high-spatial-resolution gray-scale image and the low-spatial-resolution spectral aliasing image.

Description

Technical Field

[0001] The present invention relates to a method for high-resolution spectral imaging, and in particular to a dual-path space-spectral domain combined compressed sensing image reconstruction method, belonging to the field of high-resolution spectral imaging and compressed imaging. Background Art

[0002] Spectral imaging technology is renowned for its powerful information acquisition capabilities and exceptional spectral resolution. It can acquire spectral information about a target object at different wavelengths, enabling precise analysis of its composition, structure, and state. In recent years, with the continuous advancement of sensor technology, image processing algorithms, and data analysis methods, spectral imaging technology has rapidly developed and gained widespread application. Traditional spectral imaging systems acquire spectral information about a target object through spectroscopic analysis. This spectroscopic process causes the incident light energy to disperse across the detector surface, affecting the detector's response sensitivity and the signal-to-noise ratio of the spectral image. Therefore, high-quality spectral imaging requires the use of detector arrays with larger pixel areas. However, this inevitably leads to a decrease in spatial resolution. Consequently, the spatial and spectral resolutions of traditional spectral imaging systems are often mutually constrained. Furthermore, because spectral imaging systems must capture information about the target scene across multiple spectral bands, the amount of data generated is far greater than with traditional imaging systems. This significantly increases the pressure on data readout, transmission, and storage, while also placing higher demands on the computing power and storage capacity of back-end information processing equipment. Therefore, more efficient spectral imaging systems with high spatial and spectral resolution are crucial for the development of spectral imaging.

[0003] In recent years, compressive sensing technology has attracted extensive attention due to its excellent performance in the field of signal processing. Based on the sparsity of the target signal, compressive sensing technology compresses and collects the original signal at a sampling frequency lower than the Nyquist sampling theorem, and combines with the back-end reconstruction algorithm to realize the reconstruction of the original signal. The compression measurement of the target signal not only reduces the hardware performance requirements of the signal acquisition system, but also significantly reduces the amount of data acquisition, the pressure of system data transmission and back-end data processing. Spectral data has redundancy. Compressive spectral imaging technology combines compressive sensing with spectral imaging, and through the introduction of encoding of the spectral dimension of the target scene, only a single frame of spectral aliasing compression measurement value is needed to realize spectral imaging combined with the back-end reconstruction algorithm. Compressive spectral imaging acquires spectral aliasing measurement values, avoids the problem of insufficient detector response caused by spectral channel acquisition in traditional spectral imaging systems, and reduces the amount of data acquisition, which is of great significance to alleviate the constraints of spatial-spectral resolution and data processing pressure of spectral imaging systems. Coded aperture snapshot spectral imaging (CASSI) is a typical compressive spectral imaging system. CASSI encodes the target scene through a high-resolution coded aperture, and the coded information is dispersed by a dispersion element and imaged onto the detector surface. The information collected by the detector is a coded spectral aliasing image, and the spectral reconstruction is realized through the back-end reconstruction algorithm. In order to improve the accuracy of spectral reconstruction, Wang et al. proposed to increase a non-coded and non-dispersed gray image acquisition path based on CASSI, and to realize high-quality spectral imaging based on the gray image and the coded spectral compression image.

[0004] Although the current research on spectral compressive imaging has attracted widespread attention, the current research is more focused on simple spectral domain compression, and the measurement data is mostly single compression measurement value. Therefore, we propose a dual-channel spatial-spectral domain joint compressive sensing imaging method. Based on the theory of compressive sensing, a high-resolution spatial light modulation device, digital micromirror device (DMD), is introduced into the imaging system to realize spatial-spectral domain joint high-resolution coding of the target scene, and finally a single frame of low spatial resolution spectral aliasing measurement value is obtained. In addition, a high spatial resolution gray image acquisition path is added to the system as a supplement to the reconstruction of high compression ratio spatial-spectral domain compression data in the reconstruction process, and high spatial and high spectral resolution image reconstruction is realized. SUMMARY

[0005] In order to solve the problem of mutual limitation between spatial resolution and spectral resolution in existing spectral imaging methods, the purpose of the present invention is to provide a dual-path spatial-spectral domain joint compressed sensing imaging method and device, which divides the target light into two paths through a spectroscopic prism, one of which is imaged to a high spatial resolution detector through an imaging lens. The other path enters the spatial-spectral compression measurement optical path. For the spatial-spectral compression measurement optical path, a spatial-spectral joint coding matrix is ​​generated, and a DMD is introduced into the optical path as a spatial light modulation device to load the joint coding matrix to perform high-resolution spatial-spectral joint coding on the target scene. Then, a prism is used as a dispersion element to disperse the coded scene. Finally, the dispersed coded information is spatially and spectrally downsampled and measured using an imaging lens and a low spatial resolution detector to obtain a low-spatial-resolution two-dimensional spectral aliasing measurement value. The collected high-spatial-resolution grayscale image and the low-spatial-resolution spectral aliasing image can be reconstructed into the original high-spatial and high-spectral resolution image through the back-end reconstruction algorithm.

[0006] The purpose of the present invention is achieved through the following technical solutions.

[0007] The present invention discloses a dual-path spatial-spectral domain combined compressed sensing imaging method, comprising the following steps:

[0008] Step 1: After the target scene is split by the spectroscope, it enters the high spatial resolution imaging optical path HRI and the spatial spectrum coding compression measurement optical path SSCI. The spectral data entering the HRI optical path is recorded as X HR ∈R H×W×C The spectral data entering the SSCI optical path is recorded as X SSCI ∈R H×W×C Where R represents a set of real numbers, and the superscript H×W×C represents the data dimension. H and W represent the height and width of the image, respectively. C represents the number of spectral bands.

[0009] Step 2, X in step 1 HR The output through the HRI optical path is Y HR ∈R H×W , for Y HR and X HR All are vectorized to obtain vector Y HR .

[0010] Step 3: X in step 1 SSCI Imaging is onto DMD, which loads the coding template and uses it to align X SSCI Perform spatial modulation to obtain the spatial modulated image Y' SSCI .

[0011] Step 4: Y' obtained in step 3 SSCI Arriving at the dispersion prism, the dispersion prism is Y' SSCI Modulate the spectral dimension to obtain the spectral modulation image Y" SSCI. Further get Y' SSCI (i,j,k) and Y" SSCI (i,j',k).

[0012] Step 5, Y obtained in step 4" SSCI Imaging onto a low-resolution grayscale camera yields Y SSCI , achieving space-spectral domain compression. SSCI and X SSCI All are vectorized to obtain Y SSCI .

[0013] Step 6: Combine the HRI optical path with the SSCI optical path and convert the original target scene 3D spectral data X∈R H×W×C Vectorize and get Y.

[0014] Step 7: Reconstruct Y obtained in step 6 using a reconstruction method to reconstruct the original target scene three-dimensional spectral data X.

[0015] Furthermore, the high spatial resolution imaging optical path HRI in step 1 includes an imaging lens and a high-resolution grayscale camera.

[0016] The spatial spectrum coding compression measurement optical path SSCI described in step 1 includes an imaging lens, a DMD, a dispersion prism, a relay lens, and a low-resolution grayscale camera.

[0017] Furthermore, the specific implementation method of step 2 is: X in step 1 HR The image is directly formed on the high-resolution grayscale camera through the imaging lens. The response function value of the high-resolution grayscale camera to the kth band is r HR (k), the image Y output by the high-resolution grayscale camera is HR ∈R H×W It is expressed as follows:

[0018]

[0019] Among them, Y HR (i, j) represents the value of the pixel in the i-th row and j-th column on the high-resolution grayscale camera, X HR (i, j, k) represents the three-dimensional spectral data X HR The value of the pixel in the kth band, ith row, and jth column. HR and X HR All are vectorized and the vector-matrix form is:

[0020] Y HR =H HR X HR (2)

[0021] Among them, Y HR ∈RHW×1 With X HR ∈R HWC×1 Represents vectorized Y HR and X HR , H HR ∈R HW×HWC Spectral response matrix of the optical path for high spatial resolution imaging.

[0022] Furthermore, the specific implementation method of step three is:

[0023] Use the coding template to encode X in step 1 SSCI Perform spatial encoding, that is, load the encoding template onto the DMD and SSCI Perform spatial encoding, then X SSCI The spatial dimension information of X is changed, that is, SSCI The spatial modulation of the image Y' is obtained SSCI ∈R H×W×C .

[0024] Furthermore, in step 4, Y' SSCI (i,j,k) and Y" SSCI (i,j',k) is represented as follows,

[0025] Will Y' SSCI The modulation result of the kth band, ith row and jth column is recorded as Y' SSCI (i,j,k), Y' SSCI (i,j,k) is represented as follows:

[0026] Y' SSCI (i,j,k)=M(i,j)X SSCI (i,j,k),k=1,...,C (3)

[0027] Among them, M(i,j) represents the value of the encoding template M in the i-th row and j-th column, X SSCI (i,j,k) represents X SSCI The value of the pixel in the kth band, i-th row, and j-th column.

[0028] Then Y" SSCI ∈R H×(W+D(C))×C It is expressed as follows:

[0029] Y” SSCI (i,j',k)=Y' SSCI (i,j+D(k),k),k=1,...,C (4)

[0030] Among them, Y SSCI (i,j',k) represents Y" SSCIThe value of the pixel in the kth band, the ith row, and the j'th column is j'=j+D(k), where D(k) represents the dispersion function caused by the dispersion prism in the kth band.

[0031] Furthermore, the specific implementation method of step five is:

[0032] The response function value of the low-resolution grayscale camera to the kth band is r LR (k), the image resolution is h×w, and the spatial compression ratio is a(0 <a<1),则有:

[0033] h=H×a (5)

[0034] w=(W+D(C))×a (6)

[0035] Where D(C) is the dispersion function corresponding to band C. Spatial spectral domain compression is to perform spatial downsampling and spectral downsampling on the modulated image in a low-resolution grayscale camera to obtain a compressed image Y SSCI , change Y SSCI In the low-resolution grayscale camera coordinate system, the pixel value in the pth row and qth column is recorded as Y SSCI (p,q), then Y SSCI (p,q) is represented as follows:

[0036]

[0037] Among them, the relationship between i, j and p, q is:

[0038]

[0039] The value ranges of p and q are p = 1, 2, ..., h and q = 1, 2, ..., w, respectively. The colon sign indicates incrementing from the previous number in steps of 1 to the next number.

[0040] Y SSCI and X SSCI All are vectorized and the vector-matrix form is:

[0041] Y SSCI =H SSCI X SSCI (10)

[0042] Among them, Y SSCI ∈R hw×1 With X SSCI ∈R HWC×1 Represents the vectorized Y SSCI and X SSCI , H SSCI ∈R H(W+D(C))×HWC is the equivalent linear operator of the discrete summation in Eq. (7).

[0043] Furthermore, the specific implementation method of step six is:

[0044] The high spatial resolution imaging optical path is combined with the SSCI optical path, and the original target scene three-dimensional spectral data X∈R H ×W×C Vectorized, the model of the entire system is expressed as:

[0045] Y=HX (11)

[0046]

[0047] in, is the measurement value of the entire system, H is the measurement matrix of the entire system, r HR (k) is the response function value of the high-resolution camera in the kth band, and the encoding template M is The size of is divided into Ha×Wa sub-blocks, then is the row vectorized form of the mth row and nth column of the encoding template M after deformation, X∈R HWC×1 is the vectorized X.

[0048] Furthermore, the reconstruction method described in step seven is a deep learning method.

[0049] The present invention also discloses a dual-path spatial-spectral domain combined compressed sensing imaging device for implementing the dual-path spatial-spectral domain combined compressed sensing imaging method for a spectral image. The dual-path spatial-spectral domain combined compressed sensing imaging device comprises a collimating lens, a filter, a beam splitter, three imaging lenses, a high-resolution grayscale camera, a DMD, a relay lens, a dispersion prism, a low-resolution grayscale camera, and a computer. Light from a target is first split into two paths by the beam splitter: one path is reflected by the beam splitter and enters a high-spatial-resolution imaging optical path, while the other path passes through the beam splitter and enters a spatial-spectral coding compression measurement optical path. The light beam entering the high-spatial-resolution imaging optical path is directly imaged onto the high-resolution grayscale camera through the imaging lens, resulting in a high-spatial-resolution grayscale image. The light beam entering the spatial-spectral coding compression measurement is imaged onto the surface of the DMD micromirror array through the imaging lens. The DMD loads a preset coding template and spatially modulates the target light beam. The spatially modulated target light beam then passes through the relay lens to the dispersion prism. The dispersion prism disperses light of different wavelengths along the dispersion direction, spectrally modulating the target light beam. The spatial-spectral modulated target beam is then imaged by an imaging lens onto a low-resolution grayscale camera. Because it passes through only one dispersion element, the low-resolution grayscale camera produces a two-dimensional compressed image that blends the spatial and spectral dimensions. The resulting high-resolution grayscale image and spatial-spectral compressed image are uploaded to a computer, where a high-resolution spectral image is reconstructed using a reconstruction algorithm.

[0050] Furthermore, the function of the beam splitter is to split the target light beam into two paths, one path entering the high spatial resolution imaging light path, and the other path entering the spatial spectrum coding compression measurement light path.

[0051] The functions of the imaging lens are to image the target onto a high-resolution grayscale camera, to image the target light beam onto a digital micromirror array (DMD), and to image the coded target after spatial spectrum modulation onto a low-resolution grayscale camera.

[0052] The function of a high-resolution grayscale camera is to obtain grayscale images with high spatial resolution.

[0053] The function of DMD is to store the designed coding template and load the coding template to spatially modulate the target light beam imaged onto the DMD.

[0054] The function of the relay lens is to transmit the light beam reflected by the DMD to the dispersion prism.

[0055] The function of the dispersion prism is to disperse the light beam along the dispersion direction according to the wavelength and perform spectral modulation on the target light beam.

[0056] The function of the low-resolution grayscale camera is to receive the modulated light beam and obtain a compressed image.

[0057] The role of the computer is to run a dual-path spatial-spectral domain joint compressed sensing imaging recovery algorithm, and use the measurement matrix to reconstruct a high-resolution spectral image of the target scene from the high-resolution grayscale image and spatial-spectral image taken by the dual-path camera.

[0058] Beneficial effects:

[0059] 1. The present invention discloses a dual-path spatial-spectral domain combined compressed sensing imaging method and device, which breaks through the limitations of camera spatial resolution and spectral resolution through spatial domain compression imaging and spectral domain compression imaging, so that high-resolution spectral images can be obtained using a beam splitter, two grayscale cameras, DMD, dispersion prism, imaging optical path and control equipment.

[0060] 2. The present invention discloses a dual-path spatial-spectral domain joint compressed sensing imaging method and device, which utilizes the spatial and spectral correlation of spectral images and adds high-resolution grayscale images as prior information. By applying the compressed imaging method, the data is compressed before data collection, so that only a small amount of data needs to be collected to complete the acquisition of high-resolution spectral images, significantly reducing the amount of data collected, stored, and transmitted, and improving the efficiency of spatial-spectral domain joint compressed sensing. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 Schematic diagram of the principle of dual-camera spatial-spectral domain joint compressed sensing imaging;

[0062] Figure 2 is a device structure diagram of the system;

[0063] Figure 3 Figure 1 is a schematic diagram of the image modulation and acquisition process of the dual-path hyperspectral domain joint compressive sensing imaging;

[0064] Figure 4 Figure 2 is an encoding template used in the hollow spectral encoding compressive measurement optical path in the embodiment;

[0065] Figure 5 Figure 3 is a schematic diagram of the encoding template deformation process in the embodiment;

[0066] Figure 6 Figure 4 is a low-resolution spectral aliasing image and a corresponding high-resolution grayscale image used by the algorithm in the embodiment;

[0067] Figure 7 Figure 5 is a reconstructed spectral image in the embodiment;

[0068] Figure 1 is a schematic diagram of the image modulation and acquisition process of the dual-path hyperspectral domain joint compressive sensing imaging, wherein 1 is a target or scene, 2 is a collimating lens, 3 is a filter, 4 is a beam splitter, 5 is an imaging lens, 6 is a digital micromirror array (DMD), 7 is a relay lens, 8 is a dispersive prism, 9 is an imaging lens, 10 is a low-resolution grayscale camera, 11 is an imaging lens, 12 is a high-resolution grayscale camera, and 13 is a computer. DETAILED DESCRIPTION

[0069] In order to better illustrate the purposes and advantages of the present application, the content of the application is further described below in combination with the drawings and examples.

[0070] A 17-band image with a spatial resolution of 256x256, a pixel depth of 8 bits, and a spectral resolution of 10 nm at a wavelength of 540-700 nm is modulated using a DMD and a dispersive prism to obtain a compressed image with a resolution of 64x80 after downsampling. Finally, a 17-band reconstructed image with a spatial resolution of 256x256 is reconstructed from a high-resolution grayscale image with a resolution of 256x256 and a compressed image with a resolution of 64x80 using a reconstruction algorithm, thereby realizing a high-resolution spectral image based on a high-resolution grayscale image and a low-resolution compressed image. The spatial resolution of the reconstructed image is improved by 16 times (4x4) and the spectral resolution is improved by 17 times relative to the hyperspectral joint compressive measurement image.

[0071] The compressive imaging step in the embodiment is shown in Figure 1 The hyperspectral modulation and image compression process is shown in Figure 3 The dual-path hyperspectral domain joint compressive sensing imaging method disclosed in the embodiment is implemented as follows:

[0072] Step 1: The target scene passes through the collimator and filter to the spectrometer, and then is reflected by the spectrometer into the high spatial resolution imaging optical path, which includes the imaging lens and high-resolution grayscale camera. It then passes through the spectrometer into the spatial spectrum coding compression measurement optical path (SSCI), which includes the imaging lens, DMD, dispersion prism, relay lens, low-resolution grayscale camera, and other components. The target scene is represented as the original three-dimensional spectral data X∈R H×W×C , where R represents a real number set, the superscript H×W×C represents the data dimension; H and W represent the length and width of the image; C represents the number of spectral bands; the three-dimensional spectral data entering the grayscale camera optical path is recorded as X HR ∈R H×W×C , the three-dimensional spectral data entered into SSCI is recorded as X SSCI ∈R H×W×C ;

[0073] The target used in this embodiment is an original spectral image X containing 17 wavelength bands from 540 nm to 700 nm, with a spatial resolution of 256×256 and a pixel depth of 8 bits. That is, in this example, H=W=256 and C=17.

[0074] Step 2, X in step 1 HR The image is directly formed on the high-resolution grayscale camera through the imaging lens. The response function value of the high-resolution grayscale camera to the kth band is r HR (k), in this embodiment, r HR (k) = 1 (k = 1, ..., C), then the image Y output by the high-resolution grayscale camera is HR ∈R H×W It is expressed as follows:

[0075]

[0076] Among them, Y HR (i, j) represents the value of the pixel in the i-th row and j-th column on the high-resolution grayscale camera, X HR (i, j, k) represents the three-dimensional spectral data X HR The value of the pixel in the kth band, ith row, and jth column. HR and X HR All are vectorized, and the vector-matrix form is:

[0077] Y HR =H HR X HR (14)

[0078] Among them, Y HR ∈R HW×1 With X HR ∈R HWC×1 Represents vectorized Y HR and XHR , H HR ∈R HW×HWC is the response matrix of the high-resolution grayscale camera optical path.

[0079] Step 3: X in step 1 SSCI The image is formed on the DMD through the imaging lens, and the DMD loads the coding template and uses the coding template to align the X SSCI Perform spatial modulation to obtain the spatial modulated image Y' SSCI ;

[0080] Using the encoding template M∈R H×W For X in step 1 SSCI Perform spatial coding. In this embodiment, the coding template M is as follows: Figure 4 As shown, the coding template M is loaded onto the DMD and X SSCI Perform spatial encoding, then X SSCI The spatial dimension information of X is changed, that is, SSCI The spatial modulation of the image Y' is obtained SSCI ∈R H×W×C , change Y' SSCI The modulation result of the kth band, ith row and jth column is recorded as Y' SSCI (i,j,k), then Y' SSCI (i,j,k) is represented as follows:

[0081] Y' SSCI (i,j,k)=M(i,j)X SSCI (i,j,k),k=1,...,C (15)

[0082] Among them, M(i,j) represents the value of the encoding template M in the i-th row and j-th column, X SSCI (i,j,k) represents X SSCI The value of the pixel in the kth band, i-th row, and j-th column;

[0083] Step 4, Y' in step 3 SSCI ∈R H×W×C After passing through the relay lens and reaching the dispersion prism, the dispersion makes Y' SSCI Scatter along a certain direction of the spatial dimension; adjust the position of the dispersion prism so that the information light is incident on the surface of the dispersion prism, and the dispersion prism is Y' SSCII Modulate the spectral dimension to obtain the spectral modulation image Y" SSCI , remember Y' SSCI All pixels in the kth band and the jth column disperse along the y-axis of the spatial coordinate system and reach the j'th column, then Y" SSCI ∈R H×(W+D(C))×C It is expressed as follows:

[0084] Y” SSCI (i,j',k)=Y' SSCI (i,j+D(k),k),k=1,...,C (16)

[0085] Among them, Y SSCI (I,j',k) represents Y" SSCI The value of the pixel in the k-th band, the i-th row, and the j'-th column is j'=j+D(k), where D(k) represents the dispersion function caused by the dispersion prism in the k-th band, which in this embodiment is:

[0086] D(k)=(k-1)×α (17)

[0087] Here, α is the dispersion coefficient of the prism, and in this embodiment, α=4.

[0088] Step 5: The imaging lens takes the Y" obtained in step 4 SSCI When imaging onto a low-resolution grayscale camera, since the spatial resolution of the DMD is higher than that of the low-resolution grayscale camera, it is necessary to focus the light reflected by multiple micro-mirrors onto one pixel of the camera to achieve spatial compression; and since the low-resolution grayscale camera plane is a two-dimensional plane, Y" SSCI On a low-resolution grayscale camera, it will be compressed into a two-dimensional image to achieve spectral domain compression;

[0089] The response function value of the low-resolution grayscale camera to the kth band is r LR (k), in this embodiment, r HR (k)=1(k=1,...,C), the image resolution is h×w, and the spatial compression ratio is a(0 <a<1),本实施例中a=0.25,则有:

[0090] h=H×a=256×0.25=64 (18)

[0091] w=(W+D(C))×a=(256+(17-1)×4)×0.25=80 (19)

[0092] The spatial-spectral domain compression is to perform spatial downsampling and spectral downsampling on the modulated image in a low-resolution grayscale camera to obtain a compressed image Y SSCI , change Y SSCI In the low-resolution grayscale camera coordinate system, the pixel value in the pth row and qth column is recorded as Y SSCI (p,q), then Y SSCI (p,q) is represented as follows:

[0093]

[0094] Among them, the relationship between i, j and p, q is:

[0095]

[0096] Among them, the value ranges of p and q are p=1, 2, ..., h, q=1, 2, ..., w respectively; colon: indicates increment from the previous number, with a step length of 1, until the next number. Same as the last part of step 2, for Y SSCI and X SSCI All are vectorized, and the vector-matrix form is:

[0097] Y SSCI =H SSCI X SSCI (twenty three)

[0098] Among them, Y SSCI ∈R hw×1 With X SSCI ∈R HWC×1 Represents the vectorized Y SSCI and Y SSCI , H SSCI ∈R H(W+D(C))×HWC is the equivalent linear operator of the discrete summation in Equation (20), which integrates the combined effects of the low-resolution grayscale camera spectral response, DMD spatial modulation, and dispersion prism spectral modulation.

[0099] Step 6: Combine the high spatial resolution imaging optical path with the SSCI optical path and vectorize the X in step 1. The model of the entire system can be expressed as:

[0100] Y=HX (24)

[0101]

[0102] in, is the measurement value of the entire system, H is the measurement matrix of the entire system, r HR (k) is the response function value of the high-resolution camera in the kth band. In this embodiment, r HR (k) = 1 (k = 1, ..., C), the coding template M is The size of is divided into Ha×Wa sub-blocks, then The row vectorized form of the transformed block in the mth row and nth column of the coding template M is shown in the following example. Figure 5 As shown, X∈R HWC×1 is the vectorized x.

[0103] Step 7: Use the reconstruction algorithm to reconstruct y obtained in step 6 to reconstruct the original three-dimensional spectral data X in step 1.

[0104] The reconstruction algorithm described in step seven adopts a deep learning method.

[0105] The image restoration algorithm described in step 7 adopts a deep learning method, and the network model used is the DAUHST model; the three-dimensional spectral data is reconstructed using the measurement value Y, the measurement matrix H and the trained network model.

[0106] In this example, a high-resolution spectral image of 17 bands can be reconstructed. Therefore, this example uses the spatial spectrum domain compression imaging method to achieve a 16-fold (4×4) increase in the spatial resolution of the compressed image and a 17-fold increase in the spectral resolution. The high-resolution grayscale image collected by the system and the compressed measurement image are shown in Figure 2. Figure 6 As shown, the reconstruction effect is as Figure 7 In order to quantitatively analyze the reconstruction effect, the peak signal-to-noise ratio (PSNR) and spectral angle (SAM) of the reconstructed image are calculated. PSNR is defined as follows:

[0107]

[0108] Among them, MSE represents the mean square error, X(:,:,k) represents the original data corresponding to the kth band in x, express The reconstructed data corresponding to the kth band in , MAX represents the maximum pixel value of the image.

[0109] SAM is defined as follows:

[0110]

[0111] Among them, X(:,k) represents the vectorized form of the original data corresponding to the kth band in X, express The reconstructed data corresponding to the k-th band in the vectorized form.

[0112] The device for implementing the above method is as follows: Figure 2As shown, it includes a collimator 2, a filter 3, a spectroscope 4, three groups of imaging lenses 5, 9, and 11, a high-resolution grayscale camera 12, a DMD 6, a relay lens 7, a dispersion prism 8, a low-resolution grayscale camera 10, and a computer 13; the light from the target 1 first passes through the collimator 2 and the filter 3, and is divided into two paths after reaching the spectroscope 4. One path is reflected by the spectroscope 4 and enters the high spatial resolution imaging optical path, and the other path passes through the spectroscope 4 and enters the spatial spectrum coding compression measurement optical path; the light beam entering the grayscale camera optical path passes through the imaging lens 11 and is directly imaged on the high-resolution grayscale camera 12 to obtain a high spatial resolution grayscale image; the light beam entering the spatial spectrum coding compression measurement optical path passes through the imaging lens 11 and is directly imaged on the high-resolution grayscale camera 12 to obtain a high spatial resolution grayscale image; After the image lens 5, it reaches the DMD 6. The DMD loads the coding template according to the preset setting and spatially modulates the target light beam. The spatially modulated target light beam then passes through the relay lens 7 to reach the dispersion prism 8. The dispersion prism 8 disperses the light of different wavelengths along the dispersion direction and spectrally modulates the target light beam. The spatially spectrally modulated target light beam is then imaged by the imaging lens 9 onto the low-resolution grayscale camera 10. Since it only passes through one dispersion element, the low-resolution grayscale camera 10 obtains a two-dimensional compressed image of the two dimensions of aliasing space and spectrum. The obtained high-resolution grayscale image and compressed image are uploaded to the computer 13, and the high-resolution spectral image is reconstructed by solving the reconstruction algorithm.

[0113] The function of the collimating lens 2 is to collimate the incident light to obtain the parallel light of the target;

[0114] The function of filter 3 is to control the spectral band range;

[0115] The function of the beam splitter 4 is to split the target beam into two paths, one path enters the high spatial resolution imaging optical path, and the other path enters the spatial spectrum coding compression measurement optical path;

[0116] The functions of imaging lenses 5, 9, and 11 are to image the target onto a high-resolution grayscale camera, to image the target beam onto a digital micromirror array (DMD), and to image the target beam after spatial spectrum modulation onto a low-resolution grayscale camera.

[0117] The function of DMD6 is to store the designed coding template and load the coding template to spatially modulate the target light beam imaged onto the DMD;

[0118] The function of the relay lens 7 is to transmit the light beam reflected by the DMD to the dispersion prism;

[0119] The function of the dispersion prism 8 is to disperse the light beam along the dispersion direction according to the wavelength and perform spectral modulation on the target light beam;

[0120] The low-resolution grayscale camera 10 is used to receive the modulated light beam and obtain a compressed image;

[0121] The high-resolution grayscale camera 12 is used to obtain a grayscale image with high spatial resolution;

[0122] The computer 13 is used to run a spatial spectrum domain compression imaging restoration algorithm and reconstruct an original high-resolution spectral image from the high-resolution grayscale image and the compressed image taken by the dual-channel camera using a measurement matrix.

[0123] The above specific description further illustrates the purpose, technical solutions and beneficial effects of the invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A dual-path spatial-spectral domain joint compressed sensing imaging method, characterized by: The following steps are included: Step 1: After the target scene is split by the spectroscope, it enters the high spatial resolution imaging optical path HRI and the spatial spectrum coding compression measurement optical path SSCI respectively; the spectral data entering the HRI optical path is recorded as X HR ∈R H×W×C The spectral data entering the SSCI optical path is recorded as X SSCI ∈R H×W×C ; Where R represents a set of real numbers, the superscript H×W×C represents the data dimension; H and W represent the length and width of the image; C represents the number of spectral bands; Step 2, Step 1 X HR The output through the HRI optical path is Y HR ∈R H×W , for Y HR and X HR All are vectorized to obtain vector Y HR ; Step 3: X in step 1 SSCI Imaging is onto DMD, which loads the coding template and uses it to align X SSCI Perform spatial modulation to obtain the spatial modulated image Y' SSCI ; Step 4: Y' obtained in step 3 SSCI Arriving at the dispersion prism, the dispersion prism is Y' SSCI Modulate the spectral dimension to obtain the spectral modulation image Y" SSCI ; further get Y' SSCI (i,j,k) and Y" SSCI (i,j',k); Step 5, Y obtained in step 4" SSCI Imaging onto a low-resolution grayscale camera yields Y SSCI , realize space-spectral domain compression; for Y SSCI and X SSCI All are vectorized to obtain Y SSCI ; Step 6: Combine the HRI optical path with the SSCI optical path and convert the original target scene 3D spectral data X∈R H×W×C Vectorize and get Y; Step 7: Reconstruct Y obtained in step 6 using a reconstruction method to reconstruct the original target scene three-dimensional spectral data X.

2. The dual-path spatial-spectral domain joint compressed sensing imaging method according to claim 1, characterized in that: The high spatial resolution imaging optical path HRI described in step 1 includes an imaging lens and a high-resolution grayscale camera; The spatial spectrum coding compression measurement optical path SSCI described in step 1 includes an imaging lens, a DMD, a dispersion prism, a relay lens, and a low-resolution grayscale camera.

3. The dual-path spatial-spectral domain joint compressed sensing imaging method according to claim 1, characterized in that: The specific implementation method of step 2 is: X in step 1 HR The image is directly formed on the high-resolution grayscale camera through the imaging lens. The response function value of the high-resolution grayscale camera to the kth band is r HR (k), the image Y output by the high-resolution grayscale camera is HR ∈R H×W It is expressed as follows: Among them, Y HR (i, j) represents the value of the pixel in the i-th row and j-th column on the high-resolution grayscale camera, X HR (i, j, k) represents the three-dimensional spectral data X HR The value of the pixel in the kth band, ith row, and jth column; HR and X HR All are vectorized and the vector-matrix form is: Y HR =H HR X HR (2) Among them, Y HR ∈R HW×1 With X HR ∈R HWC×1 Represents vectorized Y HR and X HR , H HR ∈R HW×HWC Spectral response matrix of the optical path for high spatial resolution imaging.

4. The dual-path spatial-spectral domain joint compressed sensing imaging method according to claim 1, characterized in that: The specific implementation method of step three is: Use the coding template to encode X in step 1 SSCI Perform spatial encoding, that is, load the encoding template onto the DMD and SSCI Perform spatial encoding, then X SSCI The spatial dimension information of X is changed, that is, SSCI The spatial modulation of the image Y' is obtained SSCI ∈R H×W×C .

5. The dual-path spatial-spectral domain joint compressed sensing imaging method according to claim 1, characterized in that: Step 4 Y' SSCI (i,j,k) and Y" SSCI (i,j',k) is represented as follows, Will Y' SSCI The modulation result of the kth band, ith row and jth column is recorded as Y' SSCI (i,j,k), Y' SSCI (i,j,k) is represented as follows: Y' SSCI (i,j,k)=M(i,j)X SSCI (i,j,k),k=1,...,C (3) Among them, M(i,j) represents the value of the encoding template M in the i-th row and j-th column, X SSCI (i,j,k) represents X SSCI The value of the pixel in the kth band, i-th row, and j-th column; Then Y" SSCI ∈R H×(W+D(C))×C It is expressed as follows: Y” SSCI (i,j',k)=Y' SSCI (i,j+D(k),k),k=1,...,C (4) Among them, Y SSCI (i,j',k) represents Y" SSCI In the kth band, ith row, jth ' The value of the pixel in the column, j ' =j+D(k), where D(k) represents the dispersion function caused by the dispersion prism in the kth band.

6. The dual-path spatial-spectral domain joint compressed sensing imaging method according to claim 1, characterized in that: The specific implementation method of step five is: Let the response function value of the low-resolution grayscale camera for the k-th band be r LR (k). The resolution of the acquired image is h×w, and the spatial compression ratio is a (0 < a < 1). Then we have: h=H×a (5) w=(W+D(C))×a (6) Where D(C) is the dispersion function corresponding to band C; spatial spectral domain compression is to perform spatial downsampling and spectral downsampling on the modulated image in a low-resolution grayscale camera to obtain the compressed image Y SSCI , change Y SSCI In the low-resolution grayscale camera coordinate system, the pixel value in the pth row and qth column is recorded as Y SSCI (p,q), then Y SSCI (p,q) is represented as follows: Among them, the relationship between i, j and p, q is: Wherein, the value ranges of p and q are p=1, 2, ..., h, and q=1, 2, ..., w respectively; the colon indicates incrementing from the previous number with a step size of 1 until the next number; Y SSCI and X SSCI All are vectorized and the vector-matrix form is: Y SSCI =H SSCI X SSCI (10) Among them, Y SSCI ∈R hw×1 With X SSCI ∈R HWC×1 Represents the vectorized Y SSCI and X SSCI , H SSCI ∈R H(W+D(C))×HWC is the equivalent linear operator of the discrete summation in Eq. (7).

7. The dual-path spatial-spectral domain joint compressed sensing imaging method according to claim 1, characterized in that: The specific implementation method of step six is: The high spatial resolution imaging optical path is combined with the SSCI optical path, and the original target scene three-dimensional spectral data X∈R H×W×C Vectorized, the model of the entire system is expressed as: Y=HX (11) in, is the measurement value of the entire system, H is the measurement matrix of the entire system, r HR (k) is the response function value of the high-resolution camera in the kth band, and the encoding template M is The size of is divided into Ha×Wa sub-blocks, then is the row vectorized form of the mth row and nth column of the encoding template M after deformation, X∈R HWC×1 is the vectorized X.

8. The dual-path spatial-spectral domain joint compressed sensing imaging method according to claim 1, characterized in that: The reconstruction method described in step seven is a deep learning method.

9. An apparatus for implementing the method according to claim 1, characterized in that: It includes a collimating lens, a filter, a spectroscope, three sets of imaging lenses, a high-resolution grayscale camera, a DMD, a relay lens, a dispersion prism, a low-resolution grayscale camera and a computer. The light from the target is divided into two paths by the spectroscope, one of which is reflected by the spectroscope and enters the high-spatial-resolution imaging optical path, and the other passes through the spectroscope and enters the spatial-spectrum coding compression measurement optical path. The light beam entering the high-spatial-resolution imaging optical path is directly imaged on the high-resolution grayscale camera through the imaging lens to obtain a high-spatial-resolution grayscale image. The light beam entering the spatial-spectrum coding compression measurement is imaged onto the surface of the DMD micromirror array through the imaging lens. The DMD loads the coding template according to the preset setting and spatially modulates the target light beam. The spatially modulated target light beam then passes through a relay lens to a dispersive prism, which disperses light of different wavelengths along the dispersion direction and performs spectral modulation on the target light beam. The spatially spectrally modulated target light beam is then imaged onto a low-resolution grayscale camera by an imaging lens. Since it only passes through one dispersive element, the low-resolution grayscale camera obtains a two-dimensional compressed image that is a mixture of spatial and spectral dimensions. The obtained high-resolution grayscale image and spatial-spectral compressed image are uploaded to a computer, and a high-resolution spectral image is reconstructed by solving a reconstruction algorithm.

10. The device according to claim 9, characterized in that: The function of the beam splitter is to split the target light beam into two paths, one path entering the high spatial resolution imaging light path, and the other path entering the spatial spectrum coding compression measurement light path; The functions of the imaging lens are to image the target onto a high-resolution grayscale camera, to image the target beam onto a digital micromirror array (DMD), and to image the coded target after spatial spectrum modulation onto a low-resolution grayscale camera. The function of high-resolution grayscale camera is to obtain grayscale images with high spatial resolution; The function of DMD is to store the designed coding template and load the coding template to spatially modulate the target light beam imaged onto the DMD; The function of the relay lens is to transmit the light beam reflected by the DMD to the dispersion prism; The function of the dispersion prism is to disperse the light beam along the dispersion direction according to the wavelength and perform spectral modulation on the target light beam; The function of the low-resolution grayscale camera is to receive the modulated light beam and obtain a compressed image; The role of the computer is to run a dual-path spatial-spectral domain joint compressed sensing imaging recovery algorithm, and use the measurement matrix to reconstruct a high-resolution spectral image of the target scene from the high-resolution grayscale image and spatial-spectral image taken by the dual-path camera.

Images