Visible light infrared fusion compression spectrum imaging system and method based on single pixel

Abstract

The invention provides a single-pixel-based visible light infrared fusion compression spectrum imaging system and method, and the method comprises the following steps: 1, carrying out the signal coding of a to-be-detected target signal, and obtaining a coded to-be-detected target infrared signal and a coded to-be-detected target visible light signal; step 2, inputting a to-be-measured target infrared signal into an infrared imaging light path, collecting a medium-wave infrared spectrum intensity value, and inputting a to-be-measured target visible light signal into a visible light imaging light path to collect a visible light spectrum intensity value; step 3, carrying out compressed sampling and reconstruction operation on the visible light spectrum intensity value and the medium wave infrared spectrum intensity value to obtain an infrared hyperspectral image and a visible light hyperspectral image of the target to be detected; 4, performing image fusion according to the infrared hyperspectral image and the visible light hyperspectral image to obtain a fused image with rich details; the fusion imaging of the method has the characteristic of keeping the feature information of the target scene so as to realize high-quality imaging.

Description

Technical Field

[0001] The present invention belongs to the technical field of multispectral imaging, and particularly relates to a visible light-infrared fusion compressed spectral imaging system and method based on single pixel. Background Technique

[0002] Infrared images have strong anti-interference ability, can distinguish targets from backgrounds, detect scene details that cannot be detected in the visible light band, and have the ability to work all-weather; visible light detectors are similar to the spectral bands that the human eye can perceive, and their imaging results are more in line with the human visual system. Infrared images can obtain significant information of targets, but their resolution is low and there are few image details. At the same time, visible light has higher resolution and richer detailed texture information; the fusion of visible light and infrared images can obtain images with richer information, more details, more prominent targets, and more in line with human vision.

[0003] At present, most of the infrared image and visible light fusion images are fused using visible light and infrared images of a single or a small number of bands, lacking spectral information, and some characteristic information may be lost during the fusion process, resulting in the problem of missing scene information. Summary of the Invention

[0004] In order to solve the problem in the prior art that most of the infrared image and visible light fusion images are fused using visible light and infrared images of a single or a small number of bands, lacking spectral information, and some characteristic information may be lost during the fusion process, resulting in the problem of missing scene information.

[0005] Therefore, the present invention provides a visible light-infrared fusion compressed spectral imaging method based on single pixel, including the following steps:

[0006] Step 1: Perform signal encoding on the signal of the target to be measured to obtain the encoded infrared signal of the target to be measured and the encoded visible light signal of the target to be measured;

[0007] Step 2: Input the infrared signal of the target to be measured into the infrared imaging optical path to collect the mid-wave infrared spectral intensity value, and input the visible light signal of the target to be measured into the visible light imaging optical path to collect the visible light spectral intensity value;

[0008] Step 3: Perform compressive sampling and reconstruction operations on the visible light spectral intensity value and the mid-wave infrared spectral intensity value to obtain the infrared hyperspectral image and the visible light hyperspectral image of the target to be measured;

[0009] Step 4: Perform image fusion according to the infrared hyperspectral image and the visible light hyperspectral image to obtain a fusion image with rich details.

[0010] Further, the specific process of Step 1, which is to perform signal encoding on the target signal to be measured and obtain the encoded infrared signal and visible light signal of the target to be measured, is as follows:

[0011] The target to be measured is converged onto the target surface of the infrared DMD modulation module through the front imaging lens group. The infrared DMD modulation module loads the adaptive encoding matrix template. By changing the flipping state of the micromirrors, the target to be measured is randomly reflected to complete the encoding modulation of the target to be measured, obtaining an N×S-dimensional original signal X. Compressive observation is performed through the adaptive encoding template matrix Φ, and the adaptive encoding template matrix Φ = (Φ 1 Φ 2 ···Φ M ) T , Φ is a binary matrix, and Φ i (i = 1, 2, ··· M) corresponds to the flipping state of the DMD micromirrors;

[0012] The values of the matrix elements of the adaptive encoding template matrix Φ are 0 or 1, and the element distribution is adaptively modulated according to the image information.

[0013] Further, the specific process of Step 2, which is to input the infrared signal of the target to be measured into the infrared imaging optical path to collect the mid-wave infrared spectral intensity value, and input the visible light signal of the target to be measured into the visible light imaging optical path to collect the visible light spectral intensity value, is as follows:

[0014] The infrared signal of the target to be measured is input into the first relay lens group for correction and imaging, so that its image plane is at the incident plane of the dispersion prism group. After dispersion by the dispersion prism group, an infrared hyperspectral signal is formed. The infrared hyperspectral signal is input into the second relay lens group, and the second relay lens group performs correction and imaging, so that its image plane is on the photosensitive area of the data acquisition module;

[0015] The visible light signal of the target to be measured is input into the first relay lens group for correction and imaging, so that its image plane is at the incident plane of the dispersion prism group. After dispersion by the dispersion prism group, a visible light hyperspectral signal is formed. The visible light hyperspectral signal is input into the second relay lens group, and the second relay lens group performs correction and imaging, so that its image plane is on the photosensitive area of the data acquisition module.

[0016] Further, the observed value obtained by the data acquisition module can be expressed as:

[0017] Y = ΦX

[0018] Wherein, Y is an M×S - dimensional observation signal obtained by compressive observation of an N×S - dimensional original signal X, Φ is a binary M×N - dimensional observation matrix, S is the spectral dimension. By controlling the DMD modulation module H times, the synchronization control module controls two single - pixel detectors of the data acquisition module to collect H times of mid - wave infrared and H times of visible light hyperspectral data, and the measured value Y is obtained through H times of observations.

[0019] Further, the specific process of step 3, performing compressive sampling and reconstruction operations on the visible light spectral intensity value and the mid - wave infrared spectral intensity value to obtain the infrared hyperspectral image and the visible light hyperspectral image of the target to be measured is as follows:

[0020] To ensure the accurate inversion of the hyperspectral image, the signal X λ is subjected to sparse transformation. X λ is the signal of a single spectral dimension in the original signal X. To make it meet the prerequisite conditions of the compressive sensing theory, it is sparsely represented as:

[0021] X λ = Ψ λ θ λ

[0022] Wherein, θ λ is the sparse coefficient of the original signal X λ under the sparse basis Ψ λ Ψ λ is a Q×Q - dimensional sparse transformation matrix. Then, Y λ = ΦX λ is re - expressed as:

[0023] Y λ = ΦX λ = ΦΨ λ θ λ = A λ θ λ

[0024] Wherein, A λ is a P×Q - dimensional sensing matrix, Y λ is the original signal of a single spectral dimension and

[0025]

[0026] Sparse coefficient solution:

[0027] The sparse coefficient θ λ is solved by solving the inverse problem of the above formula, and this process can be expressed as:

[0028] min||θ λ || 1 s.t.Y λ = Aλ θ λ

[0029] After considering the error term and the sparse basis, the above equation is equivalent to the convex optimization solution for the original sparse coefficient θ λ :

[0030]

[0031] In the formula, the first term is the l 2 norm minimization of the difference between the model and the measured data, and the second term is the l 1 norm of the reconstruction coefficient, indicating its sparsity. The parameter γ > 0 is a regularization factor. The optimal solution of the above equation is solved by the compressive sensing reconstruction algorithm: the two-step iterative shrinkage TWIST algorithm and the original target scene single spectral dimension X is inversely obtained therefrom λ image, and the original signal X of the single spectral dimension is reconstructed λ which is expressed as:

[0032]

[0033] Then the reconstructed hyperspectral original signal is expressed as:

[0034]

[0035] By continuously loading 1000 groups of adaptive coding templates and when the sampling rate is 32.55%, the compressive sampling of the target scene is completed, and the hyperspectral image reconstructed by using the compressive sensing reconstruction algorithm TWIST is obtained.

[0036] Furthermore, the specific process of step 4, that is, performing image fusion according to the infrared hyperspectral image and the visible light hyperspectral image to obtain a fused image with rich details, is as follows:

[0037] By using a deep learning convolutional neural network, the features of the visible light and mid-wave infrared hyperspectral images are extracted, and LReLU is used as the activation function to realize the fusion of the visible light and mid-wave infrared hyperspectral images;

[0038] The fusion process can be expressed as:

[0039]

[0040] where F represents the fusion operation, i represents the i-th input image, is the image feature information of the fused image, and f i (x, y) represents the image feature information of the i-th input image.

[0041] A visible light and infrared fusion compressed spectral imaging system based on single pixel, comprising an active illumination module, a front imaging lens group, an infrared DMD modulation module, which are sequentially arranged on one side of a target to be measured, and two groups of first relay lens groups, two groups of dispersion prism groups, two groups of second relay lens groups, and a data acquisition module, which are sequentially arranged along the beam direction. The data acquisition module includes an infrared single pixel detector and a visible light single pixel detector; the beam propagation directions of the infrared single pixel detector and the visible light single pixel detector correspond to the data acquisition module, and the data acquisition module corresponds to a hyperspectral image fusion module;

[0042] It further includes a synchronous control module, which is electrically connected to the infrared DMD modulation module (4) and the control data acquisition module respectively.

[0043] The advantages of the present invention are as follows: The present invention provides this visible light and infrared fusion compressed spectral imaging method based on single pixel, which simultaneously uses visible light hyperspectral images and mid-wave infrared spectral images for fusion imaging, and has the characteristics of retaining the feature information of the target scene to achieve high-quality imaging; in the visible light and infrared fusion compressed spectral imaging system based on single pixel, by utilizing the ability of the infrared DMD to the bidirectional reflection light field, a dual-channel single pixel compressive sensing spectral imaging system is set up, one channel corresponding to visible light and the other channel corresponding to mid-wave infrared, having the effect of simultaneously acquiring visible light and mid-wave infrared hyperspectral images of the target scene.

[0044] The following will describe the present invention in detail with reference to the drawings and embodiments. Description of the Drawings

[0045] Figure 1 It is a schematic structural diagram of the system of the present invention.

[0046] Figure 2 It is a flowchart of the method of the present invention.

[0047] Figure 3a It is a schematic diagram of a visible light image of the target scene.

[0048] Figure 3b It is a schematic diagram of an infrared image of the target scene.

[0049] Figure 4 It is a schematic diagram of a fusion image of the target scene.

[0050] In the figure: 1. Target to be measured; 2. Active illumination module; 3. Front imaging lens group; 4. Infrared DMD modulation module; 5. First relay lens group; 6. Dispersion prism group; 7. Second relay lens group; 8. Infrared single pixel detector; 9. Visible light single pixel detector; 10. Data acquisition module; 11. Synchronous control module; 12. Hyperspectral imaging calculation module; 13. Hyperspectral image fusion module. Detailed Embodiment

[0051] To further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined purpose, the specific implementation manners, structural features and their effects of the present invention will be described in detail below in conjunction with the accompanying drawings and embodiments.

[0052] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0053] All the features disclosed in this specification, or all the steps in the disclosed methods or processes, except for mutually exclusive features and / or steps, can be combined in any manner.

[0054] Any feature disclosed in this specification (including any additional claims, abstract and drawings), unless specifically stated, can be replaced by other equivalent or similar-purpose alternative features. That is, unless specifically stated, each feature is only an example of a series of equivalent or similar features.

[0055] Embodiment 1

[0056] This embodiment proposes a visible light-infrared fusion compressed spectral imaging method based on single pixels as Figure 1 shown, including the following steps:

[0057] Step 1: Perform signal encoding on the signal of the target to be measured to obtain the encoded infrared signal of the target to be measured and the encoded visible light signal of the target to be measured;

[0058] Step 2: Input the infrared signal of the target to be measured into the infrared imaging optical path to collect the mid-wave infrared spectral intensity value, and input the visible light signal of the target to be measured into the visible light imaging optical path to collect the visible light spectral intensity value;

[0059] Step 3: Perform compressive sampling and reconstruction operations on the visible light spectral intensity value and the mid-wave infrared spectral intensity value to obtain the infrared hyperspectral image and the visible light hyperspectral image of the target to be measured;

[0060] Step 4: Perform image fusion according to the infrared hyperspectral image and the visible light hyperspectral image to obtain a fused image with rich details.

[0061] Furthermore, the specific process of the step 1: performing signal encoding on the signal of the target to be measured to obtain the encoded infrared signal of the target to be measured and the encoded visible light signal of the target to be measured is:

[0062] The target to be measured is converged by the front imaging lens group 3 onto the target surface of the infrared DMD modulation module 4. The infrared DMD modulation module 4 loads an adaptive coding matrix template. By changing the flipping state of the micromirrors, the target to be measured is randomly reflected, completing the coding modulation of the target to be measured, and obtaining an N×S-dimensional original signal X. Compressive observation is performed through the adaptive coding template matrix Φ. The adaptive coding template matrix Φ = (Φ 1 Φ 2 ···Φ M ) T , Φ is a binary matrix, and Φ i (i = 1, 2, ··· M) corresponds to the flipping state of the DMD micromirrors;

[0063] The values of the matrix elements of the adaptive coding template matrix Φ are 0 or 1, and the element distribution is adaptively modulated according to the image information.

[0064] Further, the specific process of step 2, inputting the infrared signal of the target to be measured into the infrared imaging optical path to collect the mid-wave infrared spectral intensity value, and inputting the visible light signal of the target to be measured into the visible light imaging optical path to collect the visible light spectral intensity value is as follows:

[0065] The infrared signal of the target to be measured is input into the first relay lens group 5 for correction and imaging, so that its image plane is at the incident plane of the dispersion prism group 6. After dispersion by the dispersion prism group 6, an infrared hyperspectral signal is formed. The infrared hyperspectral signal is input into the second relay lens group 7, and the second relay lens group 7 performs correction and imaging, so that its image plane is on the photosensitive area of the data acquisition module 10;

[0066] The visible light signal of the target to be measured is input into the first relay lens group 5 for correction and imaging, so that its image plane is at the incident plane of the dispersion prism group 6. After dispersion by the dispersion prism group 6, a visible light hyperspectral signal is formed. The visible light hyperspectral signal is input into the second relay lens group 7, and the second relay lens group 7 performs correction and imaging, so that its image plane is on the photosensitive area of the data acquisition module 10.

[0067] Further, the observed value obtained by the data acquisition module 10 can be expressed as:

[0068] Y = ΦX

[0069] In the formula, Y is an M×S-dimensional observation signal Y obtained by compressive observation of the N×S-dimensional original signal X, Φ is a binary M×N-dimensional observation matrix, S is the spectral dimension. By controlling the DMD modulation module 4 for H times, the synchronization control module 11 controls the two single-pixel detectors of the data acquisition module 10 to collect H times of mid-wave infrared and H times of visible light hyperspectral data, and the measured value Y is obtained through H times of observation;

[0070] Further, the specific process of step 3, which performs compressive sampling and reconstruction operations on the visible light spectral intensity value and the mid-wave infrared spectral intensity value to obtain the infrared hyperspectral image and the visible light hyperspectral image of the target to be measured, is as follows:

[0071] To ensure the accurate inversion of the hyperspectral image, the signal X λ is subjected to sparse transformation, where X λ is the signal of a single spectral dimension in the original signal X, so that it meets the prerequisite conditions of the compressive sensing theory, and its sparse representation is:

[0072] X λ = Ψ λ θ λ

[0073] In the formula, θ λ is the sparse coefficient of the original signal X λ under the sparse basis Ψ λ , and Ψ λ is a Q×Q-dimensional sparse transformation matrix; thus, Y λ = ΦX λ is re-expressed as:

[0074] Y λ = ΦX λ = ΦΨ λ θ λ = A λ θ λ

[0075] In the formula, A λ is a P×Q-dimensional sensing matrix, and Y λ is the original signal of a single spectral dimension and

[0076] Sparse coefficient solution:

[0077] The sparse coefficient θ λ can be solved by solving the inverse problem of the above formula, and this process can be expressed as:

[0078] min||θ λ || 1 s.t.Y λ = A λ θ λ

[0079] After considering the error term and the sparse basis, the above formula is equivalent to the convex optimization solution of the original sparse coefficient θ λ :

[0080]

[0081] In the formula, the first term is the l of the difference between the model and the measured data2 The norm is minimized, and the second term is the l 1 norm of the reconstruction coefficient, representing its sparsity. The parameter γ > 0 is a regularization factor. The optimal solution of the above formula is solved by a compressive sensing reconstruction algorithm such as the two-step iterative shrinkage TWIST algorithm. And the original target scene single spectral dimension X is inversely obtained therefrom. λ The image of the single spectral dimension of the original signal X is reconstructed. λ It is expressed as:

[0082]

[0083] Then the reconstructed hyperspectral original signal is expressed as:

[0084]

[0085] By continuously loading 1000 groups of adaptive coding templates, when the sampling rate is 32.55%, the compressive sampling of the target scene is completed, and the hyperspectral image reconstructed by the compressive sensing reconstruction algorithm TWIST is obtained.

[0086] Furthermore, the specific process of step 4, performing image fusion according to the infrared hyperspectral image and the visible light hyperspectral image to obtain a fusion image with rich details, is as follows:

[0087] By using a deep learning convolutional neural network, the features of the visible light and mid-wave infrared hyperspectral images are extracted, and LReLU is used as the activation function to realize the fusion of the visible light and mid-wave infrared hyperspectral images. The fusion process can be expressed as:

[0088]

[0089] Among them, F represents the fusion operation, and i represents the i-th input image. is the image feature information of the fused image, and f i (x, y) represents the image feature information of the i-th input image.

[0090] In summary, this embodiment provides this visible light-infrared fusion compressive spectral imaging method based on single pixels, which simultaneously performs fusion imaging using visible light hyperspectral images and mid-wave infrared spectral images, and has the characteristics of retaining the feature information of the target scene to achieve high-quality imaging.

[0091] Embodiment 2

[0092] This embodiment provides a method such as Figure 2The single-pixel-based visible light and infrared fusion compression spectral imaging system shown in the figure includes an active illumination module 2, a front imaging lens group 3, an infrared DMD modulation module 4, which are sequentially arranged on one side of the target to be measured 1, and two groups of first relay lens groups 5, two groups of dispersion prism groups 6, two groups of second relay lens groups 7, and a data acquisition module 10, which are sequentially arranged along the light beam direction. The data acquisition module 10 includes an infrared single-pixel detector 8 and a visible light single-pixel detector 9. The light beam propagation directions of the infrared single-pixel detector 8 and the visible light single-pixel detector 9 correspond to the data acquisition module 10, and the data acquisition module 10 corresponds to the hyperspectral image fusion module 13.

[0093] It further includes a synchronization control module 11, which is electrically connected to the infrared DMD modulation module 4 and the control data acquisition module 10 respectively.

[0094] The active illumination module 2 is used to actively illuminate the target to be measured 1, the target scene, etc.

[0095] The front imaging lens group 3 is used to converge and image the light reflected by the target to be measured 1, so that the imaging plane is located on the target surface of the DMD of the DMD modulation module 4.

[0096] The DMD modulation module 4 is used to generate a preset adaptive coding template matrix, and adaptively code the target signal to be measured incident on the DMD target surface according to the coding template matrix and reflect it in two directions to obtain two-way coded target signals to be measured.

[0097] The first relay lens group 5 is used to correct and image the coded target signal to be measured, so that its image plane is on the incident surface of the dispersion prism.

[0098] The dispersion prism group 6 is used to disperse the two-way coded target signals to be measured to form hyperspectral signals.

[0099] The second relay lens group 7 is used to correct and image the dispersed hyperspectral signal, so that its image plane is on the photosensitive area of the data acquisition module.

[0100] The data acquisition module 10 is used to collect and convert the coded hyperspectral data of the target to be measured after dispersion, and respectively obtain the spectral intensity values of the visible light and mid-wave infrared of the target to be measured in the two reflection directions. The data acquisition module 10 includes an infrared single-pixel detector 8 and a visible light single-pixel detector 9. The infrared single-pixel detector 8 and the visible light single-pixel detector 9 are respectively used to collect the visible light band intensity values and mid-wave infrared band intensity values of the target coded information in two different reflection directions after coding modulation.

[0101] The synchronization control module 11 is used to control the infrared single-pixel detector 8 and the visible-light single-pixel detector 9 to perform synchronous data acquisition each time the infrared DMD modulation module 4 loads a new adaptive coding matrix template. Specifically, the synchronization control module 11 connects the infrared DMD modulation module 4, the infrared single-pixel detector 8, and the visible-light single-pixel detector 9 through a serial cable, and connects the infrared DMD modulation module 4 to the host computer. The host computer loads the adaptive coding matrix template into the infrared DMD modulation module 4. At the same time, each time an adaptive coding matrix template is loaded, the infrared DMD modulation module 4 generates a pulse signal, which is synchronously transmitted to the infrared single-pixel detector 8 and the visible-light single-pixel detector 9 through the serial cable for synchronous data acquisition.

[0102] The hyperspectral computational imaging module 12 is used to perform compressive sensing reconstruction operations on the intensity values of the hyperspectral information of the target scene obtained by the data acquisition module according to the reconstruction principle of computational coded imaging to obtain mid-wave infrared and visible-light hyperspectral images.

[0103] The hyperspectral image fusion module 13 is used to fuse the computationally reconstructed mid-wave infrared and visible-light hyperspectral images to obtain a fused image with rich details.

[0104] The active illumination module 2 includes an incandescent light source for irradiating the imaging target.

[0105] The DMD modulation module 4 includes a modified infrared DMD for loading the adaptive coding template.

[0106] The adaptive coding matrix generation template is used to generate an adaptive coding template matrix through a deep learning network. The values of the matrix elements are "0" and "1", and the element distribution is adaptively adjusted according to the image information.

[0107] The DMD modulation module 4 is used to control the flipping angle of the micromirrors according to the generated adaptive coding template matrix. Among them, "0" means the micromirror flips -12°, and "1" means the micromirror flips +12°. The flipping states of all micromirrors correspond to the elements of the generated adaptive coding template matrix. The modulated target scene information has two different reflection directions, and the matrix elements in the two reflection directions are complementary.

[0108] In summary, in this single-pixel-based visible-light and infrared fusion compressive spectral imaging system provided in this embodiment, by utilizing the ability of the infrared DMD to generate a bidirectional reflection light field, a dual-channel single-pixel compressive sensing spectral imaging system is set up, one channel corresponding to visible light and the other channel corresponding to mid-wave infrared, achieving the effect of simultaneously obtaining the visible-light and mid-wave infrared hyperspectral images of the target scene.

[0109] Embodiment 3

[0110] Using the single-pixel-based visible-infrared fusion compressed spectral imaging system and method provided in Embodiment 1 and Embodiment 2, the target scene shown in FIG. 3 is fused, where Figure 3a is a schematic diagram of the visible light image of the target scene; Figure 3b is a schematic diagram of the infrared image of the target scene. As Figure 4 shown in the image is the fused image. Compared with the separate visible light and infrared spectral images, the processed fused image prominently retains the detail feature information and infrared information of the target scene, thereby achieving high-quality fused imaging.

[0111] The above content is a further detailed description of the present invention in combination with specific preferred embodiments. It cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A single-pixel visible light infrared fusion compressed spectral imaging method, characterized in that: The steps include: Step 1: Encode the target signal to be measured, and obtain the encoded infrared signal of the target to be measured and the encoded visible light signal of the target to be measured; Step 2: Input the infrared signal of the target to be measured into the infrared imaging optical path to collect the intensity value of the medium-wave infrared spectrum, and input the visible light signal of the target to be measured into the visible light imaging optical path to collect the intensity value of the visible light spectrum; Step 3: compressively sample and reconstruct the visible light spectrum intensity values ​​and the medium-wave infrared spectrum intensity values ​​to obtain an infrared hyperspectral image and a visible light hyperspectral image of the target to be measured; Step 4: performing image fusion based on the infrared hyperspectral image and the visible light hyperspectral image to obtain a fused image with rich details.

2. The method for single-pixel visible-light infrared fusion compressed spectral imaging according to claim 1, characterized in that: The specific process of step 1, encoding the target signal to be measured, and obtaining the encoded infrared signal of the target to be measured and the encoded visible light signal of the target to be measured is: The target to be measured is converged to the target surface of the infrared DMD modulation module (4) through the front imaging lens group (3). The infrared DMD modulation module (4) loads an adaptive coding matrix template, and by changing the flipping state of the micro-mirror, the target to be measured is randomly reflected to complete the coding modulation of the target to be measured, and an N×S dimensional original signal X is obtained. The adaptive coding template matrix Φ is used for compression observation. The adaptive coding template matrix Φ=(Φ1Φ2···Φ M ) T , Φ is a binary matrix, Φ i (i=1,2,···M) corresponds to the flipping state of the DMD micromirror; The values ​​of the matrix elements of the adaptive coding template matrix Φ are 0 or 1, and the element distribution is adaptively modulated according to the image information.

3. The single-pixel visible-light infrared fusion compressed spectral imaging method according to claim 1, characterized in that: The specific process of step 2, inputting the infrared signal of the target to be measured into the infrared imaging optical path, collecting the medium-wave infrared spectrum intensity value, and inputting the visible light signal of the target to be measured into the visible light imaging optical path to collect the visible light spectrum intensity value is: The infrared signal of the target to be measured is input into the first relay lens group (5) for correction and imaging, so that its image plane is located at the incident surface of the dispersion prism group (6), the dispersion prism group (6) forms an infrared hyperspectral signal after dispersion, and the infrared hyperspectral signal is input into the second relay lens group (7), and the second relay lens group (7) performs correction and imaging, so that its image plane is located on the photosensitive area of ​​the data acquisition module (10); The visible light signal of the target to be measured is input into the first relay lens group (5) for correction and imaging, so that its image plane is located at the incident surface of the dispersion prism group (6), and the dispersion prism group (6) forms a visible light hyperspectral signal after dispersion, and the visible light hyperspectral signal is input into the second relay lens group (7), and the second relay lens group (7) performs correction and imaging, so that its image plane is located on the photosensitive area of ​​the data acquisition module (10).

4. The single-pixel visible-light infrared fusion compressed spectral imaging method according to claim 1, characterized in that: The observed value acquired by the data acquisition module (10) can be expressed as: Y=ΦX In the formula, Y is an M×S dimensional observation signal Y obtained by compressing and observing an N×S dimensional original signal X, Φ is a binary M×N dimensional observation matrix, S is a spectral dimension, and by controlling the DMD modulation module (4) H times, the synchronization control module (11) controls two single-pixel detectors of the data acquisition module (10) to collect H times of mid-wave infrared and H times of visible light hyperspectral data, and the measurement value Y is obtained through H times of observation.

5. The single-pixel visible-light infrared fusion compressed spectral imaging method according to claim 1, characterized in that: The specific process of step 3, performing compression sampling and reconstruction operation on the visible light spectrum intensity value and the medium-wave infrared spectrum intensity value to obtain the infrared hyperspectral image and the visible light hyperspectral image of the target to be measured is: To ensure accurate inversion of hyperspectral images, the signal X λ Perform sparse transformation, X λ is the signal of a single spectral dimension in the original signal X, To satisfy the prerequisite of compressed sensing theory, its sparse representation is: X λ =Ψ λ i λ In the formula, θ λ is the original signal X λ In the sparse basis λ The sparse coefficient under λ is a Q×Q dimensional sparse transformation matrix; therefore, Y λ =ΦX λ Re-expressed as: Y λ =ΦX λ =FW λ i λ =A λ i λ In the formula, A λ is a P×Q dimensional sensor matrix, Y λ is the original signal of a single spectral dimension and Sparse coefficient solution: Solve the sparse coefficient θ by solving the inverse problem of the above formula λ , this process can be expressed as: min||θ λ ||1 stY λ =A λ i λ After considering the error term and the sparse basis, the above formula is equivalent to the original sparse coefficient θ λ Convex optimization solution of : In the formula, the first term is the minimization of the l2 norm of the difference between the model and the measured data, the second term is the l1 norm of the reconstruction coefficient, indicating its sparsity, and the parameter γ>0 is the adjustment factor; the optimal solution of the above formula is solved by the compressed sensing reconstruction algorithm: the two-step iterative shrinkage TWIST algorithm The original target scene single spectral dimension X is obtained by inversion λ The image is reconstructed to obtain the original signal X in a single spectral dimension λ It is expressed as: The reconstructed hyperspectral original signal is expressed as: By continuously loading 1000 sets of adaptive coding templates with a sampling rate of 32.55%, the compressed sampling of the target scene is completed, and the hyperspectral image is reconstructed using the compressed sensing reconstruction algorithm TWIST.

6. The single-pixel visible-light infrared fusion compressed spectral imaging method according to claim 1, characterized in that: The specific process of step 4, fusing the infrared hyperspectral image and the visible light hyperspectral image to obtain a fused image with rich details is: By using deep learning convolutional neural network, the features of visible light and medium-wave infrared hyperspectral images are extracted, and LReLU is used as the activation function to achieve the fusion of visible light and medium-wave infrared hyperspectral images; The fusion process can be expressed as: Where F represents the fusion operation, i represents the i-th input image, is the image feature information after fusion, f i (x, y) represents the image feature information of the input i-th image.

7. A single-pixel visible-infrared fusion compressed spectral imaging system, characterized in that: The invention comprises an active illumination module (2), a front imaging lens group (3), an infrared DMD modulation module (4) which are sequentially arranged on one side of a target to be measured (1), and two groups of first relay lens groups (5), two groups of dispersion prism groups (6), two groups of second relay lens groups (7), and a data acquisition module (10) which are sequentially arranged along the direction of a light beam, wherein the data acquisition module (10) comprises an infrared single-pixel detector (8) and a visible light single-pixel detector (9); the light beam propagation directions of the infrared single-pixel detector (8) and the visible light single-pixel detector (9) correspond to the data acquisition module (10), and the data acquisition module (10) corresponds to a hyperspectral image fusion module (13); It also comprises a synchronous control module (11), wherein the synchronous control module (11) is electrically connected to the infrared DMD modulation module (4) and the control data acquisition module (10) respectively.

Images