Spectral coding imaging system and reconstruction method

Abstract

The invention is suitable for the technical field of spectral imaging, and particularly relates to a spectral coding imaging system and a reconstruction method. The spectral imaging system provided by the invention comprises spectral acquisition and spectral super-resolution reconstruction of an upper computer. An image detector with a surface integrated deterministic filtering structure is used for spectrum acquisition and is responsible for spectrum coding of a scene to be detected. And the spectrum reconstruction module realizes spectrum super-resolution and space image reconstruction aiming at the spectrum coding value. The device has the advantages of being simple in structure, low in cost, high in flux and the like, and the reconfiguration wavelength resolution can be tuned according to actual application scene requirements.

Description

Technical Field

[0001] The present invention belongs to the technical field of spectral imaging, and in particular relates to a spectral coding imaging system and a reconstruction method based on a deterministic filtering structure. Background Art

[0002] The spectral imaging system based on deterministic filtering can encode the spectral signal of the target to be measured by integrating a filtering structure on the surface of the image sensor, forming a two-dimensional compressed image containing spectral information, and then reconstructing a hyperspectral image through decoding.

[0003] The traditional push-broom spectral imaging system has low temporal resolution and is not suitable for timely scenarios; FP cavity line array scanning overcomes the shortcomings of traditional recommendations such as large size, but has poor temporal resolution; snapshot spectral imaging based on pixel-level array filter array integrated array detector is the trend of the future, but because array filtering generally adopts a thin-film FP cavity structure, the channel coding efficiency does not exceed 5%, which limits its application; the array metasurface developed in recent years has the ability of single-pixel channel multi-spectral encoding, but the device preparation process is complex, consistency, stability encounters challenges, and energy utilization.

[0004] Therefore, technical personnel in this field urgently need to develop a spectral imaging system that is simple to manufacture, low in cost, high in energy utilization, and high in spatial resolution, temporal resolution, and spectral resolution. This is critical for the development of future spectral imaging chips in smart agriculture, military, and other applications. Summary of the invention

[0005] The object of the present invention is to provide a spectral coding imaging system and reconstruction method which can modulate the spectral signal of a target to be measured to form a two-dimensional compressed image containing spectral information, and reconstruct a hyperspectral image after decoding.

[0006] To achieve the above-mentioned purpose, the system of the present invention comprises targets to be measured arranged in sequence along the light transmission direction, an imaging lens for imaging the object light emitted by the targets to be measured, a deterministic filter structure for encoding the targets to be measured to obtain a coded measurement value, and an image detector for converting the coded measurement value information into a grayscale signal and then decoding and reconstructing it through a spectral super-resolution reconstruction module, so as to construct a complete hyperspectral three-dimensional data cube;

[0007] The deterministic filtering structure is composed of a number of coding units arranged in a matrix, each coding unit corresponds to its own image detector, and the transmittance curve of each coding unit is obtained by the following method;

[0008] Step 1.1, adjusting the central wavelength of the monochromatic light output by the electric monochromator to the minimum operating wavelength of the image detector;

[0009] Step 1.2, the monochromatic light passes through the integrating sphere and the collimator and is incident on the deterministic filter structure in the form of parallel light. The encoded monochromatic light forms a grayscale electrical signal on the surface of the image detector, and the response signal intensity of the monochromatic light is obtained;

[0010] Step 1.3, adjusting the central wavelength of the monochromatic light output by the electric monochromator with a fixed step length d, keeping the incident intensity of the monochromatic light unchanged, repeating step 1.2 until the central wavelength of the monochromatic light output by the monochromator reaches the maximum working wavelength of the image detector, and obtaining the response signal intensity change curve of the image detector of the coding unit to the incident monochromatic light;

[0011] Step 1.4, change the spectral light signal of the electric monochromator, repeat steps 1.1 to 1.3, and obtain the transmittance curve of each coding unit of the deterministic filter structure 3.

[0012] The wavelength encoding characteristic of the deterministic filtering structure encoding unit has a high throughput greater than or equal to 60%.

[0013] The correlation of transmittance curves corresponding to adjacent pixels of each coding unit of the deterministic filtering structure is between 0 and 0.5.

[0014] The spectral encoding imaging reconstruction method according to the above imaging system comprises the following steps:

[0015] Step 1, obtaining the transmittance curve of the deterministic filter structure for light signals in different spectral bands;

[0016] Step 2: Divide into several coding units (n*n), where n is between 2 and 100. According to the transmittance curve obtained in step 1, construct the measurement matrix Φ i (λ);

[0017] Step 3: Encode the spectral information S(λ,x,y) of the target to be measured at the spectral information space (x,y) received by n*n different channels in each coding unit of the deterministic filtering structure. The image detector converts the modulated spectral information S(λ,x,y) of the target to be measured into a grayscale signal Yi(x,y), which has the following relationship:

[0018]

[0019] λmin and λmax are the lower and upper limits of the spectrum range, respectively. i (λ) is the spectral transmittance curve of the i-th deterministic filter structure, i=1,2,...,M, D(λ) is the spectral response curve of the image detector, and these M light intensity values ​​constitute a set of codes of the original spectral signal S(λ,x,y);

[0020] Step 4: According to the spectral information of the target to be measured, S(λ) is sparsely represented, and the optimization equation is obtained by using a greedy algorithm based on compressed sensing:

[0021]

[0022] Among them, Φ i (λ) is the measurement matrix composed of the spectral transmittance of the coding unit, with n rows and N columns;

[0023] D(λ) is the matrix obtained by discretizing the spectral response curve of the image detector into N*1;

[0024] Ψ is a sparse basis matrix, α is a sparse coefficient, Yi is the electrical signal strength matrix Y of one of the coding units received by the image detector i =(Y 1 , Y 2 ,…,Y n );

[0025] Step 5: Solve the sparse coefficients according to the algorithm and reconstruct the target spectrum S(λ, x, y) of a coding unit by super-resolution;

[0026] Step 6: Repeat steps 2 to 5 to solve the target spectrum S(λ, x, y) for all divided coding units in turn, so as to construct a complete hyperspectral three-dimensional data cube and complete the super-resolution reconstruction of the spectrum.

[0027] The step 2 is specifically as follows:

[0028] Step 2.1, each coding unit of the deterministic filtering structure is arranged periodically;

[0029] Step 2.2, according to the transmittance curves of the deterministic filter structure for optical signals of different spectral bands obtained in step 1, the curve values ​​of each channel of the coding unit are sequentially used as each row of the measurement matrix to construct a measurement matrix;

[0030] Step 2.3, repeat step 2.2 to sequentially obtain the measurement matrices corresponding to the coding units of all deterministic filtering structures.

[0031] Compared with the prior art, the present invention has the following beneficial technical effects:

[0032] First, the system uses a deterministic filter structure integrated on the surface of the image detector, including multiple coding units, each filter unit has a different spectral transmittance curve, and the spatial structure is two-dimensionally continuously changing. The spectral characteristics have a high spectral transmittance contrast in the pixel feature size.

[0033] Second, compared to the traditional method of designing a deterministic filter structure based on a curve, the present invention uses an electric monochromator, an integrating sphere, a collimator, and an image detector to obtain the pixel-level spectral transmittance curve of the deterministic filter structure, directly measuring the actual spectral transmittance curve, and the result is more accurate, which is used as the measurement matrix in the algorithm reconstruction. Afterwards, by accurately measuring the spectral characteristics of each pixel, there is no need to design special filters for different applications, avoiding the problem of precise alignment at the pixel level.

[0034] Third, the resolution can precisely adjust the characteristics and improve the adaptability of the system. The system can independently select the size of the filter unit according to the needs to achieve spectral super-resolution of different magnifications, and the encoding flexibility is high. The resolution can precisely adjust the characteristics and improve the adaptability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a structural principle diagram of a spectral imaging system based on a deterministic filtering structure provided by the present invention;

[0036] Figure 2 A schematic diagram of spectral response curve calibration of a deterministic filter structure provided by the present invention;

[0037] Figure 3 It is a schematic diagram of coding unit division provided by the present invention;

[0038] Figure 4 This is a schematic diagram of spectral super-resolution reconstruction in an embodiment of the present invention; DETAILED DESCRIPTION

[0039] In order to make the purpose, advantages and features of the present invention more clear, a spectral imaging system based on a deterministic filtering structure proposed by the present invention is further described in detail in conjunction with the accompanying drawings and specific embodiments.

[0040] like Figure 1 As shown, the system of the present invention includes a target to be measured 1, an imaging lens 2, a deterministic filter structure 3 and an image detector 4 arranged in sequence along the light transmission direction. The imaging lens 2 is used to image the object light emitted by the target to be measured 1 on the deterministic filter structure 3. The deterministic filter structure 3 is composed of a plurality of coding units arranged in a matrix, and each coding unit corresponds to a respective image detector 4. The image detector 4 converts the coded measurement value obtained by encoding each coding unit of the deterministic filter structure 3 into a grayscale signal, and then decodes and reconstructs it through a spectral super-resolution reconstruction module to construct a complete hyperspectral three-dimensional data cube.

[0041] like Figure 2 As shown, the transmittance curve of each coding unit of the deterministic filter structure 3 of the present invention is obtained by the following method;

[0042] Step 1.1, adjusting the central wavelength of the monochromatic light output by the electric monochromator 5 to the minimum operating wavelength of the image detector 4;

[0043] Step 1.2, the monochromatic light passes through the integrating sphere 6 and the collimator 7 and is incident on the deterministic filter structure 3 in the form of parallel light. The encoded monochromatic light forms a grayscale electrical signal on the surface of the image detector 4, and the response signal intensity of the monochromatic light is obtained;

[0044] Step 1.3, adjusting the central wavelength of the monochromatic light output by the electric monochromator 5 with a fixed step length d, keeping the incident intensity of the monochromatic light unchanged, repeating step 1.2 until the central wavelength of the monochromatic light output by the monochromator 5 reaches the maximum working wavelength of the image detector 4, and obtaining the response signal intensity change curve of the image detector 4 of the encoding unit to the incident monochromatic light;

[0045] Step 1.4, change the spectral light signal of the electric monochromator 5, repeat steps 1.1 to 1.3, and obtain the transmittance curve of each coding unit of the deterministic filter structure 3.

[0046] The monochromator 5 of the present invention is arranged on the incident light path of the integrating sphere 6, and then passes through the collimator 7 and the deterministic filter structure 3. The integrating sphere 6 is used to scatter the monochromatic light, so that the monochromatic light is uniformly incident on the deterministic filter structure 3 and generates an electrical signal on the image detector 4 after being encoded. The wavelength of the monochromatic light is changed in sequence with a step length d, and the light intensity encoding values ​​of different bands are obtained, and the pixel-level spectral transmittance curve of the deterministic filter structure is obtained by extraction.

[0047] The spectral imaging system sets an integrating sphere and a collimator in the calibration optical path of the spectral transmittance curve of the deterministic filter structure to ensure the uniformity and collimation of the incident light, thereby improving the accuracy and reliability of the measurement. This design effectively overcomes the problems of uneven illumination and optical path deviation in traditional methods, and ensures the accurate measurement of the spectral transmittance curve of the deterministic filter structure.

[0048] like Figure 3 As shown, the deterministic filter structure includes multiple coding units, and different unit channels have different spectral transmittance curves with low internal correlation. The specifications and sizes of multiple coding unit channels are consistent and evenly arranged, and their length and width are integer multiples of the pixel size of the image sensor. The deterministic filter structure and the image sensor are fixed together by physical bonding. The coding unit channels are combined in a fixed order with different spectral transmittance curves and arranged in an n*n periodic pattern to achieve mosaic snapshot imaging.

[0049] In this embodiment, the image detector is a CMOS detector. Taking the division of 5*5 coding units as an example, the construction principle of the measurement matrix is ​​explained. Figure 2The detector converts the light path into grayscale values, and each coding unit forms a matrix of 25*(λmax-λmin) / d. According to the above reorganization method, each coding unit has a measurement matrix centered on itself.

[0050] See also Figure 4 The spectral super-resolution reconstruction process of this embodiment includes the following steps:

[0051] Step 1, obtaining the transmittance curve of the deterministic filter structure 3 for light signals in different spectral bands;

[0052] Step 1.1, adjusting the central wavelength of the monochromatic light output by the electric monochromator 5 to the minimum operating wavelength of the image detector 4;

[0053] Step 1.2, the monochromatic light passes through the integrating sphere 6 and the collimator 7 and is incident on the deterministic filter structure 3 in the form of parallel light. The encoded monochromatic light forms a grayscale electrical signal on the surface of the image detector 4, and the response signal intensity of the monochromatic light is obtained;

[0054] Step 1.3, adjusting the central wavelength of the monochromatic light output by the electric monochromator 5 with a fixed step length d, keeping the incident intensity of the monochromatic light unchanged, repeating step 1.2 until the central wavelength of the monochromatic light output by the monochromator 5 reaches the maximum working wavelength of the image detector 4, and obtaining the response signal intensity change curve of the image detector 4 of the encoding unit to the incident monochromatic light;

[0055] Step 1.4, change the spectral light signal of the electric monochromator 5, repeat steps 1.1 to 1.3, and obtain the transmittance curve of each coding unit of the deterministic filter structure 3.

[0056] Step 2, divide the coding unit (5*5), and construct the measurement matrix Φi(λ) according to the transmittance curve obtained in step 1;

[0057] Step 2.1, each coding unit of the deterministic filter structure 3 is arranged periodically;

[0058] Step 2.2, according to the transmittance curves of the deterministic filter structure for optical signals of different spectral bands obtained in step 1, the curve values ​​of each channel of the coding unit are sequentially used as each row of the measurement matrix to construct a measurement matrix;

[0059] Step 2.3, repeat step 2.2 to sequentially obtain the measurement matrices corresponding to the coding units of all deterministic filtering structures 3.

[0060] Step 3: Encode the spectral information S(λ,x,y) of the target to be measured received by 5*5 different channels in each coding unit of the deterministic filter structure at the spectral information space (x,y), and the image detector converts the modulated spectral information S(λ,x,y) of the target to be measured into a grayscale signal Yi(x,y). The relationship is as follows:

[0061]

[0062] λmin and λmax are the lower and upper limits of the spectral range, respectively. Φi(λ) is the spectral transmittance curve of the i-th deterministic filter structure, i=1,2,...,25, and D(λ) is the spectral response curve of the image detector. These 25 light intensity values ​​constitute a set of codes of the original spectral signal S(λ,x,y).

[0063] Step 4: According to the spectral information of the target to be measured, S(λ) is sparsely represented, and the optimization equation is obtained by using a greedy algorithm based on compressed sensing:

[0064]

[0065] Wherein, Φi(λ) is the measurement matrix composed of the spectral transmittance of the coding unit, with 25 rows and N columns;

[0066] D(λ) is the matrix obtained by discretizing the spectral response curve of the image detector into N*1;

[0067] Ψ is a sparse base matrix, α is a sparse coefficient, Yi is the electrical signal strength matrix of one of the coding units received by the image detector Yi=(Y 1 , Y 2 ,…,Y n ).

[0068] Step 5: According to the greedy algorithm, the sparse coefficients are solved and the target spectrum S(λ, x, y) of a coding unit is reconstructed by super-resolution.

[0069] Step 6: Repeat steps 2 to 5 to solve the target spectrum S(λ, x, y) for all divided coding units in turn, so as to construct a complete hyperspectral three-dimensional data cube and complete the super-resolution reconstruction of the spectrum.

Claims

1. A spectrally coded imaging system, characterized in that: The invention comprises targets to be measured (1) arranged in sequence along the light transmission direction, an imaging lens (2) for imaging the object light emitted by the targets to be measured (1), a deterministic filter structure (3) for encoding the targets to be measured (1) to obtain a coded measurement value, and an image detector (4) for converting the coded measurement value information into a grayscale signal and then decoding and reconstructing it through a spectral super-resolution reconstruction module, thereby constructing a complete hyperspectral three-dimensional data cube; The deterministic filter structure (3) is composed of a plurality of coding units arranged in a matrix, each coding unit corresponds to a respective image detector (4), and the transmittance curve of each coding unit is obtained by the following method; Step 1.1, adjusting the central wavelength of the monochromatic light output by the electric monochromator (5) to the minimum operating wavelength of the image detector (4); Step 1.2, the monochromatic light passes through the integrating sphere (6) and the collimator (7) and is incident on the deterministic filter structure (3) in the form of parallel light, and the encoded monochromatic light forms a grayscale electrical signal on the surface of the image detector (4), and the response signal intensity of the monochromatic light is obtained; Step 1.3, adjusting the central wavelength of the monochromatic light output by the electric monochromator (5) with a fixed step length d, keeping the incident intensity of the monochromatic light unchanged, repeating step 1.2 until the central wavelength of the monochromatic light output by the monochromator (5) reaches the maximum working wavelength of the image detector (4), and obtaining a response signal intensity change curve of the image detector (4) of the encoding unit to the incident monochromatic light; Step 1.4, changing the spectral light signal of the electric monochromator (5), repeating steps 1.1 to 1.3, and obtaining the transmittance curve of each coding unit of the deterministic filter structure (3).

2. The spectrally coded imaging system according to claim 1, characterized in that: The wavelength encoding characteristics of the encoding unit of the deterministic optical filtering structure (3) have a high throughput greater than or equal to 60%.

3. The spectrally coded imaging system according to claim 1, characterized in that: The correlation of the transmittance curves corresponding to adjacent pixels of each coding unit of the deterministic filter structure 3 is between 0 and 0.

5.

4. A spectral encoding imaging reconstruction method for an imaging system according to any one of claims 1 to 3, characterized in that: Step 1, obtaining the transmittance curve of the deterministic filter structure (3) for light signals in different spectral bands; Step 2: Divide into several coding units (n*n), where n is between 2 and 100. According to the transmittance curve obtained in step 1, construct the measurement matrix Φ i (λ); Step 3: Encode the spectral information S(λ, x, y) of the target to be measured at the spectral information space (x, y) received by n*n different channels in each coding unit of the deterministic filtering structure. The image detector converts the modulated spectral information S(λ, x, y) of the target to be measured into a grayscale signal Yi(x, y), which has the following relationship: λmin and λmax are the lower and upper limits of the spectrum range, respectively. i (λ) is the spectral transmittance curve of the i-th deterministic filter structure, i=1, 2, ..., M, D(λ) is the spectral response curve of the image detector, and these M light intensity values ​​constitute a set of codes of the original spectral signal S(λ, x, y); Step 4: According to the spectral information of the target to be measured, S(λ) is sparsely represented, and the optimization equation is obtained by using a greedy algorithm based on compressed sensing: Among them, Φ i (λ) is the measurement matrix composed of the spectral transmittance of the coding unit, with n rows and N columns; D(λ) is the matrix obtained by discretizing the spectral response curve of the image detector into N*1; Ψ is a sparse basis matrix, α is a sparse coefficient, Yi is the electrical signal strength matrix Y of one of the coding units received by the image detector i =(Y1, Y2, ..., Y n ); Step 5: Solve the sparse coefficients according to the algorithm, and reconstruct the target spectrum S (λ, x, y) of a coding unit by super resolution; Step 6: Repeat steps 2 to 5 to solve the target spectrum S(λ, x, y) for all divided coding units in turn, thereby constructing a complete hyperspectral three-dimensional data cube and completing the super-resolution reconstruction of the spectrum.

5. The spectrally coded imaging reconstruction method according to claim 4, characterized in that: The step 2 is specifically as follows: Step 2.1, each coding unit of the deterministic filtering structure (3) is arranged periodically; Step 2.2, according to the transmittance curves of the deterministic filter structure for optical signals of different spectral bands obtained in step 1, the curve values ​​of each channel of the coding unit are sequentially used as each row of the measurement matrix to construct a measurement matrix; Step 2.3, repeat step 2.2 to sequentially obtain the measurement matrices corresponding to the coding units of all deterministic filtering structures (3).

Images